This thesis applies modern shrinkage methods in the context of discrete choice models and time series models, identifies shortcomings of existing estimators, and proposes methods for their improvement. Chapter 2 deals with a special case of the LASSO regression and elastic net regression in a discrete choice framework which models unobserved heterogeneity with a nonparametric approach. Building on the estimators of Chapter 2, Chapter 3 considers a random elastic net estimator. Chapter 4 estimates flexible forms of observed heterogeneity in discrete choice models using neural networks. To this end, an influence function approach is applied together with ℓ2-regularization, which shrinks the weights of the neural network towards zero. Chapter 5 applies a LASSO-Type GMM estimator to select valid and relevant moment conditions in a SVAR model in a data-driven way. This thesis also reveals that there are open questions with respect to the studied approaches to be solved in the future. For example, Chapter 2 relates the FKRB estimator to the LASSO estimator, implying that it might be challenging to construct a valid inference procedure for this estimator and its extensions. Neither Chapter 2 nor Chapter 3 develop an inference procedure for the proposed estimators, which, however, would be important for applications. Furthermore, the estimator presented in Chapter 2 does not allow for high-dimensional random coefficients. In principle, the estimator developed in Chapter 3 can deal with high-dimensional random coefficients. However, in this case it would be interesting to explore ways to reduce computation time when including a larger number of random coefficients. The estimation of the neural network in Chapter 4 involves many tuning parameters. Further simulations could help to guide the choice of those tuning parameters and shed light on the robustness of the applied influence function method. In theory, the LASSO-type GMM estimator in Chapter 5 is not distorted by invalid moment conditions. Further simulations could illustrate this property. Additionally, developing a shrinkage estimator which can identify potential zero restrictions on the interactions of the shocks of the SVAR from the data could complement and further justify the restrictions imposed by ecomonic theory.