Implementation of Adaptive Lasso Regression Based on Multiple Theil-Sen Estimators Using Differential Evolution Algorithm with Heavy Tailed Errors

Abstract

The last decade has witnessed that penalized regression methods have become an alternative to classical methods. Adaptive lasso is one type of method in penalized regression and is commonly used in statistical modelling to perform variable selection. Apart from the classical lasso setting, the adaptive lasso requires the coefficient weights inside the target function. The main issue in adaptive lasso is to select the optimal weights in the model since the selected weights have serious impacts on the estimation results. However, there is no compromise for choosing the weights as a universal approach, and they should he chosen properly with the statistical assumptions. When the error terms are heavy tailed; classical estimation (such as least squares) gives poor results in adaptive lasso because of the lacking robustness. This article deals with the selection of optimal weights in the presence of heavy-tailed errors for the adaptive lasso. To solve the distributional problem, we integrated the Theil-Sen estimation (ISE) approach into the adaptive lasso for heavy tailed erroneous cases while choosing the weights. During the selection of the optimal tuning parameters, we employed a differential evolution algorithm (DEA) between a range of lambda values. The simulation studies and real data examples confirm the power of our combination of Theil-Sen estimators and differential evolution algorithm in the presence of heavy tailed errors in the adaptive lasso.