Which technique should you use?

You are building a linear model with over 100 input features, all with values between -1 and 1. You suspect that many features are non-informative. You want to remove the non-informative features from your model while keeping the informative ones in their original form.

Which technique should you use?
A . Use Principal Component Analysis to eliminate the least informative features.
B . Use L1 regularization to reduce the coefficients of uninformative features to 0.
C . After building your model, use Shapley values to determine which features are the most informative.
D . Use an iterative dropout technique to identify which features do not degrade the model when removed.

View Answer

Answer: B

Explanation:

L1 regularization, also known as Lasso regularization, adds the sum of the absolute values of the model’s coefficients to the loss function1. It encourages sparsity in the model by shrinking some coefficients to precisely zero2. This way, L1 regularization can perform feature selection and remove the non-informative features from the model while keeping the informative ones in their original form. Therefore, using L1 regularization is the best technique for this use case.

Reference: Regularization in Machine Learning – GeeksforGeeks

Regularization in Machine Learning (with Code Examples) – Dataquest L1 And L2 Regularization Explained & Practical How To Examples L1 and L2 as Regularization for a Linear Model