9. What is regularization in machine learning and what are its methods?

Regularization is a technique used in machine learning to prevent overfitting of the model to the training data. It involves adding a penalty for the model's complexity to the loss function, forcing the model to simplify.

Main regularization methods:

1. L1 Regularization (Lasso): Adds a penalty equal to the sum of the absolute values of the model's weights.

from sklearn.linear_model import Lasso
model = Lasso(alpha=0.1)

2. L2 Regularization (Ridge): Adds a penalty equal to the sum of the squares of the model's weights.

from sklearn.linear_model import Ridge
model = Ridge(alpha=0.1)

3. Elastic Net: A combination of L1 and L2 regularization.

from sklearn.linear_model import ElasticNet
model = ElasticNet(alpha=0.1, l1_ratio=0.5)

4. Dropout: A technique used in neural networks, randomly dropping neurons during training.

from tensorflow.keras.layers import Dropout
model.add(Dropout(0.5))

5. Early Stopping: Stopping the training of the model when its performance on validation data stops improving.

Advantages of regularization:

• Prevents overfitting.
• Improves the model's generalization ability.
• Simplifies the model.

Regularization is a crucial component in the machine learning process, allowing for the creation of more efficient and general models.