Which method should the Specialist try to improve model performance?

A Machine Learning Specialist deployed a model that provides product recommendations on a company’s website Initially, the model was performing very well and resulted in customers buying more products on average However within the past few months the Specialist has noticed that the effect of product recommendations has diminished and customers are starting to return to their original habits of spending less The Specialist is unsure of what happened, as the model has not changed from its initial deployment over a year ago

Which method should the Specialist try to improve model performance?
A . The model needs to be completely re-engineered because it is unable to handle product inventory changes
B . The model’s hyperparameters should be periodically updated to prevent drift
C . The model should be periodically retrained from scratch using the original data while adding a regularization term to handle product inventory changes
D . The model should be periodically retrained using the original training data plus new data as product inventory changes

View Answer

Answer: D

Explanation:

The problem that the Machine Learning Specialist is facing is likely due to concept drift, which is a phenomenon where the statistical properties of the target variable change over time, making the model less accurate and relevant. Concept drift can occur due to various reasons, such as changes in customer preferences, market trends, product inventory, seasonality, etc. In this case, the product recommendations model may have become outdated as the product inventory changed over time, making the recommendations less appealing to the customers. To address this issue, the model should be periodically retrained using the original training data plus new data as product inventory changes. This way, the model can learn from the latest data and adapt to the changing customer behavior and preferences. Retraining the model from scratch using the original data while adding a regularization term may not be sufficient, as it does not account for the new data. Updating the model’s hyperparameters may not help either, as it does not address the underlying data distribution change. Re-engineering the model completely may not be necessary, as the model may still be valid and useful with periodic retraining.

Reference: Concept Drift – Amazon SageMaker

Detecting and Handling Concept Drift – Amazon SageMaker Machine Learning Concepts – Amazon Machine Learning