What issue is most likely causing the steady decline in model accuracy?

You built and manage a production system that is responsible for predicting sales numbers. Model accuracy is crucial, because the production model is required to keep up with market changes. Since being deployed to production, the model hasn’t changed; however the accuracy of the model has steadily deteriorated.

What issue is most likely causing the steady decline in model accuracy?
A . Poor data quality
B . Lack of model retraining
C . Too few layers in the model for capturing information
D . Incorrect data split ratio during model training, evaluation, validation, and test

Answer: B

Explanation:

Model retraining is the process of updating an existing machine learning model with new data and parameters to improve its performance and accuracy. Model retraining is essential for maintaining the relevance and validity of the model, especially when the data or the environment changes over time. Model retraining can help to avoid or reduce the effects of model degradation, which is the phenomenon of the model’s predictive performance decreasing as it is tested on new datasets within rapidly evolving environments1.

For the use case of predicting sales numbers, model accuracy is crucial, because the production model is required to keep up with market changes. Market changes can affect the demand, supply, price, and preference of the products, and thus influence the sales numbers. If the model is not retrained with new data that reflects the market changes, it may become outdated and inaccurate, and fail to capture the patterns and trends of the sales numbers. Therefore, the most likely issue that is causing the steady decline in model accuracy is the lack of model retraining.

The other options are not as likely as option B, because they are not directly related to the model’s ability to adapt to market changes.

Option A, poor data quality, may affect the model’s accuracy, but it is not a specific cause of model degradation over time.

Option C, too few layers in the model for capturing information, may affect the model’s complexity and expressiveness, but it is not a specific cause of model degradation over time.

Option D, incorrect data split ratio during model training, evaluation, validation, and test, may affect the model’s generalization and validation, but it is not a specific cause of model degradation over time. Therefore, option B, lack of model retraining, is the best answer for this question.

Reference: Beware Steep Decline: Understanding Model Degradation In Machine Learning Models