Reliability of AI Systems
Ideally, supermarket profits increase during the Christmas time because the number of customers in supermarkets increases as does the number of products sold. When we train the predictive model based on the historical data, the model expects profit to increase during the Christmas season. However, this could be misleading or even wrong in the real world because it is subject to various external factors in 2021 such as the number of COVID cases, government regulations, etc. So in this example, the question of how much we can rely on an AI system remains rather unanswered.
Already in 2020, the models did not provide good forecasts , because it was the first year of the pandemic. Everything that happened to sales that year was different from anything previously observed by ML models. Nevertheless, this was not unexpected for Christmas 2021. In the third year of the pandemic, most models already »know« the situation and are prepared. They have initial historical data to recognize and learn from a »Covid19 pattern.« However, there will always be situations, such as unexpected lockdowns, shutdowns, infection rates, etc., that make forecasting difficult.