Predictive Accuracy Index in evaluating the dataset shift (case study)
[ 1 ] Wydział Techniczny, Akademia im. Jakuba z Paradyża | [ P ] pracownik
2023
artykuł naukowy / referat
angielski
- dataset shift
- Univaria Predictive Accuracy Index
- Multivariate Predictive Accuracy Index
- monitoring of predictive model
EN A dataset shift takes place in a situation, where the joint distribution of inputs and outputs differs between the stages of training, testing, and using predictive models. If the distribution of current data for the implemented forecasting model changed significantly compared to the distribution of data used to develop it, them it could lead to its incorrect operation. The aim of the study was to compare the properties of two indicators, the Univariate Predictive Accuracy Index (UPAI) and Multivariate Predictive Accuracy Index (MPAI). The research procedure was carried out in 2 scenarios. The first involved a comparison of UPAI and MPAI for the distributions of categorical variables, while the second for distributions of continuous variables. The obtained MPAI values for both scenarios were summarized and compared with the UPAI and PSI values calculated and published in our previous article [1]. The results of the experiment proved that basing the decision on the need to calibrate the model or build a completely new model on the basis of MPAI, a measure that takes into account the multivariate distribution of variables, is superior to one-dimensional measures
3342 - 3351
CC BY-NC-ND (uznanie autorstwa - użycie niekomercyjne - bez utworów zależnych)
otwarte czasopismo
ostateczna wersja opublikowana
12.2023 (Data domniemana)
w momencie opublikowania
5,0
70,0
