Depending on the amount of data to process, file generation may take longer.

If it takes too long to generate, you can limit the data by, for example, reducing the range of years.

Article

Download BibTeX

Title

Predictive Accuracy Index in evaluating the dataset shift (case study)

Authors

[ 1 ] Wydział Techniczny, Akademia im. Jakuba z Paradyża | [ P ] employee

Scientific discipline (Law 2.0)

[2.3] Information and communication technology

Year of publication

2023

Published in

Procedia Computer Science

Journal year: 2023 | Journal volume: 225

Article type

scientific article / paper

Publication language

english

Keywords
EN
  • dataset shift
  • Univaria Predictive Accuracy Index
  • Multivariate Predictive Accuracy Index
  • monitoring of predictive model
Abstract

EN A dataset shift takes place in a situation, where the joint distribution of inputs and outputs differs between the stages of training, testing, and using predictive models. If the distribution of current data for the implemented forecasting model changed significantly compared to the distribution of data used to develop it, them it could lead to its incorrect operation. The aim of the study was to compare the properties of two indicators, the Univariate Predictive Accuracy Index (UPAI) and Multivariate Predictive Accuracy Index (MPAI). The research procedure was carried out in 2 scenarios. The first involved a comparison of UPAI and MPAI for the distributions of categorical variables, while the second for distributions of continuous variables. The obtained MPAI values for both scenarios were summarized and compared with the UPAI and PSI values calculated and published in our previous article [1]. The results of the experiment proved that basing the decision on the need to calibrate the model or build a completely new model on the basis of MPAI, a measure that takes into account the multivariate distribution of variables, is superior to one-dimensional measures

Pages (from - to)

3342 - 3351

DOI

10.1016/j.procs.2023.10.328

URL

https://www.sciencedirect.com/science/article/pii/S1877050923014862?via%3Dihub

Presented on

27th International Conference on Knowledge Based and Intelligent Information and Engineering Sytems (KES 2023), 6-8.09.2023, Athens, Grecja

License type

CC BY-NC-ND (attribution - noncommercial - no derivatives)

Open Access Mode

open journal

Open Access Text Version

final published version

Release date

12.2023 (Date presumed)

Date of Open Access to the publication

at the time of publication

Ministry points / journal

5.0

Ministry points / conference (CORE)

70.0