000K utf8 1100 2021$c2021-11-26 1500 eng 2050 urn:nbn:de:hbz:465-20220727-135305-6 2051 10.1016/j.mlwa.2021.100220 3000 Ameyaw, Daniel Adofo 3010 Deng, Qi 3010 Söffker, Dirk 4000 How to evaluate classifier performance in the presence of additional effects$dA new POD-based approach allowing certification of machine learning approaches [Ameyaw, Daniel Adofo] 4209 Classifiers are useful and well-known learning algorithms classifications. A classifier may be suited for a specific task depending on the application and datasets. To select an approach for a task, performance evaluation may be imperative. Existing approaches like the receiver operating characteristic and precision–recall curves are popular in evaluating classifier performance, however both measures do not directly address the influence of additional and possibly unknown (process) parameters on the classification results. In this contribution, this limitation is discussed and addressed by adapting the Probability of Detection (POD) measure. The POD is aprobabilistic method to quantify the reliability of a diagnostic procedure taking into account statistical variability of sensor and measurements properties. In this contribution the POD approach is adapted and extended. The introduced approach is implemented on driving behavior prediction data serving as illustrative example. Based on the introduced POD-related evaluation, different classifiers can be clearly distinguished with respect to their ability to predict the correct intended driver behavior as a function of remaining time (here assumed as process parameter) before the event itself. The introduced approach provides a new diagnostic and comprehensive interpretation of the quality of a classification model. 4950 https://doi.org/10.1016/j.mlwa.2021.100220$xR$3Volltext$534 4950 https://nbn-resolving.org/urn:nbn:de:hbz:465-20220727-135305-6$xR$3Volltext$534 4961 https://duepublico2.uni-due.de/receive/duepublico_mods_00076337 5051 620 5550 Performance evaluation Classifier Probability of Detection Human behavior prediction