Detection of Rare Events: The Need to Know the Customer

Abstract

The prediction of customer complaints based on a time series of invoices is a two-stage process consisting of determining anomalies in the sequence of invoices and assessing the response of the customers to these anomalies. In the telecommunication sector, the average complaint rate is approximately 10?? hence the prediction of customer complaints falls in the realm of rare event detection. Detecting rare events poses a significant challenge when working with unbalanced datasets. In machine learning applications, oversampling of the minority class and under sampling of the majority class in the training set are well-known preprocessing tools for creating a more balanced set. In previous work, [14] we proposed a cluster based under sampling approach as an alternative to random under sampling of the majority class, based on splitting heterogeneous data into homogeneous subsets, using Principal Component Analysis, to reduce variability within clusters. In the present work we propose a method for assessing the response of the customers to anomalies detected in the time series of invoices.