From Research to Practice: Operational Deployment of Temporal Drift Detection in Financial Fraud Prevention

Abstract

Temporal drift detection is well established in research, but fraud teams that deploy it face practical questions that research benchmarks rarely answer: how often the tests will raise false alarms, how large a shift they can see, whether they detect changes in fraud itself, and how quickly they can respond when confirmed labels arrive late. This study takes the temporal-drift framework of Zuberi et al. [1] Population Stability Index (PSI), Kolmogorov–Smirnov (KS) and Jensen–Shannon divergence (JSD) tests on monitored features, combined with prevalence-shift z-tests and evaluates it as an operational system on a simulated mobile-money transaction stream. Four controlled experiments examined false-alarm rates on stable data, detection power for population shifts, sensitivity to drift confined to fraudulent transactions, and supervised detection delay under label delays of 0–28 days. KS significance tests were unsuitable as stand-alone alarms: under ordinary week-to-week variation, the share of monitoring windows with a KS p < 0.001 alarm rose from 12% to 95% as window size grew from 1,000 to 80,000 transactions. PSI thresholds and the combined rule of Zuberi et al. produced almost no false alarms, but detected a shift in transaction amount only once it approached half a standard deviation. Feature-level tests missed a new fraud pattern until fraudulent transactions reached about 20% of traffic, whereas the prevalence z-test detected an accompanying rise in fraud rate in 81% of weeks at 1% prevalence. Supervised detection delay was dominated by label delay: a sequential Page–Hinkley test reached detection one day after labels arrived, against five days for a weekly test, but raised a false alarm in 41% of 120-day stable periods against 5%. The results translate into a deployment lifecycle — calibration, shadow mode, tiered response and periodic review — and into operating rules that treat feature drift as information, prevalence and performance shifts as triggers, and label latency as the main lever for faster response.