Weak signalsaid plainly
250,000 UPI transactions, with fraud a small minority of them. I handled the class imbalance, evaluated carefully, and found the available features carried weak fraud signal. I reported that as it was instead of dressing up the numbers.
Read this firstNot every project ends in a high score. Knowing when the data can't support a claim is part of the job.
Transactions to an honest finding
Fraud is rare, so most dots are ordinary payments. The output here is a finding, not a model in production. Click a stage to read what happens there.
A model that catches nothing can still score 99%
When fraud is rare, a model that calls every transaction legitimate looks excellent on accuracy and is useless. That's why imbalance has to be handled first, and accuracy can't be the yardstick. Move the fraud rate and watch.
The lessonScore fraud models on what they catch and what they wrongly flag, never on accuracy alone.
DraftAdd the real fraud rate in the data and how you rebalanced it (resampling, class weights or both).
When the two groups overlap, no threshold saves you
A model gives each transaction a score. If the features separate fraud from normal behaviour, the two curves pull apart. If they don't, the curves sit on top of each other: every threshold that catches more fraud also flags more honest users. No tuning fixes that; it needs better features.
Try thisDrag signal strength to zero. Fraud caught and honest users flagged move together, whatever the threshold.
Honest readingSpotting this early saves weeks of tuning a model that can't improve.
Six stages, ending in a finding
The same six stages as the homepage. Here the last one isn't a deployed model. It's a clear statement of what the data could and couldn't support, and what to try next.
Why it mattersA team can plan around "this data won't do it, here's what would". It can't plan around an inflated score.