**Building a Retention System with Machine Learning: A Practical Guide for Fintech**
High customer acquisition costs make user retention a top priority in digital banking and fintech. Retaining existing users is significantly more cost-effective than acquiring new ones, especially when acquisition expenses are substantial. Retention success depends on multiple factors, including service quality, product functionality, loyalty programs, and overall user experience.
This article explores a practical approach using machine learning (ML) to retain at-risk users in a fintech debit card product. The focus is on two ML models:
1. A **pre-churn model** that identifies users likely to stop using the product.
2. An **uplift model** that determines which users in the at-risk segment will actually respond to retention offers.
—
### **Preparation: Defining Churn and Features**
Churn was defined as having no card payments for 30 days, a common benchmark in financial products. The target variable was binary: whether a user would make at least one payment in the next 30 days.
Key feature categories included:
– **User profile data**: age, region, device, product portfolio, tariff plan, loyalty status.
– **Behavioral aggregates**: transaction counts, turnover, and averages over 7, 30, and 90-day windows.
– **Timing patterns**: time since last payment, average gaps between transactions, and activity rhythm.
– **Calendar features**: day of week and month to account for seasonal effects.
– **Derived metrics**: ratios and normalized indicators to compare users fairly.
—
### **Pre-churn Model: Identifying At-Risk Users**
The goal was to estimate the probability of a user making a payment in the next 30 days. A low predicted probability indicated high churn risk.
**Modeling approach**:
– Gradient boosting models (XGBoost, LightGBM, CatBoost) were used for binary classification.
– Users were ranked by risk based on ( risk_score(x) = 1 – p_base(x) ).
– A threshold determined which users entered the pre-churn segment.
**Validation and calibration**:
– Recent data was held out for testing to simulate real-world performance.
– Ranking quality was measured using ROC-AUC.
– Calibration was monitored because probability estimates tended to drift over time due to unobserved factors.
– A logistic regression calibration model was added to correct this drift without redesigning the system.
—
### **Offer for the Pre-churn Segment: Initial Experiment Results**
A randomized experiment tested the impact of increased cashback offers. While retention improved compared to the control group, the cost per retained user exceeded customer acquisition cost (CAC). This happened because the pre-churn segment included:
– Users who would stay regardless (organic users).
– Users who would not respond to any offer.
This highlighted a key insight: knowing that a user is at risk is not enough. It is also necessary to identify who will respond to the intervention.
—
### **Uplift Model: Targeting the Right Users**
The uplift model estimates the causal effect of an offer on user behavior:
[
τ(x) = P(Y = 1 | X = x, T = 1) – P(Y = 1 | X = x, T = 0)
]
where:
– ( Y ) indicates whether a payment was made.
– ( T ) indicates whether the user received the offer.
– ( τ(x) ) represents the incremental impact of the offer for user ( x ).
Two modeling strategies were tested:
– **T-learner**: Two separate models for treated and control groups.
– **S-learner**: A single model with an intervention flag.
Both approaches yielded similar results. Users with the highest predicted uplift were targeted for offers.
**Validation results**:
– Uplift-based targeting reduced the share of offers sent to users who would have paid anyway or not at all.
– Incremental retention per 1,000 offers increased by 66%.
– Cost per retained user dropped by 28% compared to standard targeting.
—
### **Customer Retention System Workflow**
The final system operated in two sequential steps:
1. **Pre-churn filtering**: Users with a calibrated probability below a threshold were flagged as at-risk.
2. **Uplift targeting**: Within this segment, only users with high predicted uplift received offers.
The system incorporated continuous learning:
– A reference group excluded from retention offers measured baseline churn.
– Random offers within the pre-churn segment provided clean data for recalibrating both models.
—
### **FAQ**
**What is churn in this context?**
Churn is defined as no card payments for 30 days, indicating the user is no longer actively using the product.
**Why use two models instead of one?**
The pre-churn model identifies at-risk users, while the uplift model ensures offers are only sent to users who are likely to change their behavior because of them. This improves cost efficiency.
**How is model calibration handled?**
A logistic regression calibration model is applied periodically to correct probability drift and maintain reliable risk scores.
**What are the benefits of uplift modeling?**
Uplift modeling reduces wasteful spending by avoiding users who would stay naturally and by excluding those who cannot be influenced by offers.
**Is this system scalable?**
Yes, the models are designed for continuous operation with regular data updates, making them suitable for production environments with large user bases.
—
### **Conclusion**
The ML-based retention system successfully combines predictive risk modeling with causal impact estimation. By identifying at-risk users and targeting only those likely to respond to offers, the system achieves higher retention at a lower cost. This approach not only improves financial efficiency but also enhances the effectiveness of marketing efforts, ensuring that retention resources are focused where they matter most.
