REMEDI: Relative Feature Enhanced Meta-Learning with Distillation for Imbalanced Prediction

Predicting future vehicle purchases among existing owners presents a critical challenge due to extreme class imbalance (<0.5% positive rate) and complex behavioral patterns. We propose REMEDI (Relative feature Enhanced Meta-learning with Distillation for Imbalanced prediction), a novel multi-stage framework addressing these challenges. REMEDI first trains diverse base models to capture complementary aspects of user behavior. Second, inspired by comparative op-timization techniques, we introduce relative performance meta-features (deviation from ensemble mean, rank among peers) for effective model fusion through a hybrid-expert architecture. Third, we distill the ensemble's knowledge into a single efficient model via supervised fine-tuning with MSE loss, enabling practical deployment. Evaluated on approximately 800,000 vehicle owners, REMEDI significantly outperforms baseline approaches, achieving the business target of identifying ~50% of actual buyers within the top 60,000 recommendations at ~10% precision. The distilled model preserves the ensemble's predictive power while maintaining deployment efficiency, demonstrating REMEDI's effectiveness for imbalanced prediction in industry settings.

@article{liu2025_2505.07245,
  title={ REMEDI: Relative Feature Enhanced Meta-Learning with Distillation for Imbalanced Prediction },
  author={ Fei Liu and Huanhuan Ren and Yu Guan and Xiuxu Wang and Wang Lv and Zhiqiang Hu and Yaxi Chen },
  journal={arXiv preprint arXiv:2505.07245},
  year={ 2025 }
}