ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2506.01531
81
1
v1v2 (latest)

STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent Framework

2 June 2025
Wenhao Liu
Zhenyi Lu
Xinyu Hu
Jierui Zhang
Dailin Li
Jiacheng Cen
Huilin Cao
Haiteng Wang
Yuhan Li
Kun Xie
Dandan Li
Pei Zhang
Chengbo Zhang
Yuxiang Ren
Xiaohong Huang
Yan Ma
    OffRL
ArXiv (abs)PDFHTML
Main:8 Pages
3 Figures
Bibliography:1 Pages
4 Tables
Appendix:12 Pages
Abstract

High-quality math datasets are crucial for advancing the reasoning abilities of large language models (LLMs). However, existing datasets often suffer from three key issues: outdated and insufficient challenging content, neglecting human-like reasoning, and limited reliability due to single-LLM generation. To address these, we introduce STORM-BORN\textbf{STORM-BORN}STORM-BORN, an ultra-challenging dataset of mathematical derivations sourced from cutting-edge academic papers, which includes dense human-like approximations and heuristic cues. To ensure the reliability and quality, we propose a novel human-in-the-loop, multi-agent data generation framework, integrating reasoning-dense filters, multi-agent collaboration, and human mathematicians' evaluations. We curated a set of 2,000 synthetic samples and deliberately selected the 100 most difficult problems. Even most advanced models like GPT-o1 solved fewer than 5%5\%5% of them. Fine-tuning on STORM-BORN boosts accuracy by 7.84%7.84\%7.84% (LLaMA3-8B) and 9.12%9.12\%9.12% (Qwen2.5-7B). As AI approaches mathematician-level reasoning, STORM-BORN provides both a high-difficulty benchmark and a human-like reasoning training resource. Our code and dataset are publicly available at this https URL.

View on arXiv
@article{liu2025_2506.01531,
  title={ STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent Framework },
  author={ Wenhao Liu and Zhenyi Lu and Xinyu Hu and Jierui Zhang and Dailin Li and Jiacheng Cen and Huilin Cao and Haiteng Wang and Yuhan Li and Kun Xie and Dandan Li and Pei Zhang and Chengbo Zhang and Yuxiang Ren and Xiaohong Huang and Yan Ma },
  journal={arXiv preprint arXiv:2506.01531},
  year={ 2025 }
}
Comments on this paper