SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents

SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents

26 May 2025

Ibragim Badertdinov

Alexander Golubev

Maksim Nekrashevich

Andrei Andriushchenko

Maria Trofimova

Daria Litvintseva

ArXiv (abs)PDF HTML

Papers citing "SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents"

12 / 12 papers shown

Title
SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling Haoran Wang Zhenyu Hou Yao Wei J. Tang Yuxiao Dong LLMAG 11 1 0 09 Jun 2025
Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute Yingwei Ma Binhua Li Yihong Dong Xue Jiang Rongyu Cao Jingshu Chen Fei Huang Yongqian Li LLMAG LRM 117 6 0 31 Mar 2025
DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal Vaibhav Aggarwal Ojasv Kamal Abhinav Japesh Zhijing Jin Bernhard Schölkopf 90 3 0 18 Mar 2025
RepoST: Scalable Repository-Level Coding Environment Construction with Sandbox Testing Yiqing Xie Alex Xie Divyanshu Sheth Pengfei Liu Daniel Fried Carolyn Rose LRM 121 3 0 10 Mar 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 380 1,970 0 22 Jan 2025
Training Software Engineering Agents and Verifiers with SWE-Gym Jiayi Pan Xingyao Wang Graham Neubig Navdeep Jaitly Chenhui Xu Alane Suhr Yizhe Zhang 131 36 0 30 Dec 2024
SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement Antonis Antoniades Albert Örwall Kexun Zhang Yuxi Xie Anirudh Goyal William Yang Wang LLMAG 138 21 0 26 Oct 2024
From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future Haolin Jin Linghan Huang Haipeng Cai Jun Yan Bo Li Huaming Chen 149 37 0 05 Aug 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 150 1,287 0 05 Feb 2024
WizardCoder: Empowering Code Large Language Models with Evol-Instruct Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin Daxin Jiang ELM SyDa ALM 113 687 0 14 Jun 2023
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 216 2,004 0 16 Aug 2021
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He Basel Alomair Jacob Steinhardt ELM AIMat ALM 259 703 0 20 May 2021