A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learning

25 May 2025

Papers citing "A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learning"

50 / 55 papers shown

Title
Reinforcement Learning for Reasoning in Large Language Models with One Training Example Yiping Wang Qing Yang Zhiyuan Zeng Liliang Ren Liu Liu ... Jianfeng Gao Weizhu Chen Shuaiqiang Wang Simon Shaolei Du Yelong Shen OffRL ReLM LRM 299 47 0 29 Apr 2025
Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning Yixuan Even Xu Yash Savani Fei Fang Zico Kolter OffRL 94 12 0 18 Apr 2025
Efficient Reinforcement Finetuning via Adaptive Curriculum Learning Taiwei Shi Yiyang Wu Linxin Song Dinesh Manocha Jieyu Zhao LRM 153 15 0 07 Apr 2025
Behaviour Discovery and Attribution for Explainable Reinforcement Learning Rishav Rishav Somjit Nath Vincent Michalski Samira Ebrahimi Kahou FAtt OffRL 157 1 0 19 Mar 2025
DAPO: An Open-Source LLM Reinforcement Learning System at Scale Qiying Yu Zheng Zhang Ruofei Zhu Yufeng Yuan Xiaochen Zuo ... Ya Zhang Lin Yan Mu Qiao Yonghui Wu Mingxuan Wang OffRL LRM 206 218 0 18 Mar 2025
A Survey on Explainable Deep Reinforcement Learning Zelei Cheng Jiahao Yu Masashi Sugiyama OffRL 66 4 0 08 Feb 2025
Reviving The Classics: Active Reward Modeling in Large Language Model Alignment Yunyi Shen Hao Sun Jean-Francois Ton 41 3 0 04 Feb 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 380 2,000 0 22 Jan 2025
Most Influential Subset Selection: Challenges, Promises, and Beyond Yuzheng Hu Pingbang Hu Han Zhao Jiaqi W. Ma TDI 200 9 0 10 Jan 2025
Capturing the Temporal Dependence of Training Data Influence Jiachen T. Wang Dawn Song James Zou Prateek Mittal Ruoxi Jia TDI AI4TS 93 4 0 12 Dec 2024
Scalable Influence and Fact Tracing for Large Language Model Pretraining Tyler A. Chang Dheeraj Rajagopal Tolga Bolukbasi Lucas Dixon Ian Tenney TDI 81 5 0 22 Oct 2024
Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models Can Demircan Tankred Saanum Akshay K. Jagadish Marcel Binz Eric Schulz 53 4 0 02 Oct 2024
Data Shapley in One Training Run Jiachen T. Wang Prateek Mittal Dawn Song R. Jia TDI 111 15 0 16 Jun 2024
Token-wise Influential Training Data Retrieval for Large Language Models Huawei Lin Jikai Long Zhaozhuo Xu Weijie Zhao 76 7 0 20 May 2024
RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation Zelei Cheng Xian Wu Jiahao Yu Sabrina Yang Gang Wang Xinyu Xing OffRL 74 5 0 05 May 2024
Active Preference Optimization for Sample Efficient RLHF Nirjhar Das Souradip Chakraborty Aldo Pacchiano Sayak Ray Chowdhury 109 22 0 16 Feb 2024
Active Preference Learning for Large Language Models William Muldrew Peter Hayes Mingtian Zhang David Barber 80 24 0 12 Feb 2024
LESS: Selecting Influential Data for Targeted Instruction Tuning Mengzhou Xia Sadhika Malladi Suchin Gururangan Sanjeev Arora Danqi Chen 155 245 0 06 Feb 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 167 1,288 0 05 Feb 2024
Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation Tong Xie Haoyu Li Andrew Bai Cho-Jui Hsieh TDI 96 4 0 17 Jan 2024
Intriguing Properties of Data Attribution on Diffusion Models Xiaosen Zheng Tianyu Pang Chao Du Jing Jiang Min Lin TDI 111 26 1 01 Nov 2023
Learning to Identify Critical States for Reinforcement Learning from Videos Haozhe Liu Mingchen Zhuge Bing Li Yu‐Han Wang Francesco Faccio Guohao Li Jürgen Schmidhuber OffRL 65 9 0 15 Aug 2023
Studying Large Language Model Generalization with Influence Functions Roger C. Grosse Juhan Bae Cem Anil Nelson Elhage Alex Tamkin ... Karina Nguyen Nicholas Joseph Sam McCandlish Jared Kaplan Sam Bowman TDI 67 187 0 07 Aug 2023
Resetting the Optimizer in Deep RL: An Empirical Study Kavosh Asadi Rasool Fakoor Shoham Sabach ODL 73 25 0 30 Jun 2023
Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks Maxime Chevalier-Boisvert Bolun Dai Mark Towers Rodrigo de Lazcano Lucas Willems Salem Lahlou Suman Pal Pablo Samuel Castro Jordan Terry VGen 100 212 0 24 Jun 2023
Evaluating Data Attribution for Text-to-Image Models Sheng-Yu Wang Alexei A. Efros Jun-Yan Zhu Richard Y. Zhang TDI 78 33 0 15 Jun 2023
Explaining RL Decisions with Trajectories Shripad Deshmukh Arpan Dasgupta Balaji Krishnamurthy Nan Jiang Chirag Agarwal Georgios Theocharous J. Subramanian OffRL 50 5 0 06 May 2023
TRAK: Attributing Model Behavior at Scale Sung Min Park Kristian Georgiev Andrew Ilyas Guillaume Leclerc Aleksander Madry TDI 113 156 0 24 Mar 2023
Training Data Influence Analysis and Estimation: A Survey Zayd Hammoudeh Daniel Lowd TDI 107 99 0 09 Dec 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 888 13,207 0 04 Mar 2022
FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributes Haonan Wang Ziwei Wu Jingrui He 50 12 0 15 Jan 2022
Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methods Nicholay Topin Stephanie Milani Fei Fang Manuela Veloso OffRL 56 36 0 25 Feb 2021
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection Bertie Vidgen Tristan Thrush Zeerak Talat Douwe Kiela 136 273 0 31 Dec 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 168 1,221 0 24 Sep 2020
Estimating Training Data Influence by Tracing Gradient Descent G. Pruthi Frederick Liu Mukund Sundararajan Satyen Kale TDI 105 419 0 19 Feb 2020
Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution Nikaash Puri Sukriti Verma Piyush B. Gupta Dhruv Kayastha Shripad Deshmukh Balaji Krishnamurthy Sameer Singh FAtt AAML 52 79 0 23 Dec 2019
Dota 2 with Large Scale Deep Reinforcement Learning OpenAI OpenAI : Christopher Berner Greg Brockman Brooke Chan ... Szymon Sidor Ilya Sutskever Jie Tang Filip Wolski Susan Zhang GNN VLM CLL AI4CE LRM 169 1,838 0 13 Dec 2019
Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning Akanksha Atrey Kaleigh Clary David D. Jensen FAtt LRM 66 91 0 09 Dec 2019
Data Cleansing for Models Trained with SGD Satoshi Hara Atsushi Nitanda Takanori Maehara TDI 121 74 0 20 Jun 2019
Towards Interpretable Reinforcement Learning Using Attention Augmented Agents Alex Mott Daniel Zoran Mike Chrzanowski Daan Wierstra Danilo Jimenez Rezende 66 191 0 06 Jun 2019
Challenges of Real-World Reinforcement Learning Gabriel Dulac-Arnold D. Mankowitz Todd Hester OffRL 93 551 0 29 Apr 2019
Data Shapley: Equitable Valuation of Data for Machine Learning Amirata Ghorbani James Zou TDI FedML 85 791 0 05 Apr 2019
Transparency and Explanation in Deep Reinforcement Learning Neural Networks R. Iyer Yuezhang Li Huao Li M. Lewis R. Sundar Katia Sycara 49 175 0 17 Sep 2018
Learning Dexterous In-Hand Manipulation OpenAI OpenAI Marcin Andrychowicz Bowen Baker Maciek Chociej Rafal Jozefowicz ... Szymon Sidor Joshua Tobin Peter Welinder Lilian Weng Wojciech Zaremba 166 1,884 0 01 Aug 2018
Programmatically Interpretable Reinforcement Learning Abhinav Verma V. Murali Rishabh Singh Pushmeet Kohli Swarat Chaudhuri 130 355 0 06 Apr 2018
Visualizing and Understanding Atari Agents S. Greydanus Anurag Koul Jonathan Dodge Alan Fern FAtt 114 348 0 31 Oct 2017
Deep Reinforcement Learning that Matters Peter Henderson Riashat Islam Philip Bachman Joelle Pineau Doina Precup David Meger OffRL 132 1,963 0 19 Sep 2017
A Brief Survey of Deep Reinforcement Learning Kai Arulkumaran M. Deisenroth Miles Brundage Anil Anthony Bharath OffRL 133 2,830 0 19 Aug 2017
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 547 19,296 0 20 Jul 2017
Deep Reinforcement Learning framework for Autonomous Driving Ahmad El-Sallab Mohammed Abdou E. Perot S. Yogamani 102 972 0 08 Apr 2017