v1v2 (latest)

Fine-Tuning Language Models from Human Preferences

18 September 2019

Papers citing "Fine-Tuning Language Models from Human Preferences"

50 / 1,265 papers shown

Title
Fine-Tuning Language Models with Advantage-Induced Policy Alignment Banghua Zhu Hiteshi Sharma Felipe Vieira Frujeri Shi Dong Chenguang Zhu Michael I. Jordan Jiantao Jiao OSLM 80 41 0 04 Jun 2023
On Optimal Caching and Model Multiplexing for Large Model Inference Banghua Zhu Ying Sheng Lianmin Zheng Clark W. Barrett Michael I. Jordan Jiantao Jiao 90 21 0 03 Jun 2023
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Zeqiu Wu Yushi Hu Weijia Shi Nouha Dziri Alane Suhr Prithviraj Ammanabrolu Noah A. Smith Mari Ostendorf Hannaneh Hajishirzi ALM 164 335 0 02 Jun 2023
ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages Andrew Jesson Chris Xiaoxuan Lu Gunshi Gupta Angelos Filos Jakob N. Foerster Y. Gal OffRL 77 8 0 02 Jun 2023
ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity? Michael Heck Nurul Lubis Benjamin Ruppik Renato Vukovic Shutong Feng Christian Geishauser Hsien-chin Lin Carel van Niekerk Milica Gavsić 127 47 0 02 Jun 2023
Preference-grounded Token-level Guidance for Language Model Fine-tuning Shentao Yang Shujian Zhang Congying Xia Yihao Feng Caiming Xiong Mi Zhou 146 28 0 01 Jun 2023
CFL: Causally Fair Language Models Through Token-level Attribute Controlled Generation Rahul Madhavan Rishabh Garg Kahini Wadhawan S. Mehta 87 5 0 01 Jun 2023
An Invariant Learning Characterization of Controlled Text Generation Carolina Zheng Claudia Shi Keyon Vafa Amir Feder David M. Blei OOD 103 8 0 31 May 2023
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback Paul Roit Johan Ferret Lior Shani Roee Aharoni Geoffrey Cideron ... Olivier Bachem G. Elidan Avinatan Hassidim Olivier Pietquin Idan Szpektor HILM 87 87 0 31 May 2023
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 239 1,241 0 31 May 2023
Controlled Text Generation with Hidden Representation Transformations Vaibhav Kumar H. Koorehdavoudi Masud Moshtaghi Amita Misra Ankit Chadha Emilio Ferrara 60 3 0 30 May 2023
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey Chen Ling Xujiang Zhao Jiaying Lu Chengyuan Deng Can Zheng ... Chris White Quanquan Gu Jian Pei Carl Yang Liang Zhao ALM 169 139 0 30 May 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 405 4,186 0 29 May 2023
Provable Reward-Agnostic Preference-Based Reinforcement Learning Wenhao Zhan Masatoshi Uehara Wen Sun Jason D. Lee 76 11 0 29 May 2023
Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models Shuai Zhao Xiaohan Wang Linchao Zhu Yezhou Yang VLM 111 23 0 29 May 2023
Taming AI Bots: Controllability of Neural States in Large Language Models Stefano Soatto Paulo Tabuada Pratik Chaudhari Tianwei Liu LLMAG LM&Ro 96 13 0 29 May 2023
Reward Collapse in Aligning Large Language Models Ziang Song Tianle Cai Jason D. Lee Weijie J. Su ALM 74 23 0 28 May 2023
Training Socially Aligned Language Models on Simulated Social Interactions Ruibo Liu Ruixin Yang Chenyan Jia Ge Zhang Denny Zhou Andrew M. Dai Diyi Yang Soroush Vosoughi ALM 78 56 0 26 May 2023
The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model Laixi Shi Gen Li Yuting Wei Yuxin Chen Matthieu Geist Yuejie Chi OOD 83 38 0 26 May 2023
Coarse-Tuning Models of Code with Reinforcement Learning Feedback Abhinav C. P. Jain Chima Adiole Swarat Chaudhuri Thomas W. Reps Chris Jermaine Rice University ALM 56 2 0 25 May 2023
Large Language Models for User Interest Journeys Konstantina Christakopoulou Alberto Lalama Cj Adams Iris Qu Yifat Amir ... Dina Bseiso Sarah Scodel Lucas Dixon Ed H. Chi Minmin Chen 102 30 0 24 May 2023
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback Katherine Tian E. Mitchell Allan Zhou Archit Sharma Rafael Rafailov Huaxiu Yao Chelsea Finn Christopher D. Manning 152 355 0 24 May 2023
In-Context Impersonation Reveals Large Language Models' Strengths and Biases Leonard Salewski Stephan Alaniz Isabel Rio-Torto Eric Schulz Zeynep Akata 102 159 0 24 May 2023
PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions Anthony Chen Panupong Pasupat Sameer Singh Hongrae Lee Kelvin Guu 111 48 0 24 May 2023
Provable Offline Preference-Based Reinforcement Learning Wenhao Zhan Masatoshi Uehara Nathan Kallus Jason D. Lee Wen Sun OffRL 125 32 0 24 May 2023
DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4 Ye Hu Kaiqiang Song Sangwoo Cho Xiaoyang Wang H. Foroosh Fei Liu 99 13 0 24 May 2023
RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learning Alexander Scarlatos Andrew Lan OffRL LRM 104 24 0 23 May 2023
Language Model Self-improvement by Reinforcement Learning Contemplation Jing-Cheng Pang Pengyuan Wang Kaiyuan Li Xiong-Hui Chen Jiacheng Xu Zongzhang Zhang Yang Yu LRM KELM 64 52 0 23 May 2023
Improving Factuality and Reasoning in Language Models through Multiagent Debate Yilun Du Shuang Li Antonio Torralba J. Tenenbaum Igor Mordatch LLMAG LRM 184 751 0 23 May 2023
Query Rewriting for Retrieval-Augmented Large Language Models Xinbei Ma Yeyun Gong Pengcheng He Hai Zhao Nan Duan KELM LRM 111 115 0 23 May 2023
"Is the Pope Catholic?" Applying Chain-of-Thought Reasoning to Understanding Conversational Implicatures Zae Myung Kim David E. Taylor Dongyeop Kang ReLM LRM AI4CE 27 5 0 23 May 2023
Aligning Large Language Models through Synthetic Feedback Sungdong Kim Sanghwan Bae Jamin Shin Soyoung Kang Donghyun Kwak Kang Min Yoo Minjoon Seo ALM SyDa 152 70 0 23 May 2023
Training Diffusion Models with Reinforcement Learning Kevin Black Michael Janner Yilun Du Ilya Kostrikov Sergey Levine EGVM 154 379 0 22 May 2023
Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization Chenhui Shen Liying Cheng Xuan-Phi Nguyen Yang You Lidong Bing ELM ALM 107 72 0 22 May 2023
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models Oana Ignat Zhijing Jin Artem Abzaliev Laura Biester Santiago Castro ... Verónica Pérez-Rosas Siqi Shen Zekun Wang Winston Wu Rada Mihalcea LRM 139 6 0 21 May 2023
Continually Improving Extractive QA via Human Feedback Ge Gao Hung-Ting Chen Yoav Artzi Eunsol Choi 87 12 0 21 May 2023
Collaborative Development of NLP models Fereshte Khani Marco Tulio Ribeiro 80 2 0 20 May 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation Models Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 179 103 0 19 May 2023
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Zhibin Gou Zhihong Shao Yeyun Gong Yelong Shen Yujiu Yang Nan Duan Weizhu Chen KELM LRM 148 398 0 19 May 2023
Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Models Wanqiao Xu Shi Dong Dilip Arumugam Benjamin Van Roy 76 8 0 19 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 132 96 0 19 May 2023
Language Models Meet World Models: Embodied Experiences Enhance Language Models Jiannan Xiang Tianhua Tao Yi Gu Tianmin Shu Zirui Wang Zichao Yang Zhiting Hu ALM LLMAG LM&Ro CLL 121 106 0 18 May 2023
SLiC-HF: Sequence Likelihood Calibration with Human Feedback Yao-Min Zhao Rishabh Joshi Tianqi Liu Misha Khalman Mohammad Saleh Peter J. Liu 88 307 0 17 May 2023
Sasha: Creative Goal-Oriented Reasoning in Smart Homes with Large Language Models Evan King Haoxiang Yu Sangsu Lee Christine Julien LM&Ro 36 20 0 16 May 2023
NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist Iftitahu Ni'mah Meng Fang Vlado Menkovski Mykola Pechenizkiy 80 14 0 15 May 2023
The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games? S. Phelps Y. Russell 81 17 0 13 May 2023
When Giant Language Brains Just Aren't Enough! Domain Pizzazz with Knowledge Sparkle Dust Minh Le Nguyen Duy-Hung Nguyen Shahab Sabahi Hung Le Jeffrey Yang Hajime Hotta 90 1 0 12 May 2023
GFlowNets with Human Feedback Yinchuan Li Shuang Luo Yunfeng Shao Jianye Hao AI4CE 68 5 0 11 May 2023
Knowledge-enhanced Agents for Interactive Text Games P. Chhikara Jiarui Zhang Filip Ilievski Jonathan M Francis Kaixin Ma LLMAG 85 8 0 08 May 2023
Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineering Noah Hollmann Samuel G. Müller Frank Hutter 114 64 0 05 May 2023