Fine-Tuning Language Models from Human Preferences

18 September 2019

Papers citing "Fine-Tuning Language Models from Human Preferences"

50 / 362 papers shown

Title
Efficient Knowledge Infusion via KG-LLM Alignment Zhouyu Jiang Ling Zhong Mengshu Sun Jun Xu Rui Sun Hui Cai Shuhan Luo Qing Cui 42 9 0 06 Jun 2024
HYDRA: Model Factorization Framework for Black-Box LLM Personalization Yuchen Zhuang Haotian Sun Yue Yu Rushi Qiang Qifan Wang Chao Zhang Bo Dai AAML 51 15 0 05 Jun 2024
TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models Chen Zhang Chengguang Tang Dading Chong Ke Shi Guohua Tang Feng Jiang Haizhou Li 35 4 0 30 May 2024
Enhancing Reinforcement Learning with Label-Sensitive Reward for Natural Language Understanding Kuo Liao Shuang Li Meng Zhao Liqun Liu Mengge Xue Zhenyu Hu Honglin Han Chengguo Yin 40 1 0 30 May 2024
One-Shot Safety Alignment for Large Language Models via Optimal Dualization Xinmeng Huang Shuo Li Yan Sun Osbert Bastani Hamed Hassani Dongsheng Ding 47 4 0 29 May 2024
Offline Regularised Reinforcement Learning for Large Language Models Alignment Pierre Harvey Richemond Yunhao Tang Daniel Guo Daniele Calandriello M. G. Azar ... Gil Shamir Rishabh Joshi Tianqi Liu Rémi Munos Bilal Piot OffRL 46 22 0 29 May 2024
Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice Jian-Qiao Zhu Haijiang Yan Thomas L. Griffiths 84 2 0 29 May 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu Vardan Papyan 48 12 0 28 May 2024
Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models Chia-Yi Hsu Yu-Lin Tsai Chih-Hsun Lin Pin-Yu Chen Chia-Mu Yu Chun-ying Huang 49 34 0 27 May 2024
Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories Tianlong Wang Xianfeng Jiao Yifan He Zhongzhi Chen Yinghao Zhu Xu Chu Junyi Gao Yasha Wang Liantao Ma LLMSV 71 7 0 26 May 2024
Learning Generalizable Human Motion Generator with Reinforcement Learning Yunyao Mao Xiaoyang Liu Wen-gang Zhou Zhenbo Lu Houqiang Li 38 2 0 24 May 2024
Bayesian WeakS-to-Strong from Text Classification to Generation Ziyun Cui Ziyang Zhang Wen Wu Wen Wu Chao Zhang 39 2 0 24 May 2024
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations Ziqiao Ma Zekun Wang Joyce Chai 58 2 0 22 May 2024
Curriculum Direct Preference Optimization for Diffusion and Consistency Models Florinel-Alin Croitoru Vlad Hondru Radu Tudor Ionescu N. Sebe Mubarak Shah EGVM 89 6 0 22 May 2024
SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling Xingzhou Lou Junge Zhang Jian Xie Lifeng Liu Dong Yan Kaiqi Huang 45 11 0 21 May 2024
A Unified Linear Programming Framework for Offline Reward Learning from Human Demonstrations and Feedback Kihyun Kim Jiawei Zhang Asuman Ozdaglar P. Parrilo OffRL 41 1 0 20 May 2024
Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process Ermo Hua Biqing Qi Kaiyan Zhang Yue Yu Ning Ding Xingtai Lv Kai Tian Bowen Zhou 43 3 0 20 May 2024
OpenLLM-Ro -- Technical Report on Open-source Romanian LLMs Mihai Masala Denis C. Ilie-Ablachim D. Corlatescu Miruna Zavelca Marius Leordeanu Horia Velicu Marius Popescu Mihai Dascalu Traian Rebedea 48 2 0 13 May 2024
Improving Instruction Following in Language Models through Proxy-Based Uncertainty Estimation JoonHo Lee Jae Oh Woo Juree Seok Parisa Hassanzadeh Wooseok Jang ... Hankyu Moon Wenjun Hu Yeong-Dae Kwon Taehee Lee Seungjai Min 47 2 0 10 May 2024
Truthful Aggregation of LLMs with an Application to Online Advertising Ermis Soumalias Michael J. Curry Sven Seuken 39 11 0 09 May 2024
Optimizing Language Model's Reasoning Abilities with Weak Supervision Yongqi Tong Sizhe Wang Dawei Li Yifan Wang Simeng Han Zi Lin Chengsong Huang Jiaxin Huang Jingbo Shang LRM ReLM 42 8 0 07 May 2024
Reinforcement Learning-Guided Semi-Supervised Learning Marzi Heidari Hanping Zhang Yuhong Guo OffRL 39 0 0 02 May 2024
A Framework for Real-time Safeguarding the Text Generation of Large Language Model Ximing Dong Dayi Lin Shaowei Wang Ahmed E. Hassan 41 1 0 29 Apr 2024
Performance-Aligned LLMs for Generating Fast Code Daniel Nichols Pranav Polasam Harshitha Menon Aniruddha Marathe T. Gamblin A. Bhatele 35 8 0 29 Apr 2024
DPO Meets PPO: Reinforced Token Optimization for RLHF Han Zhong Guhao Feng Guhao Feng Li Zhao Di He Jiang Bian Liwei Wang Jiang Bian Liwei Wang 55 57 0 29 Apr 2024
Paint by Inpaint: Learning to Add Image Objects by Removing Them First Navve Wasserman Noam Rotstein Roy Ganz Ron Kimmel DiffM 39 15 0 28 Apr 2024
Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputs Valeriia Cherepanova James Zou AAML 33 4 0 26 Apr 2024
Don't Say No: Jailbreaking LLM by Suppressing Refusal Yukai Zhou Wenjie Wang AAML 42 15 0 25 Apr 2024
Generating Attractive and Authentic Copywriting from Customer Reviews Yu-Xiang Lin Wei-Yun Ma 42 2 0 22 Apr 2024
Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback Vincent Conitzer Rachel Freedman J. Heitzig Wesley H. Holliday Bob M. Jacobs ... Eric Pacuit Stuart Russell Hailey Schoelkopf Emanuel Tewolde W. Zwicker 43 28 0 16 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 46 7 0 13 Apr 2024
Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Data Tim Baumgärtner Yang Gao Dana Alon Donald Metzler AAML 30 18 0 08 Apr 2024
FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback Liqiang Jing Xinya Du 79 17 0 07 Apr 2024
Binary Classifier Optimization for Large Language Model Alignment Seungjae Jung Gunsoo Han D. W. Nam Kyoung-Woon On 42 21 0 06 Apr 2024
Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data Jingyu Zhang Marc Marone Tianjian Li Benjamin Van Durme Daniel Khashabi 93 9 0 05 Apr 2024
Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMs Shu Yang Jiayuan Su Han Jiang Mengdi Li Keyuan Cheng Muhammad Asif Ali Lijie Hu Di Wang 35 5 0 30 Mar 2024
Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model Qi Gou Cam-Tu Nguyen 35 8 0 28 Mar 2024
Learning To Guide Human Decision Makers With Vision-Language Models Debodeep Banerjee Stefano Teso Burcu Sayin Andrea Passerini 37 1 0 25 Mar 2024
The pitfalls of next-token prediction Gregor Bachmann Vaishnavh Nagarajan 37 62 0 11 Mar 2024
Enhancing Court View Generation with Knowledge Injection and Guidance Ang Li Yiquan Wu Yifei Liu Fei Wu Ming Cai Kun Kuang AILaw 38 0 0 07 Mar 2024
Improving Socratic Question Generation using Data Augmentation and Preference Optimization Nischal Ashok Kumar Andrew S. Lan 33 8 0 01 Mar 2024
Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency Akila Wickramasekara F. Breitinger Mark Scanlon 52 8 0 29 Feb 2024
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards Haoxiang Wang Yong Lin Wei Xiong Rui Yang Shizhe Diao Shuang Qiu Han Zhao Tong Zhang 40 71 0 28 Feb 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 56 17 0 28 Feb 2024
From Large Language Models and Optimization to Decision Optimization CoPilot: A Research Manifesto Segev Wasserkrug Léonard Boussioux D. Hertog F. Mirzazadeh Ilker Birbil Jannis Kurtz Donato Maragno LLMAG 46 3 0 26 Feb 2024
Generalizing Reward Modeling for Out-of-Distribution Preference Learning Chen Jia 41 2 0 22 Feb 2024
OmniPred: Language Models as Universal Regressors Xingyou Song Oscar Li Chansoo Lee Bangding Yang Daiyi Peng Sagi Perel Yutian Chen 56 14 0 22 Feb 2024
Mode Estimation with Partial Feedback Charles Arnal Vivien A. Cabannes Vianney Perchet 42 0 0 20 Feb 2024
A Critical Evaluation of AI Feedback for Aligning Large Language Models Archit Sharma Sedrick Scott Keh Eric Mitchell Chelsea Finn Kushal Arora Thomas Kollar ALM LLMAG 23 23 0 19 Feb 2024
Reinforcement Learning from Human Feedback with Active Queries Kaixuan Ji Jiafan He Quanquan Gu 24 17 0 14 Feb 2024