SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF

9 October 2023

Yi Dong

Zhilin Wang

Makesh Narsimhan Sreedhar

Papers citing "SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF"

30 / 30 papers shown

Title
A Survey on Progress in LLM Alignment from the Perspective of Reward Design Miaomiao Ji Yanqiu Wu Zhibin Wu Shoujin Wang Jian Yang Mark Dras Usman Naseem 39 0 0 05 May 2025
CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives Ayoung Lee Ryan Sungmo Kwon Peter Railton Lu Wang ELM 51 0 0 15 Apr 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian Guan Jian Wu J. Li Chuanqi Cheng Wei Wu LM&MA 83 0 0 21 Mar 2025
OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses Angela Lopez-Cardona Sebastian Idesis Miguel Barreda-Ángeles Sergi Abadal Ioannis Arapakis 51 0 0 13 Mar 2025
AI Alignment at Your Discretion Maarten Buyl Hadi Khalaf C. M. Verdun Lucas Monteiro Paes Caio Vieira Machado Flavio du Pin Calmon 45 0 0 10 Feb 2025
ChipAlign: Instruction Alignment in Large Language Models for Chip Design via Geodesic Interpolation Chenhui Deng Yunsheng Bai Haoxing Ren 39 1 0 31 Dec 2024
Identifying and Manipulating Personality Traits in LLMs Through Activation Engineering Rumi A. Allbert James K. Wiles Vlad Grankovsky LLMSV AI4CE 85 1 0 10 Dec 2024
Interpreting Language Reward Models via Contrastive Explanations Junqi Jiang Tom Bewley Saumitra Mishra Freddy Lecue Manuela Veloso 76 0 0 25 Nov 2024
On the Loss of Context-awareness in General Instruction Fine-tuning Yihan Wang Andrew Bai Nanyun Peng Cho-Jui Hsieh 130 1 0 05 Nov 2024
SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignment Qin Liu Fei Wang Chaowei Xiao Muhao Chen 181 0 0 18 Oct 2024
Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models Hongchuan Zeng Senyu Han Lu Chen Kai Yu 62 6 0 15 Oct 2024
Taming Overconfidence in LLMs: Reward Calibration in RLHF Jixuan Leng Chengsong Huang Banghua Zhu Jiaxin Huang 34 7 0 13 Oct 2024
Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements Jingyu Zhang Ahmed Elgohary Ahmed Magooda Daniel Khashabi Benjamin Van Durme 165 2 0 11 Oct 2024
Reward-Augmented Data Enhances Direct Preference Alignment of LLMs Shenao Zhang Zhihan Liu Boyi Liu Wenjie Qu Yingxiang Yang Y. Liu Liyu Chen Tao Sun Ziyi Wang 101 3 0 10 Oct 2024
Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models Angela Lopez-Cardona Carlos Segura Alexandros Karatzoglou Sergi Abadal Ioannis Arapakis ALM 62 2 0 02 Oct 2024
Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown Xingzhou Lou Dong Yan Wei Shen Yuzi Yan Jian Xie Junge Zhang 53 22 0 01 Oct 2024
Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Wenxuan Zhang Philip Torr Mohamed Elhoseiny Adel Bibi 88 9 0 27 Aug 2024
Towards Aligning Language Models with Textual Feedback Sauc Abadal Lloret S. Dhuliawala K. Murugesan Mrinmaya Sachan VLM 46 1 0 24 Jul 2024
OpenLLM-Ro -- Technical Report on Open-source Romanian LLMs Mihai Masala Denis C. Ilie-Ablachim D. Corlatescu Miruna Zavelca Marius Leordeanu Horia Velicu Marius Popescu Mihai Dascalu Traian Rebedea 51 2 0 13 May 2024
NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment Gerald Shen Zhilin Wang Olivier Delalleau Jiaqi Zeng Yi Dong ... Sahil Jain Ali Taghibakhshi Markel Sanz Ausin Ashwath Aithal Oleksii Kuchaiev 43 13 0 02 May 2024
Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback Vincent Conitzer Rachel Freedman J. Heitzig Wesley H. Holliday Bob M. Jacobs ... Eric Pacuit Stuart Russell Hailey Schoelkopf Emanuel Tewolde W. Zwicker 43 30 0 16 Apr 2024
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards Haoxiang Wang Yong Lin Wei Xiong Rui Yang Shizhe Diao Shuang Qiu Han Zhao Tong Zhang 40 72 0 28 Feb 2024
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 162 579 0 06 Apr 2023
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 231 446 0 23 Aug 2022
Offline RL for Natural Language Generation with Implicit Language Q Learning Charles Burton Snell Ilya Kostrikov Yi Su Mengjiao Yang Sergey Levine OffRL 139 102 0 05 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 357 12,003 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 215 1,661 0 15 Oct 2021
Extracting and Inferring Personal Attributes from Dialogue Zhilin Wang Xuhui Zhou Rik Koncel-Kedziorski Alex Marin Fei Xia 38 14 0 26 Sep 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,489 0 23 Jan 2020
NeMo: a toolkit for building AI applications using Neural Modules Oleksii Kuchaiev Jason Chun Lok Li Huyen Nguyen Oleksii Hrinchuk Ryan Leary ... Jack Cook P. Castonguay Mariya Popova Jocelyn Huang Jonathan M. Cohen 211 292 0 14 Sep 2019