Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignment

7 November 2023

Papers citing "Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignment"

26 / 26 papers shown

Title
A Survey on Progress in LLM Alignment from the Perspective of Reward Design Miaomiao Ji Yanqiu Wu Zhibin Wu Shoujin Wang Jian Yang Mark Dras Usman Naseem 39 0 0 05 May 2025
Learning Explainable Dense Reward Shapes via Bayesian Optimization Ryan Koo Ian Yang Vipul Raheja Mingyi Hong Kwang-Sung Jun Dongyeop Kang 26 0 0 22 Apr 2025
AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation Songming Zhang Xue Zhang Tong Zhang Bojie Hu Yufeng Chen Jinan Xu 50 1 0 04 Mar 2025
Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference Wenjie Qiu Yi-Chen Li Xuqin Zhang Tianyi Zhang Y. Zhang Zongzhang Zhang Yang Yu ALM 46 0 0 01 Mar 2025
Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model Yueqin Yin Shentao Yang Yujia Xie Ziyi Yang Yuting Sun Hany Awadalla Weizhu Chen Mingyuan Zhou 50 0 0 07 Jan 2025
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings Miguel Moura Ramos Tomás Almeida Daniel Vareta Filipe Azevedo Sweta Agrawal Patrick Fernandes André F. T. Martins 31 1 0 08 Nov 2024
Token-level Proximal Policy Optimization for Query Generation Yichen Ouyang Lu Wang Fangkai Yang Pu Zhao Chenghua Huang ... Saravan Rajmohan Weiwei Deng Dongmei Zhang Feng Sun Qi Zhang OffRL 134 3 0 01 Nov 2024
2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision Shilong Li Yancheng He Hui Huang Xingyuan Bu Jiaheng Liu Hangyu Guo Weixun Wang Jihao Gu Wenbo Su Bo Zheng 29 5 0 25 Oct 2024
RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style Yantao Liu Zijun Yao Rui Min Yixin Cao Lei Hou Juanzi Li OffRL ALM 20 29 0 21 Oct 2024
RevisEval: Improving LLM-as-a-Judge via Response-Adapted References Qiyuan Zhang Yufei Wang Tiezheng YU Yuxin Jiang Chuhan Wu ... Xin Jiang Lifeng Shang Ruiming Tang Fuyuan Lyu Chen Ma 31 4 0 07 Oct 2024
Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization Yuxin Jiang Bo Huang Yufei Wang Xingshan Zeng Liangyou Li Yasheng Wang Xin Jiang Lifeng Shang Ruiming Tang Wei Wang 44 5 0 14 Aug 2024
TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedback Eunseop Yoon Hee Suk Yoon Soohwan Eom Gunsoo Han D. W. Nam DaeJin Jo Kyoung-Woon On M. Hasegawa-Johnson Sungwoong Kim C. Yoo ALM 38 15 0 23 Jul 2024
Model Merging and Safety Alignment: One Bad Model Spoils the Bunch Hasan Hammoud Umberto Michieli Fabio Pizzati Philip H. S. Torr Adel Bibi Guohao Li Mete Ozay MoMe 31 15 0 20 Jun 2024
A Survey on Human Preference Learning for Large Language Models Ruili Jiang Kehai Chen Xuefeng Bai Zhixuan He Juntao Li Muyun Yang Tiejun Zhao Liqiang Nie Min Zhang 49 8 0 17 Jun 2024
Aligning Large Language Models via Fine-grained Supervision Dehong Xu Liang Qiu Minseok Kim Faisal Ladhak Jaeyoung Do 40 2 0 04 Jun 2024
Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation Chengwei Dai Kun Li Wei Zhou Song Hu LRM 49 3 0 30 May 2024
Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation Chengwei Dai Kun Li Wei Zhou Song Hu LRM 43 5 0 30 May 2024
Hummer: Towards Limited Competitive Preference Dataset Li Jiang Yusen Wu Junwu Xiong Jingqing Ruan Yichuan Ding Qingpei Guo Zujie Wen Jun Zhou Xiaotie Deng 34 6 0 19 May 2024
A Survey on Knowledge Distillation of Large Language Models Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Dinesh Manocha KELM VLM 44 101 0 20 Feb 2024
KnowTuning: Knowledge-aware Fine-tuning for Large Language Models Yougang Lyu Lingyong Yan Shuaiqiang Wang Haibo Shi Dawei Yin Pengjie Ren Zhumin Chen Maarten de Rijke Zhaochun Ren 21 5 0 17 Feb 2024
Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback Songyang Gao Qiming Ge Wei Shen Shihan Dou Junjie Ye ... Yicheng Zou Zhi Chen Hang Yan Qi Zhang Dahua Lin 57 10 0 21 Jan 2024
Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint Zhipeng Chen Kun Zhou Wayne Xin Zhao Junchen Wan Fuzheng Zhang Di Zhang Ji-Rong Wen KELM 39 32 0 11 Jan 2024
Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits Ruibo Liu Chenyan Jia Ge Zhang Ziyu Zhuang Tony X. Liu Soroush Vosoughi 99 35 0 01 Jan 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 325 4,077 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
$Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information$ Understanding Dataset Difficulty with $\mathcal{V}$ -Usable Information Kawin Ethayarajh Yejin Choi Swabha Swayamdipta 164 157 0 16 Oct 2021