MPO: Multilingual Safety Alignment via Reward Gap Optimization

22 May 2025

Papers citing "MPO: Multilingual Safety Alignment via Reward Gap Optimization"

30 / 30 papers shown

Title
The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual context Nikhil Verma Manasa Bharadwaj 72 2 0 03 Apr 2025
DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization Zhenglin Zhou Xiaobo Xia Fan Ma Hehe Fan Yi Yang Tat-Seng Chua 65 7 0 05 Feb 2025
LIMO: Less is More for Reasoning Yixin Ye Zhen Huang Yang Xiao Ethan Chern Shijie Xia Pengfei Liu LRM 160 144 0 05 Feb 2025
LiPO: Listwise Preference Optimization through Learning-to-Rank Tianqi Liu Zhen Qin Junru Wu Jiaming Shen Misha Khalman ... Mohammad Saleh Simon Baumgartner Jialu Liu Peter J. Liu Xuanhui Wang 286 55 0 28 Jan 2025
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 95 5 0 23 Oct 2024
Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks Samuele Poppi Zheng-Xin Yong Yifei He Bobbie Chern Han Zhao Aobo Yang Jianfeng Chi AAML 125 20 0 23 Oct 2024
Language Imbalance Driven Rewarding for Multilingual Self-improving Wen Yang Junhong Wu Chen Wang Chengqing Zong J.N. Zhang ALM LRM 152 7 0 11 Oct 2024
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 165 26 0 11 Oct 2024
Large Language Models Are Cross-Lingual Knowledge-Free Reasoners Peng Hu Sizhe Liu Changjiang Gao Xin Huang Xue Han Junlan Feng Chao Deng Shujian Huang LRM 83 5 0 24 Jun 2024
Sharing Matters: Analysing Neurons Across Languages and Tasks in LLMs Weixuan Wang Barry Haddow Wei Peng Alexandra Birch MILM 62 16 0 13 Jun 2024
WildChat: 1M ChatGPT Interaction Logs in the Wild Wenting Zhao Xiang Ren Jack Hessel Claire Cardie Yejin Choi Yuntian Deng 69 222 0 02 May 2024
Self-Play Preference Optimization for Language Model Alignment Yue Wu Zhiqing Sun Huizhuo Yuan Kaixuan Ji Yiming Yang Quanquan Gu 88 138 0 01 May 2024
Disentangling Length from Quality in Direct Preference Optimization Ryan Park Rafael Rafailov Stefano Ermon Chelsea Finn ALM 86 135 0 28 Mar 2024
ORPO: Monolithic Preference Optimization without Reference Model Jiwoo Hong Noah Lee James Thorne OSLM 77 249 0 12 Mar 2024
Provably Robust DPO: Aligning Language Models with Noisy Feedback Sayak Ray Chowdhury Anush Kini Nagarajan Natarajan 71 68 0 01 Mar 2024
Rethinking the Role of Proxy Rewards in Language Model Alignment Sungdong Kim Minjoon Seo SyDa ALM 46 2 0 02 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 255 533 0 02 Feb 2024
Secrets of RLHF in Large Language Models Part II: Reward Modeling Bing Wang Rui Zheng Luyao Chen Yan Liu Shihan Dou ... Qi Zhang Xipeng Qiu Xuanjing Huang Zuxuan Wu Yuanyuan Jiang ALM 91 107 0 11 Jan 2024
Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages Libo Qin Qiguang Chen Fuxuan Wei Shijue Huang Wanxiang Che LRM 83 89 0 23 Oct 2023
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 171 616 0 18 Oct 2023
A Long Way to Go: Investigating Length Correlations in RLHF Prasann Singhal Tanya Goyal Jiacheng Xu Greg Durrett 99 153 0 05 Oct 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 346 4,298 0 09 Jun 2023
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment Hanze Dong Wei Xiong Deepanshu Goyal Yihan Zhang Winnie Chow Boyao Wang Shizhe Diao Jipeng Zhang Kashun Shum Tong Zhang ALM 68 454 0 13 Apr 2023
Language Models are Multilingual Chain-of-Thought Reasoners Freda Shi Mirac Suzgun Markus Freitag Xuezhi Wang Suraj Srivats ... Yi Tay Sebastian Ruder Denny Zhou Dipanjan Das Jason W. Wei ReLM LRM 238 363 0 06 Oct 2022
No Language Left Behind: Scaling Human-Centered Machine Translation Nllb team Marta R. Costa-jussá James Cross Onur cCelebi Maha Elbayad ... Alexandre Mourachko C. Ropers Safiyyah Saleem Holger Schwenk Jeff Wang MoE 223 1,260 0 11 Jul 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 874 12,916 0 04 Mar 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 277 4,397 0 27 Oct 2021
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 749 41,932 0 28 May 2020
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 463 1,727 0 18 Sep 2019
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor Tuomas Haarnoja Aurick Zhou Pieter Abbeel Sergey Levine 305 8,334 0 04 Jan 2018