mDPO: Conditional Preference Optimization for Multimodal Large Language Models

17 June 2024

Sheng Zhang

Muhao Chen

Papers citing "mDPO: Conditional Preference Optimization for Multimodal Large Language Models"

25 / 25 papers shown

Title
Co-Reinforcement Learning for Unified Multimodal Understanding and Generation Jingjing Jiang Chongjie Si Jun Luo Hanwang Zhang Chao Ma 170 0 0 23 May 2025
VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment Yogesh Kulkarni Pooyan Fazli 98 0 0 18 Apr 2025
PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning Xinpeng Ding Kai Zhang Jinahua Han Lanqing Hong Hang Xu Xuelong Li MLLM VLM 476 0 0 08 Apr 2025
MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization Kangyu Zhu Peng Xia Yun Li Hongtu Zhu Sheng Wang Huaxiu Yao 154 3 0 09 Dec 2024
From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning Nan Xu Fei Wang Sheng Zhang Hoifung Poon Muhao Chen 112 7 0 01 Jul 2024
WPO: Enhancing RLHF with Weighted Preference Optimization Wenxuan Zhou Ravi Agrawal Shujian Zhang Sathish Indurthi Sanqiang Zhao Kaiqiang Song Silei Xu Chenguang Zhu 89 20 0 17 Jun 2024
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding Fei Wang Xingyu Fu James Y. Huang Zekun Li Qin Liu ... Kai-Wei Chang Dan Roth Sheng Zhang Hoifung Poon Muhao Chen VLM 103 59 0 13 Jun 2024
Self-Play Preference Optimization for Language Model Alignment Yue Wu Zhiqing Sun Huizhuo Yuan Kaixuan Ji Yiming Yang Quanquan Gu 105 145 0 01 May 2024
Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback Wenyi Xiao Ziwei Huang Leilei Gan Wanggui He Haoyuan Li Zhelun Yu Hao Jiang Leilei Gan Linchao Zhu MLLM 81 34 0 22 Apr 2024
FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback Liqiang Jing Xinya Du 149 17 0 07 Apr 2024
Disentangling Length from Quality in Direct Preference Optimization Ryan Park Rafael Rafailov Stefano Ermon Chelsea Finn ALM 93 145 0 28 Mar 2024
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning Yiyang Zhou Chenhang Cui Rafael Rafailov Chelsea Finn Huaxiu Yao VLM MLLM 91 120 0 18 Feb 2024
Noise Contrastive Alignment of Language Models with Explicit Rewards Huayu Chen Guande He Lifan Yuan Ganqu Cui Hang Su Jun Zhu 97 56 0 08 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 270 565 0 02 Feb 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 365 337 0 18 Jan 2024
OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation Qidong Huang Xiao-wen Dong Pan Zhang Bin Wang Conghui He Jiaqi Wang Dahua Lin Weiming Zhang Neng H. Yu MLLM 124 206 0 29 Nov 2023
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 178 645 0 18 Oct 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 389 4,139 0 29 May 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 248 1,200 0 27 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,699 0 15 Mar 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 886 13,176 0 04 Mar 2022
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 495 3,443 0 09 Mar 2020
Object Hallucination in Image Captioning Anna Rohrbach Lisa Anne Hendricks Kaylee Burns Trevor Darrell Kate Saenko 194 443 0 06 Sep 2018
Deep reinforcement learning from human preferences Paul Christiano Jan Leike Tom B. Brown Miljan Martic Shane Legg Dario Amodei 218 3,365 0 12 Jun 2017