Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images

19 February 2025

Papers citing "Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images"

29 / 29 papers shown

Title
A Stereotype Content Analysis on Color-related Social Bias in Large Vision Language Models Junhyuk Choi Minju Kim Yeseon Hong Bugeun Kim 7 0 0 27 May 2025
OViP: Online Vision-Language Preference Learning Shujun Liu Siyuan Wang Zejun Li Jianxiang Wang Cheng Zeng Zhongyu Wei MLLM VLM 26 0 0 21 May 2025
SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization Hongrui Jia Chaoya Jiang Haiyang Xu Wei Ye Mengfan Dong Ming Yan Ji Zhang Fei Huang Shikun Zhang MLLM 116 2 0 17 Nov 2024
V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization Yuxi Xie Guanzhen Li Xiao Xu Min-Yen Kan MLLM VLM 70 17 0 05 Nov 2024
Modality-Fair Preference Optimization for Trustworthy MLLM Alignment Songtao Jiang Yan Zhang Ruizhe Chen Yeying Jin Zuozhu Liu MLLM MoE 41 6 0 20 Oct 2024
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension Junzhuo Liu Xiaohu Yang Weiwei Li Peng Wang ObjD 76 3 0 23 Sep 2024
LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models Feng Li Renrui Zhang Hao Zhang Yuanhan Zhang Bo Li Wei Li Zejun Ma Chunyuan Li MLLM VLM 60 208 0 10 Jul 2024
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong Ellis L Brown Penghao Wu Sanghyun Woo Manoj Middepogu ... Xichen Pan Austin Wang Rob Fergus Yann LeCun Saining Xie 3DV MLLM 65 321 0 24 Jun 2024
Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective Meiqi Chen Yixin Cao Yan Zhang Chaochao Lu 63 16 0 27 Mar 2024
CounterCurate: Enhancing Physical and Semantic Visio-Linguistic Compositional Reasoning via Counterfactual Examples Jianrui Zhang Mu Cai Tengyang Xie Yong Jae Lee LRM 48 20 0 20 Feb 2024
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning Yiyang Zhou Chenhang Cui Rafael Rafailov Chelsea Finn Huaxiu Yao VLM MLLM 70 97 0 18 Feb 2024
Supervised Fine-tuning in turn Improves Visual Foundation Models Xiaohu Jiang Yixiao Ge Yuying Ge Dachuan Shi Chun Yuan Ying Shan VLM CLIP 68 9 0 18 Jan 2024
Silkie: Preference Distillation for Large Visual Language Models Lei Li Zhihui Xie Mukai Li Shunian Chen Peiyi Wang Liang Chen Yazheng Yang Benyou Wang Lingpeng Kong MLLM 126 69 0 17 Dec 2023
A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting Junhao Zhuang Yanhong Zeng Wenran Liu Chun Yuan Kai Chen DiffM 80 72 0 06 Dec 2023
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback M. Steyvers Yuan Yao Haoye Zhang Taiwen He Yifeng Han ... Xinyue Hu Zhiyuan Liu Hai-Tao Zheng Maosong Sun Tat-Seng Chua MLLM VLM 157 198 0 01 Dec 2023
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 136 833 0 27 Nov 2023
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V Jianwei Yang Hao Zhang Feng Li Xueyan Zou Chun-yue Li Jianfeng Gao MLLM VLM 45 179 0 17 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 67 2,593 0 05 Oct 2023
MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts Pan Lu Hritik Bansal Tony Xia Jiacheng Liu Chun-yue Li Hannaneh Hajishirzi Hao Cheng Kai-Wei Chang Michel Galley Jianfeng Gao LRM MLLM 56 541 0 03 Oct 2023
Aligning Large Multimodal Models with Factually Augmented RLHF Zhiqing Sun Sheng Shen Shengcao Cao Haotian Liu Chunyuan Li ... Liangyan Gui Yu-Xiong Wang Yiming Yang Kurt Keutzer Trevor Darrell VLM 75 351 0 25 Sep 2023
MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin Zicheng Liu Xinchao Wang Lijuan Wang MLLM 78 656 0 04 Aug 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 251 3,712 0 29 May 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 45 1,977 0 11 May 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 54 1,028 0 27 Mar 2023
The effectiveness of MAE pre-pretraining for billion-scale pretraining Mannat Singh Quentin Duval Kalyan Vasudev Alwala Haoqi Fan Vaibhav Aggarwal ... Piotr Dollár Christoph Feichtenhofer Ross B. Girshick Rohit Girdhar Ishan Misra LRM 143 64 0 23 Mar 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark Ashwin Kalyan ELM ReLM LRM 217 1,188 0 20 Sep 2022
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 510 28,659 0 26 Feb 2021
Towards VQA Models That Can Read Amanpreet Singh Vivek Natarajan Meet Shah Yu Jiang Xinlei Chen Dhruv Batra Devi Parikh Marcus Rohrbach EgoV 35 1,174 0 18 Apr 2019
A Diagram Is Worth A Dozen Images Aniruddha Kembhavi M. Salvato Eric Kolve Minjoon Seo Hannaneh Hajishirzi Ali Farhadi 3DV 19 456 0 24 Mar 2016