Sight Beyond Text: Multi-Modal Training Enhances LLMs in Truthfulness
and Ethics

Sight Beyond Text: Multi-Modal Training Enhances LLMs in Truthfulness and Ethics

13 September 2023

Papers citing "Sight Beyond Text: Multi-Modal Training Enhances LLMs in Truthfulness and Ethics"

15 / 15 papers shown

Title
Language Models Can See Better: Visual Contrastive Decoding For LLM Multimodal Reasoning Yuqi Pang Bowen Yang Haoqin Tu Yun Cao Zeyu Zhang LRM MLLM 64 0 0 17 Feb 2025
AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation Zijun Wang Haoqin Tu J. Mei Bingchen Zhao Yunhong Wang Cihang Xie 24 5 0 11 Oct 2024
VHELM: A Holistic Evaluation of Vision Language Models Tony Lee Haoqin Tu Chi Heem Wong Wenhao Zheng Yiyang Zhou ... Josselin Somerville Roberts Michihiro Yasunaga Huaxiu Yao Cihang Xie Percy Liang VLM 45 10 0 09 Oct 2024
Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models Xin Zou Yizhou Wang Yibo Yan Yuanhuiyi Lyu Kening Zheng ... Junkai Chen Peijie Jiang Jiaheng Liu Chang Tang Xuming Hu 86 7 0 04 Oct 2024
A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor? Yunfei Xie Juncheng Wu Haoqin Tu Siwei Yang Bingchen Zhao Yongshuo Zong Qiao Jin Cihang Xie Yuyin Zhou LM&MA ELM LRM 46 18 0 23 Sep 2024
The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Models Zihui Wu Haichang Gao Jianping He Ping Wang 24 6 0 25 Jul 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective Zhen Qin Daoyuan Chen Wenhao Zhang Liuyi Yao Yilun Huang Bolin Ding Yaliang Li Shuiguang Deng 57 5 0 11 Jul 2024
Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning Bingchen Zhao Haoqin Tu Chen Wei Jieru Mei Cihang Xie 20 32 0 18 Dec 2023
How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs Haoqin Tu Chenhang Cui Zijun Wang Yiyang Zhou Bingchen Zhao Junlin Han Wangchunshu Zhou Huaxiu Yao Cihang Xie MLLM 60 70 0 27 Nov 2023
Vision-Language Instruction Tuning: A Review and Analysis Chen Li Yixiao Ge Dian Li Ying Shan VLM 33 12 0 14 Nov 2023
Mitigating the Alignment Tax of RLHF Yong Lin Hangyu Lin Wei Xiong Shizhe Diao Zeming Zheng ... Han Zhao Nan Jiang Heng Ji Yuan Yao Tong Zhang MoMe CLL 29 65 0 12 Sep 2023
ReSee: Responding through Seeing Fine-grained Visual Knowledge in Open-domain Dialogue Haoqin Tu Yitong Li Fei Mi Zhongliang Yang 37 4 0 23 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 208 900 0 27 Apr 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 270 4,244 0 30 Jan 2023
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 278 1,082 0 17 Feb 2021