v1v2v3 (latest)

Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models

4 August 2024

Papers citing "Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models"

50 / 101 papers shown

Title
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models Yung-Sung Chuang Yujia Xie Hongyin Luo Yoon Kim James R. Glass Pengcheng He HILM 79 167 0 07 Sep 2023
Detecting and Preventing Hallucinations in Large Vision Language Models Anisha Gunjal Jihan Yin Erhan Bas MLLM VLM 83 174 0 11 Aug 2023
MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin Zicheng Liu Xinchao Wang Lijuan Wang MLLM 124 719 0 04 Aug 2023
LISA: Reasoning Segmentation via Large Language Model Xin Lai Zhuotao Tian Yukang Chen Yanwei Li Yuhui Yuan Shu Liu Jiaya Jia LM&Ro VLM MLLM LRM 164 461 0 01 Aug 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 422 12,091 0 18 Jul 2023
MMBench: Is Your Multi-modal Model an All-around Player? Yuanzhan Liu Haodong Duan Yuanhan Zhang Yue Liu Songyang Zhang ... Jiaqi Wang Conghui He Ziwei Liu Kai-xiang Chen Dahua Lin 142 1,059 0 12 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo MLLM VLM 160 238 0 07 Jul 2023
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic Ke Chen Zhao Zhang Weili Zeng Richong Zhang Feng Zhu Rui Zhao ObjD 113 651 0 27 Jun 2023
Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning Fuxiao Liu Kevin Qinghong Lin Linjie Li Jianfeng Wang Yaser Yacoob Lijuan Wang VLM MLLM 144 286 0 26 Jun 2023
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models Chaoyou Fu Peixian Chen Yunhang Shen Yulei Qin Mengdan Zhang ... Xiawu Zheng Ke Li Xing Sun Zhenyu Qiu Rongrong Ji ELM MLLM 136 859 0 23 Jun 2023
Scaling Open-Vocabulary Object Detection Matthias Minderer A. Gritsenko N. Houlsby VLM ObjD 105 202 0 16 Jun 2023
MIMIC-IT: Multi-Modal In-Context Instruction Tuning Yue Liu Yuanhan Zhang Liangyu Chen Jinghao Wang Fanyi Pu Jingkang Yang Cuiping Li Ziwei Liu MLLM VLM 96 240 0 08 Jun 2023
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day Chunyuan Li Cliff Wong Sheng Zhang Naoto Usuyama Haotian Liu Jianwei Yang Tristan Naumann Hoifung Poon Jianfeng Gao LM&MA MedIm 133 798 0 01 Jun 2023
DiffRate : Differentiable Compression Rate for Efficient Vision Transformers Mengzhao Chen Wenqi Shao Peng Xu Mingbao Lin Kaipeng Zhang Chia-Wen Lin Rongrong Ji Yu Qiao Ping Luo ViT 94 46 0 29 May 2023
PuMer: Pruning and Merging Tokens for Efficient Vision Language Models Qingqing Cao Bhargavi Paranjape Hannaneh Hajishirzi MLLM VLM 67 27 0 27 May 2023
How Language Model Hallucinations Can Snowball Muru Zhang Ofir Press William Merrill Alisa Liu Noah A. Smith HILM LRM 149 281 0 22 May 2023
Evaluating Object Hallucination in Large Vision-Language Models Yifan Li Yifan Du Kun Zhou Jinpeng Wang Wayne Xin Zhao Ji-Rong Wen MLLM LRM 315 814 0 17 May 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 157 2,099 0 11 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 299 956 0 27 Apr 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 165 2,072 0 20 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 575 4,936 0 17 Apr 2023
Non-Exemplar Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinement Fushuo Huo Wenchao Xu Jingcai Guo Yining Qi Yunfeng Fan Song Guo CLL 121 16 0 20 Mar 2023
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models Potsawee Manakul Adian Liusie Mark Gales HILM LRM 214 447 0 15 Mar 2023
Learning Combinatorial Prompts for Universal Controllable Image Captioning Zhen Wang Jun Xiao Yueting Zhuang Fei Gao Jian Shao Long Chen 84 5 0 11 Mar 2023
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection Shilong Liu Zhaoyang Zeng Tianhe Ren Feng Li Hao Zhang ... Chun-yue Li Jianwei Yang Hang Su Jun Zhu Lei Zhang ObjD 195 2,028 0 09 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.6K 13,490 0 27 Feb 2023
ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models Sheng Wang Zihao Zhao Xi Ouyang Qian Wang Dinggang Shen LM&MA MedIm 76 143 0 14 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 437 4,663 0 30 Jan 2023
Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning Xinyi Wang Wanrong Zhu Michael Stephen Saxon Mark Steyvers William Yang Wang BDL 165 116 0 27 Jan 2023
ProCC: Progressive Cross-primitive Compatibility for Open-World Compositional Zero-Shot Learning Fushuo Huo Wenchao Xu Song Guo Jingcai Guo Yining Qi Ziming Liu Xiaocheng Lu VLM 88 7 0 19 Nov 2022
Contrastive Decoding: Open-ended Text Generation as Optimization Xiang Lisa Li Ari Holtzman Daniel Fried Percy Liang Jason Eisner Tatsunori Hashimoto Luke Zettlemoyer M. Lewis 133 374 0 27 Oct 2022
Token Merging: Your ViT But Faster Daniel Bolya Cheng-Yang Fu Xiaoliang Dai Peizhao Zhang Christoph Feichtenhofer Judy Hoffman MoMe 130 471 0 17 Oct 2022
REQA: Coarse-to-fine Assessment of Image Quality to Alleviate the Range Effect Bingheng Li Fushuo Huo 75 12 0 05 Sep 2022
Confident Adaptive Language Modeling Tal Schuster Adam Fisch Jai Gupta Mostafa Dehghani Dara Bahri Vinh Q. Tran Yi Tay Donald Metzler 148 169 0 14 Jul 2022
Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Models Yi Zhang Junyan Wang Jitao Sang 83 28 0 03 Jul 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 295 2,521 0 15 Jun 2022
A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge Dustin Schwenk Apoorv Khandelwal Christopher Clark Kenneth Marino Roozbeh Mottaghi 74 555 0 03 Jun 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 552 4,526 0 24 May 2022
Survey of Hallucination in Natural Language Generation Ziwei Ji Nayeon Lee Rita Frieske Tiezheng Yu D. Su ... Delong Chen Wenliang Dai Ho Shu Chan Andrea Madotto Pascale Fung HILM LRM 256 2,449 0 08 Feb 2022
Let there be a clock on the beach: Reducing Object Hallucination in Image Captioning Ali Furkan Biten L. G. I. Bigorda Dimosthenis Karatzas 154 63 0 04 Oct 2021
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Yongming Rao Wenliang Zhao Benlin Liu Jiwen Lu Jie Zhou Cho-Jui Hsieh ViT 114 709 0 03 Jun 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 1.0K 29,926 0 26 Feb 2021
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 917 42,520 0 28 May 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 656 4,925 0 23 Jan 2020
The Curious Case of Neural Text Degeneration Ari Holtzman Jan Buys Li Du Maxwell Forbes Yejin Choi 209 3,213 0 22 Apr 2019
Object Hallucination in Image Captioning Anna Rohrbach Lisa Anne Hendricks Kaylee Burns Trevor Darrell Kate Saenko 206 443 0 06 Sep 2018
Hierarchical Neural Story Generation Angela Fan M. Lewis Yann N. Dauphin DiffM 183 1,631 0 13 May 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 832 132,725 0 12 Jun 2017
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations Ranjay Krishna Yuke Zhu Oliver Groth Justin Johnson Kenji Hata ... Yannis Kalantidis Li Li David A. Shamma Michael S. Bernstein Fei-Fei Li 237 5,766 0 23 Feb 2016
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 444 43,875 0 01 May 2014