Self-Bootstrapped Visual-Language Model for Knowledge Selection and Question Answering

22 April 2024

Jing Liu

Papers citing "Self-Bootstrapped Visual-Language Model for Knowledge Selection and Question Answering"

35 / 35 papers shown

Title
PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers Weizhe Lin Jingbiao Mei Jinghong Chen Bill Byrne VLM AI4Ed 87 18 0 13 Feb 2024
Filling the Image Information Gap for VQA: Prompting Large Language Models to Proactively Ask Questions Ziyue Wang Chi Chen Peng Li Yang Liu LRM 66 14 0 20 Nov 2023
Large Language Models are Visual Reasoning Coordinators Liangyu Chen Bo Li Sheng Shen Jingkang Yang Chunyuan Li Kurt Keutzer Trevor Darrell Ziwei Liu VLM LRM 56 54 0 23 Oct 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 206 11,636 0 18 Jul 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 72 2,017 0 11 May 2023
PromptCap: Prompt-Guided Task-Aware Image Captioning Yushi Hu Hang Hua Zhengyuan Yang Weijia Shi Noah A. Smith Jiebo Luo 75 103 0 15 Nov 2022
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale Yuxin Fang Wen Wang Binhui Xie Quan-Sen Sun Ledell Yu Wu Xinggang Wang Tiejun Huang Xinlong Wang Yue Cao VLM CLIP 148 702 0 14 Nov 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 146 3,072 0 20 Oct 2022
Retrieval Augmented Visual Question Answering with Outside Knowledge Weizhe Lin Bill Byrne RALM 84 71 0 07 Oct 2022
A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA Yangyang Guo Liqiang Nie Yongkang Wong Yebin Liu Zhiyong Cheng Mohan S. Kankanhalli 85 39 0 30 Jun 2022
A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge Dustin Schwenk Apoorv Khandelwal Christopher Clark Kenneth Marino Roozbeh Mottaghi 43 530 0 03 Jun 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 282 3,583 0 02 May 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 120 865 0 07 Feb 2022
Webly Supervised Concept Expansion for General Purpose Vision Models Amita Kamath Christopher Clark Tanmay Gupta Eric Kolve Derek Hoiem Aniruddha Kembhavi VLM 52 54 0 04 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 446 4,283 0 28 Jan 2022
KAT: A Knowledge Augmented Transformer for Vision-and-Language Liangke Gui Borui Wang Qiuyuan Huang Alexander G. Hauptmann Yonatan Bisk Jianfeng Gao 46 155 0 16 Dec 2021
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 208 412 0 10 Sep 2021
Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering Man Luo Yankai Zeng Pratyay Banerjee Chitta Baral RALM 97 64 0 09 Sep 2021
Zero-shot Visual Question Answering using Knowledge Graph Zhuo Chen Jiaoyan Chen Yuxia Geng Jeff Z. Pan Zonggang Yuan Huajun Chen 30 70 0 12 Jul 2021
Multi-Modal Answer Validation for Knowledge-Based VQA Jialin Wu Jiasen Lu Ashish Sabharwal Roozbeh Mottaghi 101 144 0 23 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 681 28,659 0 26 Feb 2021
KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA Kenneth Marino Xinlei Chen Devi Parikh Abhinav Gupta Marcus Rohrbach 63 180 0 20 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 397 40,217 0 22 Oct 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 500 41,106 0 28 May 2020
Dense Passage Retrieval for Open-Domain Question Answering Vladimir Karpukhin Barlas Oğuz Sewon Min Patrick Lewis Ledell Yu Wu Sergey Edunov Danqi Chen Wen-tau Yih RALM 137 3,676 0 10 Apr 2020
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 205 2,467 0 20 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 197 3,659 0 06 Aug 2019
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge Kenneth Marino Mohammad Rastegari Ali Farhadi Roozbeh Mottaghi 57 1,050 0 31 May 2019
Pythia v0.1: the Winning Entry to the VQA Challenge 2018 Yu Jiang Vivek Natarajan Xinlei Chen Marcus Rohrbach Dhruv Batra Devi Parikh VLM 39 203 0 26 Jul 2018
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould Lei Zhang AIMat 102 4,201 0 25 Jul 2017
Billion-scale similarity search with GPUs Jeff Johnson Matthijs Douze Hervé Jégou 172 3,696 0 28 Feb 2017
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge R. Speer Joshua Chin Catherine Havasi 135 2,882 0 12 Dec 2016
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering Yash Goyal Tejas Khot D. Summers-Stay Dhruv Batra Devi Parikh CoGe 291 3,187 0 02 Dec 2016
FVQA: Fact-based Visual Question Answering Peng Wang Qi Wu Chunhua Shen Anton van den Hengel A. Dick CoGe 61 455 0 17 Jun 2016
VQA: Visual Question Answering Aishwarya Agrawal Jiasen Lu Stanislaw Antol Margaret Mitchell C. L. Zitnick Dhruv Batra Devi Parikh CoGe 150 5,421 0 03 May 2015