Large Language Models are Visual Reasoning Coordinators

23 October 2023

Bo Li

Ziwei Liu

Papers citing "Large Language Models are Visual Reasoning Coordinators"

50 / 64 papers shown

Title
Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering Zeqing Wang Wentao Wan Qiqing Lao Runmeng Chen Minjie Lang Keze Wang Liang Lin Liang Lin LRM 189 3 0 17 Feb 2025
Meta-Feature Adapter: Integrating Environmental Metadata for Enhanced Animal Re-identification Yuzhuo Li Di Zhao Yihao Wu Yun Sing Koh 105 0 0 23 Jan 2025
SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generation Hang Zhang Zhuoling Li Jun Liu LRM 139 1 0 15 Dec 2024
AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations? Shouwei Ruan Hanqin Liu Yao Huang Xiaoqi Wang Caixin Kang Hang Su Yinpeng Dong Xingxing Wei VGen 162 0 0 04 Dec 2024
STAR: A Benchmark for Situated Reasoning in Real-World Videos Bo Wu Shoubin Yu Zhenfang Chen Joshua B. Tenenbaum Chuang Gan 110 192 0 15 May 2024
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 332 4,298 0 09 Jun 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 104 2,049 0 11 May 2023
Abstract Visual Reasoning with Tangram Shapes Anya Ji Noriyuki Kojima N. Rush Alane Suhr Wai Keen Vong Robert D. Hawkins Yoav Artzi LRM 39 36 0 29 Nov 2022
PromptCap: Prompt-Guided Task-Aware Image Captioning Yushi Hu Hang Hua Zhengyuan Yang Weijia Shi Noah A. Smith Jiebo Luo 86 103 0 15 Nov 2022
Composing Ensembles of Pre-trained Models via Iterative Consensus Shuang Li Yilun Du J. Tenenbaum Antonio Torralba Igor Mordatch MoMe 49 24 0 20 Oct 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 181 3,117 0 20 Oct 2022
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training A. M. H. Tiong Junnan Li Boyang Albert Li Silvio Savarese Guosheng Lin MLLM 70 107 0 17 Oct 2022
Patching open-vocabulary models by interpolating weights Gabriel Ilharco Mitchell Wortsman S. Gadre Shuran Song Hannaneh Hajishirzi Simon Kornblith Ali Farhadi Ludwig Schmidt VLM KELM 84 175 0 10 Aug 2022
LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injection Zhuo Chen Yufen Huang Jiaoyan Chen Yuxia Geng Yin Fang Jeff Z. Pan Ningyu Zhang Wen Zhang 62 36 0 26 Jul 2022
Reasoning about Actions over Visual and Linguistic Modalities: A Survey Shailaja Keyur Sampat Maitreya Patel Subhasish Das Yezhou Yang Chitta Baral ReLM LM&Ro LRM 49 12 0 15 Jul 2022
Recurrent Memory Transformer Aydar Bulatov Yuri Kuratov Andrey Kravchenko CLL 39 108 0 14 Jul 2022
Rationale-Augmented Ensembles in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Denny Zhou ReLM LRM 89 126 0 02 Jul 2022
Language Models are General-Purpose Interfaces Y. Hao Haoyu Song Li Dong Shaohan Huang Zewen Chi Wenhui Wang Shuming Ma Furu Wei MLLM 58 98 0 13 Jun 2022
A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge Dustin Schwenk Apoorv Khandelwal Christopher Clark Kenneth Marino Roozbeh Mottaghi 62 540 0 03 Jun 2022
Visual Spatial Reasoning Fangyu Liu Guy Edward Toh Emerson Nigel Collier ReLM 90 175 0 30 Apr 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 371 3,535 0 29 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 471 6,231 0 05 Apr 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Andy Zeng Maria Attarian Brian Ichter K. Choromanski Adrian S. Wong ... Michael S. Ryoo Vikas Sindhwani Johnny Lee Vincent Vanhoucke Peter R. Florence ReLM LRM 142 582 0 01 Apr 2022
UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training Daniel Khashabi Yeganeh Kordi Hannaneh Hajishirzi 65 67 0 23 Feb 2022
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 284 355 0 18 Feb 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 144 873 0 07 Feb 2022
Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices Mikolaj Malkiñski Jacek Mańdziuk 162 43 0 28 Jan 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 524 4,343 0 28 Jan 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 799 9,351 0 28 Jan 2022
KAT: A Knowledge Augmented Transformer for Vision-and-Language Liangke Gui Borui Wang Qiuyuan Huang Alexander G. Hauptmann Yonatan Bisk Jianfeng Gao 66 157 0 16 Dec 2021
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts Hangbo Bao Wenhui Wang Li Dong Qiang Liu Owais Khan Mohammed Kriti Aggarwal Subhojit Som Furu Wei VLM MLLM MoE 69 553 0 03 Nov 2021
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 190 749 0 03 Nov 2021
MetaICL: Learning to Learn In Context Sewon Min M. Lewis Luke Zettlemoyer Hannaneh Hajishirzi LRM 206 488 0 29 Oct 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 342 1,701 0 15 Oct 2021
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 249 419 0 10 Sep 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 184 3,743 0 03 Sep 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 224 5,518 0 07 Jul 2021
Scaling Vision with Sparse Mixture of Experts C. Riquelme J. Puigcerver Basil Mustafa Maxim Neumann Rodolphe Jenatton André Susano Pinto Daniel Keysers N. Houlsby MoE 101 600 0 10 Jun 2021
Evaluating Deep Neural Network Ensembles by Majority Voting cum Meta-Learning scheme Anmol Jain A. Kumar Seba Susan FedML 13 12 0 09 May 2021
Cross-Task Generalization via Natural Language Crowdsourcing Instructions Swaroop Mishra Daniel Khashabi Chitta Baral Hannaneh Hajishirzi LRM 141 746 0 18 Apr 2021
BASE Layers: Simplifying Training of Large, Sparse Models M. Lewis Shruti Bhosale Tim Dettmers Naman Goyal Luke Zettlemoyer MoE 191 278 0 30 Mar 2021
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision Wonjae Kim Bokyung Son Ildoo Kim VLM CLIP 116 1,745 0 05 Feb 2021
KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA Kenneth Marino Xinlei Chen Devi Parikh Abhinav Gupta Marcus Rohrbach 92 182 0 20 Dec 2020
Open-Retrieval Conversational Question Answering Chen Qu Liu Yang Cen Chen Minghui Qiu W. Bruce Croft Mohit Iyyer RALM 66 175 0 22 May 2020
StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics Angelos Chatzimparmpas Rafael M. Martins Kostiantyn Kucher Andreas Kerren DiffM BDL 45 78 0 04 May 2020
TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages J. Clark Eunsol Choi Michael Collins Dan Garrette Tom Kwiatkowski Vitaly Nikolaev J. Palomaki 141 609 0 10 Mar 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 344 1,613 0 21 Jan 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 419 20,127 0 23 Oct 2019
CLEVRER: CoLlision Events for Video REpresentation and Reasoning Kexin Yi Yuta Saito Yunzhu Li Pushmeet Kohli Jiajun Wu Antonio Torralba J. Tenenbaum NAI 115 473 0 03 Oct 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,193 0 27 Aug 2019