Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers

3 January 2024

Papers citing "Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers"

38 / 38 papers shown

Title
BlendRL: A Framework for Merging Symbolic and Neural Policy Learning Hikaru Shindo Quentin Delfosse Devendra Singh Dhami Kristian Kersting 84 5 0 15 Oct 2024
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents Shilong Liu Hao Cheng Haotian Liu Hao Zhang Feng Li ... Hang Su Jun Zhu Lei Zhang Jianfeng Gao Chun-yue Li MLLM VLM 76 119 0 09 Nov 2023
Graph of Thoughts: Solving Elaborate Problems with Large Language Models Maciej Besta Nils Blach Aleš Kubíček Robert Gerstenberger Michal Podstawski ... Joanna Gajda Tomasz Lehmann H. Niewiadomski Piotr Nyczyk Torsten Hoefler LRM AI4CE LM&Ro 104 662 0 18 Aug 2023
Modular Visual Question Answering via Code Generation Sanjay Subramanian Medhini Narasimhan Kushal Khangaonkar Kevin Kaichuang Yang Arsha Nagrani Cordelia Schmid Andy Zeng Trevor Darrell Dan Klein 52 50 0 08 Jun 2023
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models Lei Wang Wanyu Xu Yihuai Lan Zhiqiang Hu Yunshi Lan Roy Ka-wei Lee Ee-Peng Lim ReLM LRM 105 344 0 06 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 265 945 0 27 Apr 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 128 1,119 0 27 Mar 2023
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin E. Azarnasab Faisal Ahmed Zicheng Liu Ce Liu Michael Zeng Lijuan Wang ReLM KELM LRM 85 383 0 20 Mar 2023
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning Zhiyang Xu Ying Shen Lifu Huang MLLM 81 117 0 21 Dec 2022
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 168 1,603 0 15 Dec 2022
GPS: Genetic Prompt Search for Efficient Few-shot Learning Hanwei Xu Yujun Chen Yulun Du Nan Shao Yanggang Wang Haiyu Li Zhilin Yang VLM 45 31 0 31 Oct 2022
Code4Struct: Code Generation for Few-Shot Event Structure Prediction Xingyao Wang Sha Li Heng Ji 59 82 0 23 Oct 2022
Vision-Language Pre-training: Basics, Recent Advances, and Future Trends Zhe Gan Linjie Li Chunyuan Li Lijuan Wang Zicheng Liu Jianfeng Gao VLM 51 164 0 17 Oct 2022
Language Models of Code are Few-Shot Commonsense Learners Aman Madaan Shuyan Zhou Uri Alon Yiming Yang Graham Neubig ReLM LRM 90 218 0 13 Oct 2022
When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul Federico Bianchi Pratyusha Kalluri Dan Jurafsky James Zou VLM CoGe 62 387 0 04 Oct 2022
PaLI: A Jointly-Scaled Multilingual Language-Image Model Xi Chen Tianlin Li Soravit Changpinyo A. Piergiovanni Piotr Padlewski ... Andreas Steiner A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut MLLM VLM 79 711 0 14 Sep 2022
VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relations Tiancheng Zhao Tianqi Zhang Mingwei Zhu Haozhan Shen Kyusong Lee Xiaopeng Lu Jianwei Yin VLM CoGe MLLM 89 97 0 01 Jul 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 123 546 0 27 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 320 3,515 0 29 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 416 6,202 0 05 Apr 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Andy Zeng Maria Attarian Brian Ichter K. Choromanski Adrian S. Wong ... Michael S. Ryoo Vikas Sindhwani Johnny Lee Vincent Vanhoucke Peter R. Florence ReLM LRM 131 581 0 01 Apr 2022
GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models Archiki Prasad Peter Hase Xiang Zhou Joey Tianyi Zhou 90 123 0 14 Mar 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 135 870 0 07 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 738 9,267 0 28 Jan 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 122 1,589 0 20 Jan 2022
Grounded Language-Image Pre-training Liunian Harold Li Pengchuan Zhang Haotian Zhang Jianwei Yang Chunyuan Li ... Lu Yuan Lei Zhang Lei Li Kai-Wei Chang Jianfeng Gao ObjD VLM 87 1,058 0 07 Dec 2021
Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts Yan Zeng Xinsong Zhang Hang Li VLM CLIP 51 303 0 16 Nov 2021
A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models Woojeong Jin Yu Cheng Yelong Shen Weizhu Chen Xiang Ren VLM VPVLM MLLM 54 136 0 16 Oct 2021
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models Yuan Yao Ao Zhang Zhengyan Zhang Zhiyuan Liu Tat-Seng Chua Maosong Sun MLLM VPVLM VLM 277 222 0 24 Sep 2021
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 236 417 0 10 Sep 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 201 5,454 0 07 Jul 2021
Probing Image-Language Transformers for Verb Understanding Lisa Anne Hendricks Aida Nematzadeh 53 118 0 16 Jun 2021
Big Transfer (BiT): General Visual Representation Learning Alexander Kolesnikov Lucas Beyer Xiaohua Zhai J. Puigcerver Jessica Yung Sylvain Gelly N. Houlsby MQ 252 1,204 0 24 Dec 2019
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark Xiaohua Zhai J. Puigcerver Alexander Kolesnikov P. Ruyssen C. Riquelme ... Michael Tschannen Marcin Michalski Olivier Bousquet Sylvain Gelly N. Houlsby SSL 73 438 0 01 Oct 2019
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer René Ranftl Katrin Lasinger David Hafner Konrad Schindler V. Koltun MDE 188 1,781 0 02 Jul 2019
Inferring and Executing Programs for Visual Reasoning Justin Johnson B. Hariharan Laurens van der Maaten Judy Hoffman Li Fei-Fei C. L. Zitnick Ross B. Girshick NAI 69 545 0 10 May 2017
Learning to Reason: End-to-End Module Networks for Visual Question Answering Ronghang Hu Jacob Andreas Marcus Rohrbach Trevor Darrell Kate Saenko KELM GNN ReLM LRM 118 577 0 18 Apr 2017
Neural Module Networks Jacob Andreas Marcus Rohrbach Trevor Darrell Dan Klein CoGe 129 1,071 0 09 Nov 2015