v1v2 (latest)

Advancing Multimodal In-Context Learning in Large Vision-Language Models with Task-aware Demonstrations

5 March 2025

Papers citing "Advancing Multimodal In-Context Learning in Large Vision-Language Models with Task-aware Demonstrations"

34 / 34 papers shown

Title
CAMA: Enhancing Multimodal In-Context Learning with Context-Aware Modulated Attention Yanshu Li JianJiang Yang Bozheng Li Ruixiang Tang 46 2 0 21 May 2025
TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configuration Yanshu Li Tian Yun Jianjiang Yang Pinyuan Feng Jinfa Huang Ruixiang Tang 52 2 0 21 May 2025
Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning Brandon Huang Chancharik Mitra Assaf Arbelle Leonid Karlinsky Trevor Darrell Roei Herzig 88 21 0 21 Jun 2024
ICLEval: Evaluating In-Context Learning Ability of Large Language Models Wentong Chen Yankai Lin ZhenHao Zhou HongYun Huang Yantao Jia Bo Zhao Ji-Rong Wen ELM 71 4 0 21 Jun 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 113 637 0 25 Apr 2024
Visual In-Context Learning for Large Vision-Language Models Yucheng Zhou Xiang Li Qianning Wang Jianbing Shen MLLM 69 63 0 18 Feb 2024
Comparable Demonstrations are Important in In-Context Learning: A Novel Perspective on Demonstration Selection Caoyun Fan Jidong Tian Yitian Li Hao He Yaohui Jin 48 5 0 12 Dec 2023
How to Configure Good In-Context Sequence for Visual Question Answering Li Li Jiawei Peng Huiyi Chen Chongyang Gao Xu Yang MLLM 81 22 0 04 Dec 2023
Meta-Adapter: An Online Few-shot Learner for Vision-Language Model Cheng Cheng Lin Song Ruoyi Xue Hang Wang Hongbin Sun Yixiao Ge Ying Shan VLM ObjD 91 25 0 07 Nov 2023
An Early Evaluation of GPT-4V(ision) Yang Wu Shilong Wang Hao Yang Tian Zheng Hongbo Zhang Yanyan Zhao Bing Qin MLLM ELM 46 39 0 25 Oct 2023
Multimodal Neurons in Pretrained Text-Only Transformers Sarah Schwettmann Neil Chowdhury Samuel J. Klein David Bau Antonio Torralba MILM 66 32 0 03 Aug 2023
OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models Anas Awadalla Irena Gao Josh Gardner Jack Hessel Yusuf Hanafy ... Simon Kornblith Pang Wei Koh Gabriel Ilharco Mitchell Wortsman Ludwig Schmidt MLLM 118 431 0 02 Aug 2023
Learning to Retrieve In-Context Examples for Large Language Models Liang Wang Nan Yang Furu Wei RALM 66 43 0 14 Jul 2023
Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations Chenglei Si Dan Friedman Nitish Joshi Shi Feng Danqi Chen He He 63 48 0 22 May 2023
What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning Jane Pan Tianyu Gao Howard Chen Danqi Chen 74 126 0 16 May 2023
Larger language models do in-context learning differently Jerry W. Wei Jason W. Wei Yi Tay Dustin Tran Albert Webson ... Xinyun Chen Hanxiao Liu Da Huang Denny Zhou Tengyu Ma ReLM LRM 104 374 0 07 Mar 2023
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering Zhiyong Wu Yaoxiang Wang Jiacheng Ye Lingpeng Kong 84 138 0 20 Dec 2022
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations Xinxi Lyu Sewon Min Iz Beltagy Luke Zettlemoyer Hannaneh Hajishirzi VLM 58 68 0 19 Dec 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 141 513 0 01 Aug 2022
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models Denny Zhou Nathanael Scharli Le Hou Jason W. Wei Nathan Scales ... Dale Schuurmans Claire Cui Olivier Bousquet Quoc Le Ed H. Chi RALM LRM AI4CE 97 1,130 0 21 May 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 165 1,495 0 25 Feb 2022
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 224 3,989 0 28 Jul 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 183 788 0 25 Jun 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 409 1,194 0 18 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 584 4,084 0 18 Apr 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 390 1,389 0 17 Jan 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 404 1,972 0 31 Dec 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 862 42,379 0 28 May 2020
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes Douwe Kiela Hamed Firooz Aravind Mohan Vedanuj Goswami Amanpreet Singh Pratik Ringshia Davide Testuggine 99 608 0 10 May 2020
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge Kenneth Marino Mohammad Rastegari Ali Farhadi Roozbeh Mottaghi 117 1,090 0 31 May 2019
VizWiz Grand Challenge: Answering Visual Questions from Blind People Danna Gurari Qing Li Abigale Stangl Anhong Guo Chi Lin Kristen Grauman Jiebo Luo Jeffrey P. Bigham CoGe 111 861 0 22 Feb 2018
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering Yash Goyal Tejas Khot D. Summers-Stay Dhruv Batra Devi Parikh CoGe 347 3,270 0 02 Dec 2016
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 297 4,508 0 20 Nov 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 424 43,814 0 01 May 2014