KAT: A Knowledge Augmented Transformer for Vision-and-Language

16 December 2021

Liangke Gui

Borui Wang

Qiuyuan Huang

Alexander G. Hauptmann

Yonatan Bisk

Jianfeng Gao

ArXiv PDF HTML

Papers citing "KAT: A Knowledge Augmented Transformer for Vision-and-Language"

35 / 35 papers shown

Title
Structural-Temporal Coupling Anomaly Detection with Dynamic Graph Transformer Chang Zong Yueting Zhuang Jian Shao Weiming Lu 44 0 0 13 May 2025
Fine-Grained Retrieval-Augmented Generation for Visual Question Answering Zhengxuan Zhang Yin Wu Yuyu Luo Nan Tang 38 0 0 28 Feb 2025
FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQA S M Sarwar 80 1 0 25 Feb 2025
Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering Zeqing Wang Wentao Wan Qiqing Lao Runmeng Chen Minjie Lang Keze Wang Liang Lin Liang Lin LRM 103 3 0 17 Feb 2025
Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning Zijian Guo Weichao Zhou Wenchao Li OffRL 102 2 0 28 Jan 2025
Combining Knowledge Graph and LLMs for Enhanced Zero-shot Visual Question Answering Qian Tao Xiaoyang Fan Yong Xu Xingquan Zhu Yufei Tang 50 0 0 22 Jan 2025
CUE-M: Contextual Understanding and Enhanced Search with Multimodal Large Language Model Dongyoung Go Taesun Whang Chanhee Lee Hwayeon Kim Sunghoon Park Seunghwan Ji Dongchan Kim Young-Bum Kim Young-Bum Kim LRM 198 1 0 19 Nov 2024
Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent Yangning Li Hai-Tao Zheng Xinyu Wang Yong Jiang Zhen Zhang ... Hui Wang Hai-Tao Zheng Pengjun Xie Philip S. Yu Fei Huang 65 16 0 05 Nov 2024
Human-inspired Perspectives: A Survey on AI Long-term Memory Zihong He Weizhe Lin Hao Zheng Fan Zhang Matt Jones Laurence Aitchison X. Xu Miao Liu Per Ola Kristensson Junxiao Shen 77 2 0 01 Nov 2024
Disentangling Knowledge-based and Visual Reasoning by Question Decomposition in KB-VQA Elham J. Barezi Parisa Kordjamshidi CoGe 37 0 0 27 Jun 2024
Inquire, Interact, and Integrate: A Proactive Agent Collaborative Framework for Zero-Shot Multimodal Medical Reasoning Zishan Gu Fenglin Liu Changchang Yin Ping Zhang LRM LM&MA 58 0 0 19 May 2024
Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Models Alireza Salemi Hamed Zamani 36 4 0 30 Apr 2024
Self-Bootstrapped Visual-Language Model for Knowledge Selection and Question Answering Dongze Hao Qunbo Wang Longteng Guo Jie Jiang Jing Liu 36 0 0 22 Apr 2024
Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning F. Mumuni A. Mumuni AAML 37 5 0 11 Mar 2024
Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Prompts Yunshi Lan Xiang Li Xin Liu Yang Li Wei Qin Weining Qian LRM ReLM 41 24 0 15 Nov 2023
Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks Sherzod Hakimov David Schlangen VLM 36 5 0 23 May 2023
Combo of Thinking and Observing for Outside-Knowledge VQA Q. Si Yuchen Mo Zheng Lin Huishan Ji Weiping Wang 46 13 0 10 May 2023
ViperGPT: Visual Inference via Python Execution for Reasoning Dídac Surís Sachit Menon Carl Vondrick MLLM LRM ReLM 45 431 0 14 Mar 2023
The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges Maria Lymperaiou Giorgos Stamou VLM 32 4 0 04 Mar 2023
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering Zhou Yu Xuecheng Ouyang Zhenwei Shao Mei Wang Jun Yu MLLM 94 11 0 03 Mar 2023
Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions? Yang Chen Hexiang Hu Yi Luan Haitian Sun Soravit Changpinyo Alan Ritter Ming-Wei Chang 48 80 0 23 Feb 2023
Few-shot Multimodal Multitask Multilingual Learning Aman Chadha Vinija Jain 53 0 0 19 Feb 2023
Prompting for Multimodal Hateful Meme Classification Rui Cao Roy Ka-Wei Lee Wen-Haw Chong Jing Jiang VLM 25 75 0 08 Feb 2023
Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering Chenxi Whitehouse Tillman Weyde Pranava Madhyastha LRM 44 3 0 25 Jan 2023
See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning Zhenfang Chen Qinhong Zhou Songlin Yang Yining Hong Hao Zhang Chuang Gan LRM VLM 35 35 0 12 Jan 2023
Rethinking with Retrieval: Faithful Large Language Model Inference Hangfeng He Hongming Zhang Dan Roth KELM LRM 141 158 0 31 Dec 2022
REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory Ziniu Hu Ahmet Iscen Chen Sun Zirui Wang Kai-Wei Chang Yizhou Sun Cordelia Schmid David A. Ross Alireza Fathi RALM VLM 40 89 0 10 Dec 2022
PromptCap: Prompt-Guided Task-Aware Image Captioning Yushi Hu Hang Hua Zhengyuan Yang Weijia Shi Noah A. Smith Jiebo Luo 51 101 0 15 Nov 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 56 393 0 17 Jun 2022
REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering Yuanze Lin Yujia Xie Dongdong Chen Yichong Xu Chenguang Zhu Lu Yuan 47 71 0 02 Jun 2022
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization Shruti Palaskar Akshita Bhagia Yonatan Bisk Florian Metze A. Black Ana Marasović 18 4 0 24 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 46 3,349 0 29 Apr 2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals Payal Bajaj Chenyan Xiong Guolin Ke Xiaodong Liu Di He Saurabh Tiwary Tie-Yan Liu Paul N. Bennett Xia Song Jianfeng Gao 50 32 0 13 Apr 2022
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 180 402 0 10 Sep 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 322 3,708 0 11 Feb 2021