Unifying Vision-and-Language Tasks via Text Generation

4 February 2021

Papers citing "Unifying Vision-and-Language Tasks via Text Generation"

50 / 368 papers shown

Title
Fine-grained Visual-Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection Yanxin Long Jianhua Han Runhu Huang Xu Hang Yi Zhu Chunjing Xu Xiaodan Liang VLM ObjD 35 18 0 02 Nov 2022
Generative Negative Text Replay for Continual Vision-Language Pretraining Shipeng Yan Lanqing Hong Hang Xu Jianhua Han Tinne Tuytelaars Zhenguo Li Xuming He VLM CLL CLIP 36 17 0 31 Oct 2022
Visual Semantic Parsing: From Images to Abstract Meaning Representation M. A. Abdelsalam Zhan Shi Federico Fancellu Kalliopi Basioti Dhaivat Bhatt Vladimir Pavlovic Afsaneh Fazly GNN 37 4 0 26 Oct 2022
Towards Unifying Reference Expression Generation and Comprehension Duo Zheng Tao Kong Ya Jing Jiaan Wang Xiaojie Wang ObjD 32 6 0 24 Oct 2022
McQueen: a Benchmark for Multimodal Conversational Query Rewrite Yifei Yuan Chen Shi Runze Wang Liyi Chen Feijun Jiang Yuan You W. Lam 14 6 0 23 Oct 2022
SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation Zekun Li Jina Kim Yao-Yi Chiang Muhao Chen 90 28 0 21 Oct 2022
A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture Jiayun Luo Boyang Albert Li Cyril Leung 53 11 0 20 Oct 2022
Visual Spatial Description: Controlled Spatial-Oriented Image-to-Text Generation Yu Zhao Jianguo Wei Zhichao Lin Yueheng Sun Meishan Zhang Hao Fei 25 16 0 20 Oct 2022
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training A. M. H. Tiong Junnan Li Boyang Albert Li Silvio Savarese Guosheng Lin MLLM 29 101 0 17 Oct 2022
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training Wenliang Dai Zihan Liu Ziwei Ji Dan Su Pascale Fung MLLM VLM 32 62 0 14 Oct 2022
MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting Oscar Manas Pau Rodríguez López Saba Ahmadi Aida Nematzadeh Yash Goyal Aishwarya Agrawal VLM VPVLM 15 48 0 13 Oct 2022
ImaginaryNet: Learning Object Detectors without Real Images and Annotations Minheng Ni Zitong Huang Kai-Hua Feng W. Zuo VLM 19 15 0 13 Oct 2022
OpenCQA: Open-ended Question Answering with Charts Shankar Kantharaj Do Xuan Long Rixie Tiffany Ko Leong J. Tan Enamul Hoque Chenyu You 29 47 0 12 Oct 2022
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks Gregor Geigle Chen Cecilia Liu Jonas Pfeiffer Iryna Gurevych VLM 28 1 0 12 Oct 2022
MAMO: Masked Multimodal Modeling for Fine-Grained Vision-Language Representation Learning Zijia Zhao Longteng Guo Xingjian He Shuai Shao Zehuan Yuan Jing Liu 21 8 0 09 Oct 2022
VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment Shraman Pramanick Li Jing Sayan Nag Jiachen Zhu Hardik Shah Yann LeCun Ramalingam Chellappa 29 21 0 09 Oct 2022
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation Wanrong Zhu An Yan Yujie Lu Wenda Xu Qing Guo Miguel P. Eckstein William Yang Wang 82 37 0 07 Oct 2022
VIMA: General Robot Manipulation with Multimodal Prompts Yunfan Jiang Agrim Gupta Zichen Zhang Guanzhi Wang Yongqiang Dou Yanjun Chen Li Fei-Fei Anima Anandkumar Yuke Zhu Linxi Fan LM&Ro 28 335 0 06 Oct 2022
LAVIS: A Library for Language-Vision Intelligence Dongxu Li Junnan Li Hung Le Guangsen Wang Silvio Savarese Guosheng Lin VLM 129 51 0 15 Sep 2022
Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering Jingjing Jiang Zi-yi Liu Nanning Zheng 26 8 0 14 Sep 2022
PaLI: A Jointly-Scaled Multilingual Language-Image Model Xi Chen Tianlin Li Soravit Changpinyo A. Piergiovanni Piotr Padlewski ... Andreas Steiner A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut MLLM VLM 37 687 0 14 Sep 2022
MaXM: Towards Multilingual Visual Question Answering Soravit Changpinyo Linting Xue Michal Yarom Ashish V. Thapliyal Idan Szpektor J. Amelot Xi Chen Radu Soricut 33 8 0 12 Sep 2022
Multimedia Generative Script Learning for Task Planning Qingyun Wang Manling Li Hou Pong Chan Lifu Huang J. Hockenmaier Girish Chowdhary Heng Ji VGen 38 10 0 25 Aug 2022
Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in Radiology Sangjoon Park Eunha Lee Kyung Sook Shin Jeonghyeon Lee Jong Chul Ye 27 2 0 10 Aug 2022
Prompt Tuning for Generative Multimodal Pretrained Models Han Yang Junyang Lin An Yang Peng Wang Chang Zhou Hongxia Yang VLM LRM VPVLM 37 30 0 04 Aug 2022
Masked Vision and Language Modeling for Multi-modal Representation Learning Gukyeong Kwon Zhaowei Cai Avinash Ravichandran Erhan Bas Rahul Bhotika Stefano Soatto 36 67 0 03 Aug 2022
Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics Xiaoyuan Guo Jiali Duan C.-C. Jay Kuo J. Gichoya Imon Banerjee VLM 22 1 0 31 Jul 2022
LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injection Zhuo Chen Yufen Huang Jiaoyan Chen Yuxia Geng Yin Fang Jeff Z. Pan Ningyu Zhang Wen Zhang 26 35 0 26 Jul 2022
Unifying Event Detection and Captioning as Sequence Generation via Pre-Training Qi Zhang Yuqing Song Qin Jin 30 24 0 18 Jul 2022
Vision-and-Language Pretraining Thong Nguyen Cong-Duy Nguyen Xiaobao Wu See-Kiong Ng A. Luu VLM CLIP 24 2 0 05 Jul 2022
e-CLIP: Large-Scale Vision-Language Representation Learning in E-commerce Wonyoung Shin Jonghun Park Taekang Woo Yongwoo Cho Kwangjin Oh Hwanjun Song VLM 27 16 0 01 Jul 2022
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation Jiahui Yu Yuanzhong Xu Jing Yu Koh Thang Luong Gunjan Baid ... Zarana Parekh Xin Li Han Zhang Jason Baldridge Yonghui Wu EGVM 107 1,066 0 22 Jun 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 56 392 0 17 Jun 2022
BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning Xiao Xu Chenfei Wu Shachar Rosenman Vasudev Lal Wanxiang Che Nan Duan 48 64 0 17 Jun 2022
MixGen: A New Multi-Modal Data Augmentation Xiaoshuai Hao Yi Zhu Srikar Appalaraju Aston Zhang Wanqian Zhang Boyang Li Mu Li VLM 22 83 0 16 Jun 2022
Write and Paint: Generative Vision-Language Models are Unified Modal Learners Shizhe Diao Wangchunshu Zhou Xinsong Zhang Jiawei Wang MLLM AI4CE 22 16 0 15 Jun 2022
Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone Zi-Yi Dou Aishwarya Kamath Zhe Gan Pengchuan Zhang Jianfeng Wang ... Ce Liu Yann LeCun Nanyun Peng Jianfeng Gao Lijuan Wang VLM ObjD 30 124 0 15 Jun 2022
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling Linjie Li Zhe Gan Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Ce Liu Lijuan Wang MLLM VLM 20 81 0 14 Jun 2022
LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM 21 236 0 13 Jun 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 72 527 0 13 Jun 2022
Language Models are General-Purpose Interfaces Y. Hao Haoyu Song Li Dong Shaohan Huang Zewen Chi Wenhui Wang Shuming Ma Furu Wei MLLM 30 95 0 13 Jun 2022
Revealing Single Frame Bias for Video-and-Language Learning Jie Lei Tamara L. Berg Joey Tianyi Zhou 24 111 0 07 Jun 2022
Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning Yujia Xie Luowei Zhou Xiyang Dai Lu Yuan Nguyen Bach Ce Liu Michael Zeng VLM MLLM 34 28 0 03 Jun 2022
VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models Wangchunshu Zhou Yan Zeng Shizhe Diao Xinsong Zhang CoGe VLM 32 13 0 30 May 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 59 528 0 27 May 2022
Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization Aishwarya Agrawal Ivana Kajić Emanuele Bugliarello Elnaz Davoodi Anita Gergely Phil Blunsom Aida Nematzadeh OOD 40 17 0 24 May 2022
HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval Feilong Chen Xiuyi Chen Jiaxin Shi Duzhen Zhang Jianlong Chang Qi Tian VLM CLIP 34 6 0 24 May 2022
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections Chenliang Li Haiyang Xu Junfeng Tian Wei Wang Ming Yan ... Ji Zhang Songfang Huang Feiran Huang Jingren Zhou Luo Si VLM MLLM 33 213 0 24 May 2022
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization Shruti Palaskar Akshita Bhagia Yonatan Bisk Florian Metze A. Black Ana Marasović 18 4 0 24 May 2022
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models Yuan Yao Qi-An Chen Ao Zhang Wei Ji Zhiyuan Liu Tat-Seng Chua Maosong Sun VLM MLLM 26 38 0 23 May 2022