GIT: A Generative Image-to-text Transformer for Vision and Language

27 May 2022

Zicheng Liu

Papers citing "GIT: A Generative Image-to-text Transformer for Vision and Language"

50 / 406 papers shown

Title
Tag2Text: Guiding Vision-Language Model via Image Tagging Xinyu Huang Youcai Zhang Jinyu Ma Weiwei Tian Rui Feng Yuejie Zhang Yaqian Li Yandong Guo Lei Zhang CLIP MLLM VLM 3DV 63 74 0 10 Mar 2023
Interpretable Visual Question Answering Referring to Outside Knowledge He Zhu Ren Togo Takahiro Ogawa Miki Haseyama 18 0 0 08 Mar 2023
Prismer: A Vision-Language Model with Multi-Task Experts Shikun Liu Linxi Fan Edward Johns Zhiding Yu Chaowei Xiao Anima Anandkumar VLM MLLM 44 21 0 04 Mar 2023
Language Is Not All You Need: Aligning Perception with Language Models Shaohan Huang Li Dong Wenhui Wang Y. Hao Saksham Singhal ... Johan Bjorck Vishrav Chaudhary Subhojit Som Xia Song Furu Wei VLM LRM MLLM 23 535 0 27 Feb 2023
HL Dataset: Visually-grounded Description of Scenes, Actions and Rationales Michele Cafagna Kees van Deemter Albert Gatt 3DV 13 4 0 23 Feb 2023
Prompt Stealing Attacks Against Text-to-Image Generation Models Xinyue Shen Y. Qu Michael Backes Yang Zhang 30 32 0 20 Feb 2023
Knowledge from Large-Scale Protein Contact Prediction Models Can Be Transferred to the Data-Scarce RNA Contact Prediction Task Yiren Jian Chongyang Gao Chen Zeng Yunjie Zhao Soroush Vosoughi 24 0 0 13 Feb 2023
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning Zhuolin Yang Ming-Yu Liu Zihan Liu V. Korthikanti Weili Nie ... Yuke Zhu M. Shoeybi Bryan Catanzaro Chaowei Xiao Anima Anandkumar VLM RALM 34 39 0 09 Feb 2023
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video Haiyang Xu Qinghao Ye Mingshi Yan Yaya Shi Jiabo Ye ... Guohai Xu Ji Zhang Songfang Huang Feiran Huang Jingren Zhou MLLM VLM MoE 40 160 0 01 Feb 2023
AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation Bjorn Deiseroth Mayukh Deb Samuel Weinbach Manuel Brack P. Schramowski Kristian Kersting 21 22 0 19 Jan 2023
CLIPTER: Looking at the Bigger Picture in Scene Text Recognition Aviad Aberdam David Bensaid Alona Golts Roy Ganz Oren Nuriel Royee Tichauer Shai Mazor Ron Litman VLM CLIP 24 12 0 18 Jan 2023
Towards Models that Can See and Read Roy Ganz Oren Nuriel Aviad Aberdam Yair Kittenplon Shai Mazor Ron Litman 24 13 0 18 Jan 2023
HierVL: Learning Hierarchical Video-Language Embeddings Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman VLM AI4TS 22 52 0 05 Jan 2023
Noise-aware Learning from Web-crawled Image-Text Data for Image Captioning Woohyun Kang Jonghwan Mun Sungjun Lee Byungseok Roh VLM 8 18 0 27 Dec 2022
Generalized Decoding for Pixel, Image, and Language Xueyan Zou Zi-Yi Dou Jianwei Yang Zhe Gan Linjie Li ... Lu Yuan Nanyun Peng Lijuan Wang Yong Jae Lee Jianfeng Gao VLM MLLM ObjD 21 241 0 21 Dec 2022
Position-guided Text Prompt for Vision-Language Pre-training Alex Jinpeng Wang Pan Zhou Mike Zheng Shou Shuicheng Yan VLM 24 37 0 19 Dec 2022
CLIPPO: Image-and-Language Understanding from Pixels Only Michael Tschannen Basil Mustafa N. Houlsby CLIP VLM 32 47 0 15 Dec 2022
VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners Shen Yan Tao Zhu Zirui Wang Yuan Cao Mi Zhang Soham Ghosh Yonghui Wu Jiahui Yu VLM VGen 32 46 0 09 Dec 2022
GRiT: A Generative Region-to-text Transformer for Object Understanding Jialian Wu Jianfeng Wang Zhengyuan Yang Zhe Gan Zicheng Liu Junsong Yuan Lijuan Wang ObjD VLM 14 112 0 01 Dec 2022
MPT: Mesh Pre-Training with Transformers for Human Pose and Mesh Reconstruction Kevin Qinghong Lin Chung-Ching Lin Lin Liang Zicheng Liu Lijuan Wang 3DH 49 14 0 24 Nov 2022
ReCo: Region-Controlled Text-to-Image Generation Zhengyuan Yang Jianfeng Wang Zhe Gan Linjie Li Kevin Qinghong Lin ... Nan Duan Zicheng Liu Ce Liu Michael Zeng Lijuan Wang DiffM 56 140 0 23 Nov 2022
Exploring Discrete Diffusion Models for Image Captioning Zixin Zhu Yixuan Wei Jianfeng Wang Zhe Gan Zheng-Wei Zhang Le Wang G. Hua Lijuan Wang Zicheng Liu Han Hu DiffM VLM 28 17 0 21 Nov 2022
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks Hao Li Jinguo Zhu Xiaohu Jiang Xizhou Zhu Hongsheng Li ... Xiaohua Wang Yu Qiao Xiaogang Wang Wenhai Wang Jifeng Dai MLLM 26 55 0 17 Nov 2022
Feedback is Needed for Retakes: An Explainable Poor Image Notification Framework for the Visually Impaired Kazuya Ohata Shunsuke Kitada Hitoshi Iyatomi 19 0 0 17 Nov 2022
PromptCap: Prompt-Guided Task-Aware Image Captioning Yushi Hu Hang Hua Zhengyuan Yang Weijia Shi Noah A. Smith Jiebo Luo 45 101 0 15 Nov 2022
Large-Scale Bidirectional Training for Zero-Shot Image Captioning Taehoon Kim Mark A Marsden Pyunghwan Ahn Sangyun Kim Sihaeng Lee Alessandra Sala S. Kim VLM 27 4 0 13 Nov 2022
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods Evan Crothers Nathalie Japkowicz H. Viktor DeLMO 38 107 0 13 Oct 2022
VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment Shraman Pramanick Li Jing Sayan Nag Jiachen Zhu Hardik Shah Yann LeCun Ramalingam Chellappa 29 21 0 09 Oct 2022
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding Kenton Lee Mandar Joshi Iulia Turc Hexiang Hu Fangyu Liu Julian Martin Eisenschlos Urvashi Khandelwal Peter Shaw Ming-Wei Chang Kristina Toutanova CLIP VLM 169 263 0 07 Oct 2022
SmallCap: Lightweight Image Captioning Prompted with Retrieval Augmentation R. Ramos Bruno Martins Desmond Elliott Yova Kementchedjhieva VLM 30 86 0 30 Sep 2022
REST: REtrieve & Self-Train for generative action recognition Adrian Bulat Enrique Sanchez Brais Martínez Georgios Tzimiropoulos VLM 29 4 0 29 Sep 2022
Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation Xuesu Xiao Tingnan Zhang K. Choromanski Edward J. Lee Anthony G. Francis ... Leila Takayama Roy Frostig Jie Tan Carolina Parada Vikas Sindhwani 75 54 0 22 Sep 2022
PaLI: A Jointly-Scaled Multilingual Language-Image Model Xi Chen Tianlin Li Soravit Changpinyo A. Piergiovanni Piotr Padlewski ... Andreas Steiner A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut MLLM VLM 37 687 0 14 Sep 2022
PreSTU: Pre-Training for Scene-Text Understanding Jihyung Kil Soravit Changpinyo Xi Chen Hexiang Hu Sebastian Goodman Wei-Lun Chao Radu Soricut VLM 135 29 0 12 Sep 2022
MaXM: Towards Multilingual Visual Question Answering Soravit Changpinyo Linting Xue Michal Yarom Ashish V. Thapliyal Idan Szpektor J. Amelot Xi Chen Radu Soricut 33 8 0 12 Sep 2022
Statistical Foundation Behind Machine Learning and Its Impact on Computer Vision Lei Zhang H. Shum VLM SSL 22 2 0 06 Sep 2022
Exploiting Multiple Sequence Lengths in Fast End to End Training for Image Captioning J. Hu Roberto Cavicchioli Alessandro Capotondi 26 21 0 13 Aug 2022
NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis Chenfei Wu Jian Liang Xiaowei Hu Zhe Gan Jianfeng Wang Lijuan Wang Zicheng Liu Yuejian Fang Nan Duan VGen 27 72 0 20 Jul 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 56 392 0 17 Jun 2022
BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning Xiao Xu Chenfei Wu Shachar Rosenman Vasudev Lal Wanxiang Che Nan Duan 48 64 0 17 Jun 2022
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections Chenliang Li Haiyang Xu Junfeng Tian Wei Wang Ming Yan ... Ji Zhang Songfang Huang Feiran Huang Jingren Zhou Luo Si VLM MLLM 30 212 0 24 May 2022
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners Zhenhailong Wang Manling Li Ruochen Xu Luowei Zhou Jie Lei ... Chenguang Zhu Derek Hoiem Shih-Fu Chang Joey Tianyi Zhou Heng Ji MLLM VLM 170 137 0 22 May 2022
All in One: Exploring Unified Video-Language Pre-training Alex Jinpeng Wang Yixiao Ge Rui Yan Yuying Ge Xudong Lin Guanyu Cai Jianping Wu Ying Shan Xiaohu Qie Mike Zheng Shou 33 200 0 14 Mar 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 392 4,137 0 28 Jan 2022
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Faisal Ahmed Zicheng Liu Yumao Lu Lijuan Wang 27 111 0 23 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,443 0 11 Nov 2021
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 180 402 0 10 Sep 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Joey Tianyi Zhou Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 202 405 0 13 Jul 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 293 1,084 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 310 3,708 0 11 Feb 2021