GIT: A Generative Image-to-text Transformer for Vision and Language

27 May 2022

Zicheng Liu

Papers citing "GIT: A Generative Image-to-text Transformer for Vision and Language"

50 / 405 papers shown

Title
How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model Shezheng Song Xiaopeng Li Shasha Li Shan Zhao Jie Yu Jun Ma Xiaoguang Mao Weimin Zhang 71 4 0 10 Nov 2023
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities A. Piergiovanni Isaac Noble Dahun Kim Michael S. Ryoo Victor Gomes A. Angelova 36 19 0 09 Nov 2023
Vision Encoder-Decoder Models for AI Coaching Jyothi S. Nayak Afifah Khan Mohammed Ajmal Khan Chirag Manjeshwar Imadh Ajaz Banday 9 0 0 09 Nov 2023
Multitask Multimodal Prompted Training for Interactive Embodied Task Completion Georgios Pantazopoulos Malvina Nikandrou Amit Parekh Bhathiya Hemanthage Arash Eshghi Ioannis Konstas Verena Rieser Oliver Lemon Alessandro Suglia LM&Ro 36 7 0 07 Nov 2023
mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration Qinghao Ye Haiyang Xu Jiabo Ye Mingshi Yan Anwen Hu Haowei Liu Qi Qian Ji Zhang Fei Huang Jingren Zhou MLLM VLM 126 375 0 07 Nov 2023
CogVLM: Visual Expert for Pretrained Language Models Weihan Wang Qingsong Lv Wenmeng Yu Wenyi Hong Ji Qi ... Bin Xu Juanzi Li Yuxiao Dong Ming Ding Jie Tang VLM MLLM 27 446 0 06 Nov 2023
Dense Video Captioning: A Survey of Techniques, Datasets and Evaluation Protocols Iqra Qasim Alexander Horsch Dilip K. Prasad 22 5 0 05 Nov 2023
Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review Mingze Yuan Peng Bao Jiajia Yuan Yunhao Shen Zi Chen ... Jie Zhao Yang Chen Li Zhang Lin Shen Bin Dong ELM LM&MA 46 13 0 03 Nov 2023
Visual Analytics for Efficient Image Exploration and User-Guided Image Captioning Yiran Li Junpeng Wang Prince Osei Aboagye Michael Yeh Yan Zheng Liang Wang Wei Zhang Kwan-Liu Ma 19 2 0 02 Nov 2023
From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities Md Farhan Ishmam Md Sakib Hossain Shovon M. F. Mridha Nilanjan Dey 37 36 0 01 Nov 2023
Text Augmented Spatial-aware Zero-shot Referring Image Segmentation Yuchen Suo Linchao Zhu Yi Yang 31 13 0 27 Oct 2023
Impressions: Understanding Visual Semiotics and Aesthetic Impact Julia Kruk Caleb Ziems Diyi Yang 30 2 0 27 Oct 2023
CAD -- Contextual Multi-modal Alignment for Dynamic AVQA Asmar Nadeem Adrian Hilton R. Dawes Graham A. Thomas A. Mustafa 23 9 0 25 Oct 2023
HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models Tianrui Guan Fuxiao Liu Xiyang Wu Ruiqi Xian Zongxia Li ... Lichang Chen Furong Huang Yaser Yacoob Dinesh Manocha Dinesh Manocha VLM MLLM 34 155 0 23 Oct 2023
Semantic and Expressive Variation in Image Captions Across Languages Andre Ye Sebastin Santy Jena D. Hwang Amy X. Zhang Ranjay Krishna VLM 56 3 0 22 Oct 2023
SILC: Improving Vision Language Pretraining with Self-Distillation Muhammad Ferjad Naeem Yongqin Xian Xiaohua Zhai Lukas Hoyer Luc Van Gool F. Tombari VLM 26 33 0 20 Oct 2023
Visual Grounding Helps Learn Word Meanings in Low-Data Regimes Chengxu Zhuang Evelina Fedorenko Jacob Andreas 22 10 0 20 Oct 2023
ICU: Conquering Language Barriers in Vision-and-Language Modeling by Dividing the Tasks into Image Captioning and Language Understanding Guojun Wu VLM MLLM 27 0 0 19 Oct 2023
Towards Automatic Satellite Images Captions Generation Using Large Language Models Yingxu He Qiqi Sun 14 2 0 17 Oct 2023
EXMODD: An EXplanatory Multimodal Open-Domain Dialogue dataset Hang Yin Pinren Lu Ziang Li Bin Sun Kan Li 34 0 0 17 Oct 2023
Few-shot Action Recognition with Captioning Foundation Models Xiang Wang Shiwei Zhang Hangjie Yuan Yingya Zhang Changxin Gao Deli Zhao Nong Sang VLM 28 7 0 16 Oct 2023
VLIS: Unimodal Language Models Guide Multimodal Language Generation Jiwan Chung Youngjae Yu VLM 27 1 0 15 Oct 2023
PaLI-3 Vision Language Models: Smaller, Faster, Stronger Xi Chen Xiao Wang Lucas Beyer Alexander Kolesnikov Jialin Wu ... Keran Rong Tianli Yu Daniel Keysers Xiao-Qi Zhai Radu Soricut MLLM VLM 38 94 0 13 Oct 2023
Idea2Img: Iterative Self-Refinement with GPT-4V(ision) for Automatic Image Design and Generation Zhengyuan Yang Jianfeng Wang Linjie Li Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Lijuan Wang LRM MLLM DiffM 13 22 0 12 Oct 2023
Towards Robust Multi-Modal Reasoning via Model Selection Xiangyan Liu Rongxue Li Wei Ji Tao Lin LLMAG LRM 37 3 0 12 Oct 2023
Ferret: Refer and Ground Anything Anywhere at Any Granularity Haoxuan You Haotian Zhang Zhe Gan Xianzhi Du Bowen Zhang Zirui Wang Liangliang Cao Shih-Fu Chang Yinfei Yang ObjD MLLM VLM 24 301 0 11 Oct 2023
Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models Holy Lovenia Wenliang Dai Samuel Cahyawijaya Ziwei Ji Pascale Fung MLLM 25 50 0 09 Oct 2023
Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling Haogeng Liu Qihang Fan Tingkai Liu Linjie Yang Yunzhe Tao Huaibo Huang Ran He Hongxia Yang VGen 26 12 0 08 Oct 2023
HowToCaption: Prompting LLMs to Transform Video Annotations at Scale Nina Shvetsova Anna Kukleva Xudong Hong Christian Rupprecht Bernt Schiele Hilde Kuehne 45 25 0 07 Oct 2023
Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency Tianhong Li Sangnie Bhardwaj Yonglong Tian Han Zhang Jarred Barber Dina Katabi Guillaume Lajoie Huiwen Chang Dilip Krishnan VLM 44 4 0 05 Oct 2023
Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction Yiren Jian Tingkai Liu Yunzhe Tao Chunhui Zhang Soroush Vosoughi HX Yang VLM 20 7 0 05 Oct 2023
Human-centric Behavior Description in Videos: New Benchmark and Model Lingru Zhou Yi-Meng Gao Manqing Zhang Peng Wu Peng Wang Yanning Zhang 38 1 0 04 Oct 2023
TWIZ-v2: The Wizard of Multimodal Conversational-Stimulus Rafael Ferreira Diogo Tavares Diogo Glória-Silva Rodrigo Valerio João Bordalo Ines Simoes Vasco Ramos David Semedo João Magalhães 24 4 0 03 Oct 2023
Sieve: Multimodal Dataset Pruning Using Image Captioning Models Anas Mahmoud Mostafa Elhoushi Amro Abbas Yu Yang Newsha Ardalani Hugh Leather Ari S. Morcos VLM CLIP 40 19 0 03 Oct 2023
Language Models as Knowledge Bases for Visual Word Sense Disambiguation Anastasia Kritharoula Maria Lymperaiou Giorgos Stamou 26 2 0 03 Oct 2023
GRID: A Platform for General Robot Intelligence Development Sai H. Vemprala Shuhang Chen Abhinav Shukla Dinesh Narayanan Ashish Kapoor 25 10 0 02 Oct 2023
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning Mustafa Shukor Alexandre Ramé Corentin Dancette Matthieu Cord LRM MLLM 40 20 0 01 Oct 2023
Guiding Instruction-based Image Editing via Multimodal Large Language Models Johannes Frey Wenze Hu Xianzhi Du William Yang Wang Yinfei Yang Zhe Gan 40 88 0 29 Sep 2023
Targeted Image Data Augmentation Increases Basic Skills Captioning Robustness Valentin Barriere Felipe del Rio Andres Carvallo De Ferari Carlos Aspillaga Eugenio Herrera-Berg Cristian Buc Calderon DiffM 27 0 0 27 Sep 2023
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition Pan Zhang Xiaoyi Wang Bin Wang Yuhang Cao Chao Xu ... Conghui He Xingcheng Zhang Yu Qiao Da Lin Jiaqi Wang MLLM 80 222 0 26 Sep 2023
Object-Centric Open-Vocabulary Image-Retrieval with Aggregated Features Hila Levi Guy Heller Dan Levi Ethan Fetaya OCL VLM 24 3 0 26 Sep 2023
VidChapters-7M: Video Chapters at Scale Antoine Yang Arsha Nagrani Ivan Laptev Josef Sivic Cordelia Schmid VGen 23 26 0 25 Sep 2023
A Survey on Image-text Multimodal Models Ruifeng Guo Jingxuan Wei Linzhuang Sun Khai Le-Duc Guiyong Chang Dawei Liu Sibo Zhang Zhengbing Yao Mingjun Xu Liping Bu VLM 31 5 0 23 Sep 2023
Multimodal Deep Learning for Scientific Imaging Interpretation Abdulelah S. Alshehri Franklin L. Lee Shihu Wang 24 2 0 21 Sep 2023
SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gap Daehee Kim Yoon Kim Donghyun Kim Yumin Lim Geewook Kim Taeho Kil 31 3 0 21 Sep 2023
Towards Practical and Efficient Image-to-Speech Captioning with Vision-Language Pre-training and Multi-modal Tokens Minsu Kim J. Choi Soumi Maiti Jeong Hun Yeo Shinji Watanabe Y. Ro VLM 26 6 0 15 Sep 2023
MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning Haozhe Zhao Zefan Cai Shuzheng Si Xiaojian Ma Kaikai An Liang Chen Zixuan Liu Sheng Wang Wenjuan Han Baobao Chang MLLM VLM 28 133 0 14 Sep 2023
Language Models as Black-Box Optimizers for Vision-Language Models Shihong Liu Zhiqiu Lin Samuel Yu Ryan Lee Tiffany Ling Deepak Pathak Deva Ramanan VLM 27 28 0 12 Sep 2023
NExT-GPT: Any-to-Any Multimodal LLM Shengqiong Wu Hao Fei Leigang Qu Wei Ji Tat-Seng Chua MLLM 46 457 0 11 Sep 2023
InstructDiffusion: A Generalist Modeling Interface for Vision Tasks Zigang Geng Binxin Yang Tiankai Hang Chen Li Shuyang Gu ... Jianmin Bao Zheng-Wei Zhang Han Hu Dongdong Chen Baining Guo DiffM VLM 51 93 0 07 Sep 2023