Title
Improving Multimodal Datasets with Image Captioning Thao Nguyen S. Gadre Gabriel Ilharco Sewoong Oh Ludwig Schmidt VLM 19 71 0 19 Jul 2023
Multimodal Diffusion Segmentation Model for Object Segmentation from Manipulation Instructions Yui Iioka Y. Yoshida Yuiga Wada Shumpei Hatanaka K. Sugiura DiffM 50 5 0 17 Jul 2023
Tangent Model Composition for Ensembling and Continual Fine-tuning Tianlin Liu Stefano Soatto LRM MoMe CLL 27 16 0 16 Jul 2023
Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Tasks Ryosuke Korekata Motonari Kambara Yusuke Yoshida Shintaro Ishikawa Yosuke Kawasaki Masaki Takahashi K. Sugiura LM&Ro 53 5 0 14 Jul 2023
Bootstrapping Vision-Language Learning with Decoupled Language Pre-training Yiren Jian Chongyang Gao Soroush Vosoughi VLM MLLM 32 25 0 13 Jul 2023
Leveraging Vision-Language Foundation Models for Fine-Grained Downstream Tasks Denis Coquenet Clément Rambour Emanuele Dalsasso Nicolas Thome MLLM CLIP VLM 37 1 0 13 Jul 2023
GVCCI: Lifelong Learning of Visual Grounding for Language-Guided Robotic Manipulation Junghyun Kim Gi-Cheon Kang Jaein Kim Suyeon Shin Byoung-Tak Zhang LM&Ro 35 7 0 12 Jul 2023
Prototypical Contrastive Transfer Learning for Multimodal Language Understanding Seitaro Otsuki Shintaro Ishikawa K. Sugiura 49 1 0 12 Jul 2023
DRMC: A Generalist Model with Dynamic Routing for Multi-Center PET Image Synthesis Zhiwen Yang Yang Zhou Hui Zhang Bingzheng Wei Yubo Fan Yan Xu MedIm 25 3 0 11 Jul 2023
Emu: Generative Pretraining in Multimodality Quan-Sen Sun Qiying Yu Yufeng Cui Fan Zhang Xiaosong Zhang Yueze Wang Hongcheng Gao Jingjing Liu Tiejun Huang Xinlong Wang MLLM 40 126 0 11 Jul 2023
KU-DMIS-MSRA at RadSum23: Pre-trained Vision-Language Model for Radiology Report Summarization Gangwoo Kim Hajung Kim Lei Ji Seongsu Bae Chanhwi Kim Mujeen Sung Hyunjae Kim Kun Yan E. Chang Jaewoo Kang VLM 15 2 0 10 Jul 2023
Vision Language Transformers: A Survey Clayton Fields C. Kennington VLM 28 5 0 06 Jul 2023
AVSegFormer: Audio-Visual Segmentation with Transformer Sheng Gao Zhe Chen Guo Chen Wenhai Wang Tong Lu VOS 37 46 0 03 Jul 2023
Visual Instruction Tuning with Polite Flamingo Delong Chen Jianfeng Liu Wenliang Dai Baoyuan Wang MLLM 34 42 0 03 Jul 2023
JourneyDB: A Benchmark for Generative Image Understanding Keqiang Sun Junting Pan Yuying Ge Hao Li Haodong Duan ... Yi Wang Jifeng Dai Yu Qiao Limin Wang Hongsheng Li 54 103 0 03 Jul 2023
UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding Rui Sun Zhecan Wang Haoxuan You Noel Codella Kai-Wei Chang Shih-Fu Chang CLIP 32 3 0 03 Jul 2023
CLIPAG: Towards Generator-Free Text-to-Image Generation Roy Ganz Michael Elad VLM 33 7 0 29 Jun 2023
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic Ke Chen Zhao Zhang Weili Zeng Richong Zhang Feng Zhu Rui Zhao ObjD 44 598 0 27 Jun 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Xu Enhong Chen MLLM LRM 54 556 0 23 Jun 2023
Learning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimation Zihao Yue Anwen Hu Liang Zhang Qin Jin 24 2 0 23 Jun 2023
AudioPaLM: A Large Language Model That Can Speak and Listen Paul Kishan Rubenstein Chulayuth Asawaroengchai D. Nguyen Ankur Bapna Zalan Borsos ... Neil Zeghidour Yu Zhang Zhishuai Zhang Lukás Zilka Christian Frank LM&MA AuLLM VLM 39 264 0 22 Jun 2023
Generative Multimodal Entity Linking Senbao Shi Zhenran Xu Baotian Hu Hao Fei MLLM VLM 21 5 0 22 Jun 2023
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents Hugo Laurenccon Lucile Saulnier Léo Tronchon Stas Bekman Amanpreet Singh ... Siddharth Karamcheti Alexander M. Rush Douwe Kiela Matthieu Cord Victor Sanh 25 230 0 21 Jun 2023
ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining Dezhi Peng Chongyu Liu Yuliang Liu Lianwen Jin DiffM 24 9 0 21 Jun 2023
Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion Simone Bianco Luigi Celona Marco Donzella Paolo Napoletano 34 18 0 20 Jun 2023
Listener Model for the PhotoBook Referential Game with CLIPScores as Implicit Reference Chain Shih-Lun Wu Yi-Hui Chou Liang Li 18 0 0 16 Jun 2023
Tell Me Where to Go: A Composable Framework for Context-Aware Embodied Robot Navigation Harel Biggie Ajay Narasimha Mopidevi Dusty Woods Christoffer Heckman LM&Ro 16 11 0 15 Jun 2023
Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration Chenyang Lyu Minghao Wu Longyue Wang Xinting Huang Bingshuai Liu Zefeng Du Shuming Shi Zhaopeng Tu MLLM AuLLM 31 161 0 15 Jun 2023
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model Sihan Chen Xingjian He Handong Li Xiaojie Jin Jiashi Feng Jiaheng Liu VLM CLIP 30 8 0 15 Jun 2023
Training Multimedia Event Extraction With Generated Images and Captions Zilin Du Yunxin Li Xu Guo Yidan Sun Boyang Albert Li DiffM 21 7 0 15 Jun 2023
Improving Selective Visual Question Answering by Learning from Your Peers Corentin Dancette Spencer Whitehead Rishabh Maheshwary Ramakrishna Vedantam Stefan Scherer Xinlei Chen Matthieu Cord Marcus Rohrbach AAML OOD 38 16 0 14 Jun 2023
Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Models Lingxi Xie Longhui Wei Xiaopeng Zhang Kaifeng Bi Xiaotao Gu Jianlong Chang Qi Tian 38 7 0 14 Jun 2023
AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn Difei Gao Lei Ji Luowei Zhou Kevin Lin Joya Chen Zihan Fan Mike Zheng Shou MLLM 29 72 0 14 Jun 2023
Global and Local Semantic Completion Learning for Vision-Language Pre-training Rong-Cheng Tu Yatai Ji Jie Jiang Weijie Kong Chengfei Cai Wenzhe Zhao Hongfa Wang Yujiu Yang Wei Liu VLM 24 2 0 12 Jun 2023
UniBoost: Unsupervised Unimodal Pre-training for Boosting Zero-shot Vision-Language Tasks Yanan Sun Zi-Qi Zhong Qi Fan Chi-Keung Tang Yu-Wing Tai VLM 33 4 0 07 Jun 2023
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards Alexandre Ramé Guillaume Couairon Mustafa Shukor Corentin Dancette Jean-Baptiste Gaya Laure Soulier Matthieu Cord MoMe 35 136 0 07 Jun 2023
Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images! Zaid Khan B. Vijaykumar S. Schulter Xiang Yu Y. Fu Manmohan Chandraker VLM MLLM 32 17 0 06 Jun 2023
Putting Humans in the Image Captioning Loop Aliki Anagnostopoulou Mareike Hartmann Daniel Sonntag VLM 32 1 0 06 Jun 2023
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding Hang Zhang Xin Li Lidong Bing MLLM 53 961 0 05 Jun 2023
Table and Image Generation for Investigating Knowledge of Entities in Pre-trained Vision and Language Models Hidetaka Kamigaito Katsuhiko Hayashi Taro Watanabe VLM 15 1 0 03 Jun 2023
"Let's not Quote out of Context": Unified Vision-Language Pretraining for Context Assisted Image Captioning Abisek Rajakumar Kalarani P. Bhattacharyya Niyati Chhaya Sumit Shekhar CoGe VLM 21 9 0 01 Jun 2023
Chatting Makes Perfect: Chat-based Image Retrieval Matan Levy Rami Ben-Ari N. Darshan Dani Lischinski 58 14 0 31 May 2023
There is more to graphs than meets the eye: Learning universal features with self-supervision L. Das Sai Munikoti M. Halappanavar SSL OOD 30 1 0 31 May 2023
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction Rui Yang Lin Song Yanwei Li Sijie Zhao Yixiao Ge Xiu Li Ying Shan SyDa MLLM 36 209 0 30 May 2023
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Sihan Chen Handong Li Qunbo Wang Zijia Zhao Ming-Ting Sun Xinxin Zhu Jiaheng Liu 37 97 0 29 May 2023
Deeply Coupled Cross-Modal Prompt Learning Xuejing Liu Wei Tang Jinghui Lu Rui Zhao Zhaojun Guo Fei Tan VLM 28 17 0 29 May 2023
FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions Noam Rotstein David Bensaid Shaked Brody Roy Ganz Ron Kimmel VLM 26 27 0 28 May 2023
ConaCLIP: Exploring Distillation of Fully-Connected Knowledge Interaction Graph for Lightweight Text-Image Retrieval Jiapeng Wang Chengyu Wang Xiaodan Wang Jun Huang Lianwen Jin VLM 37 4 0 28 May 2023
PuMer: Pruning and Merging Tokens for Efficient Vision Language Models Qingqing Cao Bhargavi Paranjape Hannaneh Hajishirzi MLLM VLM 13 21 0 27 May 2023
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks Kai Zhang Jun Yu Eashan Adhikarla Rong Zhou Zhilin Yan ... Xun Chen Yong Chen Quanzheng Li Hongfang Liu Lichao Sun LM&MA MedIm 37 157 0 26 May 2023