Kosmos-2: Grounding Multimodal Large Language Models to the World

26 June 2023

Papers citing "Kosmos-2: Grounding Multimodal Large Language Models to the World"

42 / 142 papers shown

Title
Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation Maksim Kuprashevich Grigorii Alekseenko Irina Tolstykh ELM 56 4 0 04 Mar 2024
Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency Akila Wickramasekara F. Breitinger Mark Scanlon 52 8 0 29 Feb 2024
BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models Xueliang Zhao Xinting Huang Tingchen Fu Qintong Li Shansan Gong Lemao Liu Wei Bi Lingpeng Kong LRM 37 1 0 21 Feb 2024
SInViG: A Self-Evolving Interactive Visual Agent for Human-Robot Interaction Jie Xu Hanbo Zhang Xinghang Li Huaping Liu Xuguang Lan Tao Kong LM&Ro 38 3 0 19 Feb 2024
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition Chen Chen Ruizhe Li Yuchen Hu Sabato Marco Siniscalchi Pin-Yu Chen Ensiong Chng Chao-Han Huck Yang 33 19 0 08 Feb 2024
Convincing Rationales for Visual Question Answering Reasoning Kun Li G. Vosselman Michael Ying Yang 44 1 0 06 Feb 2024
BAT: Learning to Reason about Spatial Sounds with Large Language Models Zhisheng Zheng Puyuan Peng Ziyang Ma Xie Chen Eunsol Choi David Harwath LRM 35 14 0 02 Feb 2024
GeoDecoder: Empowering Multimodal Map Understanding Feng Qi Mian Dai Zixian Zheng Chao Wang 39 1 0 26 Jan 2024
GroundingGPT:Language Enhanced Multi-modal Grounding Model Zhaowei Li Qi Xu Dong Zhang Hang Song Yiqing Cai ... Junting Pan Zefeng Li Van Tu Vu Zhida Huang Tao Wang 36 37 0 11 Jan 2024
SonicVisionLM: Playing Sound with Vision Language Models Zhifeng Xie Shengye Yu Qile He Mengtian Li VLM VGen 28 2 0 09 Jan 2024
LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model Senqiao Yang Tianyuan Qu Xin Lai Zhuotao Tian Bohao Peng Shu-Lin Liu Jiaya Jia VLM 21 28 0 28 Dec 2023
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 48 29 0 19 Dec 2023
DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving Wenhai Wang Jiangwei Xie ChuanYang Hu Haoming Zou Jianan Fan ... Lewei Lu Xizhou Zhu Xiaogang Wang Yu Qiao Jifeng Dai 36 124 0 14 Dec 2023
See, Say, and Segment: Teaching LMMs to Overcome False Premises Tsung-Han Wu Giscard Biamby David M. Chan Lisa Dunlap Ritwik Gupta Xudong Wang Joseph E. Gonzalez Trevor Darrell VLM MLLM 39 18 0 13 Dec 2023
MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception Yiran Qin Enshen Zhou Qichang Liu Zhen-fei Yin Lu Sheng Ruimao Zhang Yu Qiao Jing Shao LM&Ro 32 39 0 12 Dec 2023
Multimodality of AI for Education: Towards Artificial General Intelligence Gyeong-Geon Lee Lehong Shi Ehsan Latif Yizhu Gao Arne Bewersdorff ... Zheng Liu Hui Wang Gengchen Mai Tiaming Liu Xiaoming Zhai 26 38 0 10 Dec 2023
GlitchBench: Can large multimodal models detect video game glitches? Mohammad Reza Taesiri Tianjun Feng Anh Nguyen C. Bezemer MLLM VLM LRM 30 9 0 08 Dec 2023
Alpha-CLIP: A CLIP Model Focusing on Wherever You Want Zeyi Sun Ye Fang Tong Wu Pan Zhang Yuhang Zang Shu Kong Yuanjun Xiong Dahua Lin Jiaqi Wang VLM CLIP 51 83 0 06 Dec 2023
IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers Chenglin Yang Siyuan Qiao Yuan Cao Yu Zhang Tao Zhu Alan L. Yuille Jiahui Yu VLM 18 3 0 27 Nov 2023
PG-Video-LLaVA: Pixel Grounding Large Video-Language Models Shehan Munasinghe Rusiru Thushara Muhammad Maaz H. Rasheed Salman Khan Mubarak Shah Fahad Khan VLM MLLM 27 34 0 22 Nov 2023
Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMs Yonghui Wang Wen-gang Zhou Hao Feng Keyi Zhou Houqiang Li 66 18 0 22 Nov 2023
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models Yunfei Chu Jin Xu Xiaohuan Zhou Qian Yang Shiliang Zhang Zhijie Yan Chang Zhou Jingren Zhou AuLLM 42 268 0 14 Nov 2023
To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuning Junke Wang Lingchen Meng Zejia Weng Bo He Zuxuan Wu Yu-Gang Jiang MLLM VLM 32 94 0 13 Nov 2023
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks Bin Xiao Haiping Wu Weijian Xu Xiyang Dai Houdong Hu Yumao Lu Michael Zeng Ce Liu Lu Yuan VLM 45 143 0 10 Nov 2023
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents Shilong Liu Hao Cheng Haotian Liu Hao Zhang Feng Li ... Hang Su Jun Zhu Lei Zhang Jianfeng Gao Chun-yue Li MLLM VLM 56 105 0 09 Nov 2023
u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model Jinjin Xu Liwu Xu Yuzhe Yang Xiang Li Fanyi Wang Yanchun Xie Yi-Jie Huang Yaqian Li MoE MLLM VLM 27 12 0 09 Nov 2023
Extending Multi-modal Contrastive Representations Zehan Wang Ziang Zhang Luping Liu Yang Zhao Haifeng Huang Tao Jin Zhou Zhao 21 5 0 13 Oct 2023
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition Pan Zhang Xiaoyi Wang Bin Wang Yuhang Cao Chao Xu ... Conghui He Xingcheng Zhang Yu Qiao Da Lin Jiaqi Wang MLLM 80 222 0 26 Sep 2023
Concepts is All You Need: A More Direct Path to AGI Peter Voss M. Jovanovic 25 4 0 04 Sep 2023
Link-Context Learning for Multimodal LLMs Yan Tai Weichen Fan Zhao Zhang Feng Zhu Rui Zhao Ziwei Liu ReLM LRM 26 17 0 15 Aug 2023
SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension Bohao Li Rui Wang Guangzhi Wang Yuying Ge Yixiao Ge Ying Shan MLLM ELM 32 502 0 30 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang F. Khan VLM 38 118 0 25 Jul 2023
Retentive Network: A Successor to Transformer for Large Language Models Yutao Sun Li Dong Shaohan Huang Shuming Ma Yuqing Xia Jilong Xue Jianyong Wang Furu Wei LRM 78 301 0 17 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo VLM MLLM 85 224 0 07 Jul 2023
LongNet: Scaling Transformers to 1,000,000,000 Tokens Jiayu Ding Shuming Ma Li Dong Xingxing Zhang Shaohan Huang Wenhui Wang Nanning Zheng Furu Wei CLL 41 151 0 05 Jul 2023
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering Rabiul Awal Le Zhang Aishwarya Agrawal LRM 46 12 0 16 Jun 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 208 900 0 27 Apr 2023
RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding Jihan Yang Runyu Ding Weipeng Deng Zhe Wang Xiaojuan Qi 20 62 0 03 Apr 2023
Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images Nitzan Bitton-Guetta Yonatan Bitton Jack Hessel Ludwig Schmidt Yuval Elovici Gabriel Stanovsky Roy Schwartz VLM 121 66 0 13 Mar 2023
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 392 4,137 0 28 Jan 2022
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 244 344 0 22 Sep 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 593 0 14 Jul 2021