Panoptic Captioning: Seeking An Equivalency Bridge for Image and Text

22 May 2025

Papers citing "Panoptic Captioning: Seeking An Equivalency Bridge for Image and Text"

50 / 92 papers shown

Title
Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalization Jiaming Zhou Ke Ye Jiayi Liu Teli Ma Zifang Wang Ronghe Qiu Kun-Yu Lin Zhilin Zhao Junwei Liang 48 2 0 21 May 2025
ActionArt: Advancing Multimodal Large Models for Fine-Grained Human-Centric Video Understanding Yi-Xing Peng Q. Yang Yu-Ming Tang Shenghao Fu Kun-Yu Lin Xihan Wei Wei-Shi Zheng 67 1 0 25 Apr 2025
Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning Qinghao Ye Xianhan Zeng Fu Li Chong Li Haoqi Fan CoGe 96 4 0 10 Mar 2025
LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models Shenghao Fu Q. Yang Qijie Mo Junkai Yan Xihan Wei Jingke Meng Xiaohua Xie Wei-Shi Zheng MLLM ObjD VLM 134 5 0 31 Jan 2025
ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations Tianming Liang Kun-Yu Lin Chaolei Tan Jianguo Zhang Wei-Shi Zheng Jian-Fang Hu VGen VOS ObjD 55 3 0 24 Jan 2025
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games Davide Paglieri Bartłomiej Cupiał Samuel Coward Ulyana Piterbarg Maciej Wolczyk ... Lerrel Pinto Rob Fergus Jakob Foerster Jack Parker-Holder Tim Rocktaschel LLMAG LRM 152 16 0 20 Nov 2024
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models Matt Deitke Christopher Clark Sangho Lee Rohun Tripathi Yue Yang ... Noah A. Smith Hannaneh Hajishirzi Ross Girshick Ali Farhadi Aniruddha Kembhavi OSLM VLM 56 58 0 25 Sep 2024
NVLM: Open Frontier-Class Multimodal LLMs Wenliang Dai Nayeon Lee Wei Ping Zhuoling Yang Zihan Liu Jon Barker Tuomas Rintamaki Mohammad Shoeybi Bryan Catanzaro Ming-Yu Liu MLLM VLM LRM 55 58 0 17 Sep 2024
ParGo: Bridging Vision-Language with Partial and Global Views An-Lan Wang Bin Shan Wei Shi Kun-Yu Lin Xiang Fei Guozhi Tang Lei Liao Jingqun Tang Can Huang Wei-Shi Zheng MLLM VLM 109 17 0 23 Aug 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 75 666 0 06 Aug 2024
DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception Xiaotong Li Fan Zhang Haiwen Diao Yueze Wang Xinlong Wang Ling-yu Duan VLM 60 29 0 11 Jul 2024
MATE: Meet At The Embedding -- Connecting Images with Long Texts Young Kyun Jang Junmo Kang Yong Jae Lee Donghyun Kim VLM 70 6 0 26 Jun 2024
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong Ellis L Brown Penghao Wu Sanghyun Woo Manoj Middepogu ... Xichen Pan Austin Wang Rob Fergus Yann LeCun Saining Xie 3DV MLLM 76 321 0 24 Jun 2024
Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation Jiaming Zhou Teli Ma Kun-Yu Lin Ronghe Qiu Zifan Wang Junwei Liang 82 7 0 20 Jun 2024
FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model Yebin Lee Imseong Park Myungjoo Kang 42 12 0 10 Jun 2024
Grasp as You Say: Language-guided Dexterous Grasp Generation Yi-Lin Wei Jian-Jian Jiang Chengyi Xing Xiantuo Tan Xiao-Ming Wu Hao Li M. Cutkosky Wei-Shi Zheng 68 17 0 29 May 2024
Benchmarking and Improving Detail Image Caption Hongyuan Dong Jiawen Li Bohong Wu Jiacong Wang Yuan Zhang Haoyuan Guo VLM MLLM 58 23 0 29 May 2024
NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models Chankyu Lee Rajarshi Roy Mengyao Xu Jonathan Raiman Mohammad Shoeybi Bryan Catanzaro Ming-Yu Liu RALM 153 170 0 27 May 2024
DOCCI: Descriptions of Connected and Contrasting Images Yasumasa Onoe Sunayana Rane Zachary Berger Yonatan Bitton Jaemin Cho ... Zarana Parekh Jordi Pont-Tuset Garrett Tanzer Su Wang Jason Baldridge 56 56 0 30 Apr 2024
DreamLIP: Language-Image Pre-training with Long Captions Kecheng Zheng Yifei Zhang Wei Wu Fan Lu Shuailei Ma Xin Jin Wei Chen Yujun Shen VLM CLIP 83 27 0 25 Mar 2024
Long-CLIP: Unlocking the Long-Text Capability of CLIP Beichen Zhang Pan Zhang Xiao-wen Dong Yuhang Zang Jiaqi Wang CLIP VLM 63 124 0 22 Mar 2024
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation Yunhao Gou Kai Chen Zhili Liu Lanqing Hong Hang Xu Zhenguo Li Dit-Yan Yeung James T. Kwok Yu Zhang MLLM 62 46 0 14 Mar 2024
PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation Junsong Chen Chongjian Ge Enze Xie Yue Wu Lewei Yao Xiaozhe Ren Zhongdao Wang Ping Luo Huchuan Lu Zhenguo Li 151 102 0 07 Mar 2024
Rethinking CLIP-based Video Learners in Cross-Domain Open-Vocabulary Action Recognition Kun-Yu Lin Henghui Ding Jiaming Zhou Yu-Ming Tang Yi-Xing Peng Zhilin Zhao Chen Change Loy Wei-Shi Zheng VLM 63 17 0 03 Mar 2024
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World Weiyun Wang Yiming Ren Hao Luo Tiantong Li Chenxiang Yan ... Qingyun Li Lewei Lu Xizhou Zhu Yu Qiao Jifeng Dai MLLM 69 49 0 29 Feb 2024
FROSTER: Frozen CLIP Is A Strong Teacher for Open-Vocabulary Action Recognition Xiaohui Huang Hao Zhou Kun Yao Kai Han VLM 78 22 0 05 Feb 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 67 302 0 11 Jan 2024
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 202 1,036 0 21 Dec 2023
Osprey: Pixel Understanding with Visual Instruction Tuning Yuqian Yuan Wentong Li Jian Liu Dongqi Tang Xinjie Luo Chi Qin Lei Zhang Jianke Zhu MLLM VLM 67 81 0 15 Dec 2023
A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions Jack Urbanek Florian Bordes Pietro Astolfi Mary Williamson Vasu Sharma Adriana Romero Soriano CLIP 3DV 55 45 0 14 Dec 2023
PixelLM: Pixel Reasoning with Large Multimodal Model Zhongwei Ren Zhicheng Huang Yunchao Wei Yao-Min Zhao Dongmei Fu Jiashi Feng Xiaojie Jin VLM MLLM LRM 47 93 0 04 Dec 2023
ShareGPT4V: Improving Large Multi-Modal Models with Better Captions Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Conghui He Jiaqi Wang Feng Zhao Dahua Lin MLLM VLM 156 622 0 21 Nov 2023
NExT-Chat: An LMM for Chat, Detection and Segmentation Ao Zhang Yuan Yao Wei Ji Zhiyuan Liu Tat-Seng Chua MLLM VLM 58 54 0 08 Nov 2023
GLaMM: Pixel Grounding Large Multimodal Model H. Rasheed Muhammad Maaz Sahal Shaji Mullappilly Abdelrahman M. Shaker Salman Khan Hisham Cholakkal Rao M. Anwer Erix Xing Ming-Hsuan Yang Fahad S. Khan MLLM VLM 74 219 0 06 Nov 2023
Sculpting Holistic 3D Representation in Contrastive Language-Image-3D Pre-training Yipeng Gao Zeyu Wang Wei-Shi Zheng Cihang Xie Yuyin Zhou 3DPC 60 9 0 03 Nov 2023
FAITHSCORE: Evaluating Hallucinations in Large Vision-Language Models Liqiang Jing Ruosen Li Yunmo Chen Mengzhao Jia Xinya Du MLLM 40 18 0 02 Nov 2023
CapsFusion: Rethinking Image-Text Data at Scale Qiying Yu Quan-Sen Sun Xiaosong Zhang Yufeng Cui Fan Zhang Yue Cao Xinlong Wang Jingjing Liu VLM 42 58 0 31 Oct 2023
Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation Jaemin Cho Yushi Hu Roopal Garg Peter Anderson Ranjay Krishna Jason Baldridge Mohit Bansal Jordi Pont-Tuset Su Wang EGVM 44 73 0 27 Oct 2023
Ferret: Refer and Ground Anything Anywhere at Any Granularity Haoxuan You Haotian Zhang Zhe Gan Xianzhi Du Bowen Zhang Zirui Wang Liangliang Cao Shih-Fu Chang Yinfei Yang ObjD MLLM VLM 56 314 0 11 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 82 2,593 0 05 Oct 2023
MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions Henghui Ding Chang Liu Shuting He Xudong Jiang Chen Change Loy VOS 68 106 0 16 Aug 2023
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World Weiyun Wang Min Shi Qingyun Li Wen Wang Zhenhang Huang ... Zhiguo Cao Yushi Chen Tong Lu Jifeng Dai Yu Qiao LRM MLLM 60 85 0 03 Aug 2023
LISA: Reasoning Segmentation via Large Language Model Xin Lai Zhuotao Tian Yukang Chen Yanwei Li Yuhui Yuan Shu Liu Jiaya Jia LM&Ro VLM MLLM LRM 80 424 0 01 Aug 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo VLM MLLM 120 231 0 07 Jul 2023
Towards Open Vocabulary Learning: A Survey Jianzong Wu Xiangtai Li Shilin Xu Haobo Yuan Henghui Ding ... Jiangning Zhang Yu Tong Xudong Jiang Guohao Li Dacheng Tao ObjD VLM 107 144 0 28 Jun 2023
Kosmos-2: Grounding Multimodal Large Language Models to the World Zhiliang Peng Wenhui Wang Li Dong Y. Hao Shaohan Huang Shuming Ma Furu Wei MLLM ObjD VLM 76 724 0 26 Jun 2023
Recognize Anything: A Strong Image Tagging Model Youcai Zhang Xinyu Huang Jinyu Ma Zhaoyang Li Zhaochuan Luo ... Tong Luo Yaqian Li Siyi Liu Yandong Guo Lei Zhang VLM 72 231 0 06 Jun 2023
FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsing Zhuang Li Yuyang Chai Terry Yue Zhuo Zhuang Li Gholamreza Haffari Fei Li Donghong Ji Quan Hung Tran 62 31 0 27 May 2023
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks Wen Wang Zhe Chen Xiaokang Chen Jiannan Wu Xizhou Zhu ... Ping Luo Tong Lu Jie Zhou Yu Qiao Jifeng Dai MLLM VLM 50 474 0 18 May 2023
VideoChat: Chat-Centric Video Understanding Kunchang Li Yinan He Yi Wang Yizhuo Li Wen Wang Ping Luo Yali Wang Limin Wang Yu Qiao MLLM 76 547 0 10 May 2023