Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts

16 November 2021

Papers citing "Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts"

50 / 72 papers shown

Title
Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models Kai Tang Jinhao You Xiuqi Ge Hanze Li Yichen Guo Xiande Huang MLLM 7 0 0 18 May 2025
Decoupled Global-Local Alignment for Improving Compositional Understanding Xiaoxing Hu Kaicheng Yang Jianmin Wang Haoran Xu Ziyong Feng Yixuan Wang VLM 177 0 0 23 Apr 2025
Low-hallucination Synthetic Captions for Large-Scale Vision-Language Model Pre-training Xinsong Zhang Yarong Zeng Xinting Huang Hu Hu Runquan Xie Han Hu Zhanhui Kang MLLM VLM 55 1 0 17 Apr 2025
CTRL-O: Language-Controllable Object-Centric Visual Representation Learning Aniket Didolkar Andrii Zadaianchuk Rabiul Awal Maximilian Seitzer E. Gavves Aishwarya Agrawal OCL VLM 94 2 0 27 Mar 2025
Stitch-a-Recipe: Video Demonstration from Multistep Descriptions Chi Hsuan Wu Kumar Ashutosh Kristen Grauman DiffM 68 0 0 18 Mar 2025
Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song H. Shen 75 1 0 11 Mar 2025
Can Hallucination Correction Improve Video-Language Alignment? Lingjun Zhao Mingyang Xie Paola Cascante-Bonilla Hal Daumé III Kwonjoon Lee HILM VLM 64 0 0 20 Feb 2025
Object-centric Binding in Contrastive Language-Image Pretraining Rim Assouel Pietro Astolfi Florian Bordes M. Drozdzal Adriana Romero Soriano OCL VLM CoGe 103 0 0 19 Feb 2025
Where am I? Cross-View Geo-localization with Natural Language Descriptions Junyan Ye Honglin Lin Leyan Ou Dairong Chen Zihao Wang Zeang Sheng Weijia Li Weijia Li 78 0 0 22 Dec 2024
Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Search Shuyu Yang Yaxiong Wang Li Zhu Zhedong Zheng 101 2 0 26 Nov 2024
Text-Guided Coarse-to-Fine Fusion Network for Robust Remote Sensing Visual Question Answering Zhicheng Zhao Changfu Zhou Yu Zhang Chenglong Li Xiaoliang Ma Jin Tang 81 0 0 24 Nov 2024
A Unified Debiasing Approach for Vision-Language Models across Modalities and Tasks Hoin Jung T. Jang Xiaoqian Wang VLM 27 2 0 10 Oct 2024
NEUSIS: A Compositional Neuro-Symbolic Framework for Autonomous Perception, Reasoning, and Planning in Complex UAV Search Missions Zhixi Cai Cristian Rojas Cardenas Kevin Leo Chenyuan Zhang Kal Backman ... Yuan-Fang Li Mor Vered Peter Stuckey M. G. D. L. Banda Hamid Rezatofighi 39 7 0 16 Sep 2024
ComAlign: Compositional Alignment in Vision-Language Models Ali Abdollah Amirmohammad Izadi Armin Saghafian Reza Vahidimajd Mohammad Mozafari Amirreza Mirzaei Mohammadmahdi Samiei M. Baghshah CoGe VLM 30 0 0 12 Sep 2024
ExpertAF: Expert Actionable Feedback from Video Kumar Ashutosh Tushar Nagarajan Georgios Pavlakos Kris Kitani Kristen Grauman VGen 44 2 0 01 Aug 2024
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions Yu-Guan Hsieh Cheng-Yu Hsieh Shih-Ying Yeh Louis Béthune Hadi Pour Ansari Pavan Kumar Anasosalu Vasu Chun-Liang Li Ranjay Krishna Oncel Tuzel Marco Cuturi 66 4 0 09 Jul 2024
One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Models Hao Fang Jiawei Kong Wenbo Yu Bin Chen Jiawei Li Hao Wu Ke Xu Ke Xu AAML VLM 40 13 0 08 Jun 2024
CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning Yiping Wang Yifang Chen Wendan Yan Alex Fang Wenjing Zhou Kevin G. Jamieson S. Du 36 7 0 29 May 2024
Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learning Neha Kalibhat Priyatham Kattakinda Arman Zarei Nikita Seleznev Sam Sharpe Samuel Sharpe Senthil Kumar S. Feizi ViT 39 0 0 26 May 2024
VISLA Benchmark: Evaluating Embedding Sensitivity to Semantic and Lexical Alterations Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad VLM CoGe 48 0 0 25 Apr 2024
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings Olivia Wiles Chuhan Zhang Isabela Albuquerque Ivana Kajić Su Wang ... Jordi Pont-Tuset Aida Nematzadeh Anant Nawalgaria Jordi Pont-Tuset Aida Nematzadeh EGVM 135 14 0 25 Apr 2024
MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities Kunxi Li Tianyu Zhan Kairui Fu Shengyu Zhang Kun Kuang Jiwei Li Zhou Zhao Fei Wu MoMe 24 0 0 20 Apr 2024
From Data Deluge to Data Curation: A Filtering-WoRA Paradigm for Efficient Text-based Person Search Jintao Sun Zhedong Zheng Gangyi Ding Gangyi Ding 40 7 0 16 Apr 2024
Enhancing Vision-Language Pre-training with Rich Supervisions Yuan Gao Kunyu Shi Pengkai Zhu Edouard Belval Oren Nuriel Srikar Appalaraju Shabnam Ghadar Vijay Mahadevan Zhuowen Tu Stefano Soatto VLM CLIP 67 12 0 05 Mar 2024
SInViG: A Self-Evolving Interactive Visual Agent for Human-Robot Interaction Jie Xu Hanbo Zhang Xinghang Li Huaping Liu Xuguang Lan Tao Kong LM&Ro 38 3 0 19 Feb 2024
ProtChatGPT: Towards Understanding Proteins with Large Language Models Chao Wang Hehe Fan Ruijie Quan Yi Yang 31 13 0 15 Feb 2024
TiMix: Text-aware Image Mixing for Effective Vision-Language Pre-training Chaoya Jiang Wei Ye Haiyang Xu Qinghao Ye Mingshi Yan Ji Zhang Shikun Zhang CLIP VLM 27 4 0 14 Dec 2023
Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding Wujian Peng Sicheng Xie Zuyao You Shiyi Lan Zuxuan Wu VLM CoGe MLLM 33 18 0 30 Nov 2023
Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models Jiayun Luo Siddhesh Khandelwal Leonid Sigal Boyang Albert Li MLLM VLM 35 7 0 28 Nov 2023
Multiscale Superpixel Structured Difference Graph Convolutional Network for VL Representation Siyu Zhang Ye-Ting Chen Fang Wang Yaoru Sun Jun Yang Lizhi Bai SSL 32 0 0 20 Oct 2023
EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE Junyi Chen Longteng Guo Jianxiang Sun Shuai Shao Zehuan Yuan Liang Lin Dongyu Zhang MLLM VLM MoE 60 9 0 23 Aug 2023
Bridging the Gap: Exploring the Capabilities of Bridge-Architectures for Complex Visual Reasoning Tasks Kousik Rajesh Mrigank Raman M. A. Karim Pranit Chawla VLM 25 2 0 31 Jul 2023
Seeing in Words: Learning to Classify through Language Bottlenecks Khalid Saifullah Yuxin Wen Jonas Geiping Micah Goldblum Tom Goldstein VLM 21 2 0 29 Jun 2023
LRVS-Fashion: Extending Visual Search with Referring Instructions Simon Lepage Jérémie Mary David Picard 25 1 0 05 Jun 2023
PuMer: Pruning and Merging Tokens for Efficient Vision Language Models Qingqing Cao Bhargavi Paranjape Hannaneh Hajishirzi MLLM VLM 15 21 0 27 May 2023
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning Chia-Wen Kuo Z. Kira 37 21 0 25 May 2023
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining Emanuele Bugliarello Aida Nematzadeh Lisa Anne Hendricks SSL 32 5 0 23 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 48 116 0 18 May 2023
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding Le Xue Ning Yu Shu Zhen Zhang Artemis Panagopoulou Junnan Li ... Jiajun Wu Caiming Xiong Ran Xu Juan Carlos Niebles Silvio Savarese 29 115 0 14 May 2023
IMAGINATOR: Pre-Trained Image+Text Joint Embeddings using Word-Level Grounding of Images Varuna Krishna S. Suryavardan Shreyash Mishra Sathyanarayanan Ramamoorthy Parth Patwa Megha Chakraborty Aman Chadha Amitava Das Amit P. Sheth VLM 33 3 0 12 May 2023
Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning Hao Zhang Nianwen Si Yaqi Chen Wenlin Zhang Xukui Yang Dan Qu Weiqiang Zhang 35 9 0 20 Apr 2023
CoVLR: Coordinating Cross-Modal Consistency and Intra-Modal Structure for Vision-Language Retrieval Yang Yang Zhongtian Fu Xiangyu Wu Wenjie Li VLM 21 1 0 15 Apr 2023
Detecting and Grounding Multi-Modal Media Manipulation Rui Shao Tianxing Wu Ziwei Liu 44 58 0 05 Apr 2023
Cross-Modal Causal Intervention for Medical Report Generation Weixing Chen Yang Liu Ce Wang Jiarui Zhu Shen Zhao Guanbin Li Cheng-Lin Liu Liang Lin 34 6 0 16 Mar 2023
ViperGPT: Visual Inference via Python Execution for Reasoning Dídac Surís Sachit Menon Carl Vondrick MLLM LRM ReLM 45 435 0 14 Mar 2023
Focusing On Targets For Improving Weakly Supervised Visual Grounding V. Pham Nao Mishima ObjD 26 1 0 22 Feb 2023
OvarNet: Towards Open-vocabulary Object Attribute Recognition Keyan Chen Xiaolong Jiang Yao Hu Xu Tang Yan Gao Jianqi Chen Weidi Xie VLM ObjD 40 40 0 23 Jan 2023
Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study Mariya Hendriksen Svitlana Vakulenko E. Kuiper Maarten de Rijke 39 5 0 12 Jan 2023
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks Xinsong Zhang Yan Zeng Jipeng Zhang Hang Li VLM AI4CE LRM 27 17 0 12 Jan 2023
HierVL: Learning Hierarchical Video-Language Embeddings Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman VLM AI4TS 28 53 0 05 Jan 2023