SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality

26 June 2023

Papers citing "SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality"

50 / 96 papers shown

Title
Visual Perturbation and Adaptive Hard Negative Contrastive Learning for Compositional Reasoning in Vision-Language Models Xin Huang Ruibin Li Tong Jia Wei Zheng Ya Wang VLM CoGe 14 0 0 21 May 2025
MINERVA: Evaluating Complex Video Reasoning Arsha Nagrani Sachit Menon Ahmet Iscen Shyamal Buch Ramin Mehran ... Yukun Zhu Carl Vondrick Mikhail Sirotenko Cordelia Schmid Tobias Weyand 60 0 0 01 May 2025
Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers Quentin Guimard Moreno DÍncà Massimiliano Mancini Elisa Ricci SSL 72 0 0 29 Apr 2025
Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs Tiancheng Gu Kaicheng Yang Ziyong Feng Xingjun Wang Yanzhao Zhang Dingkun Long Yingda Chen Weidong Cai Jiankang Deng VLM 244 2 0 24 Apr 2025
Decoupled Global-Local Alignment for Improving Compositional Understanding Xiaoxing Hu Kaicheng Yang Jianmin Wang Haoran Xu Ziyong Feng Yansen Wang VLM 183 0 0 23 Apr 2025
MIEB: Massive Image Embedding Benchmark Chenghao Xiao Isaac Chung Imene Kerboua Jamie Stirling Xin Zhang Márton Kardos Roman Solomatin Noura Al Moubayed Kenneth C. Enevoldsen Niklas Muennighoff VLM 42 0 0 14 Apr 2025
FocalLens: Instruction Tuning Enables Zero-Shot Conditional Image Representations Cheng-Yu Hsieh Pavan Kumar Anasosalu Vasu Fartash Faghri Raviteja Vemulapalli Chun-Liang Li Ranjay Krishna Oncel Tuzel Hadi Pouransari VLM 240 0 0 11 Apr 2025
Spatial Audio Processing with Large Language Model on Wearable Devices Ayushi Mishra Yang Bai Priyadarshan Narayanasamy Nakul Garg Nirupam Roy 30 0 0 11 Apr 2025
Enhancing Compositional Reasoning in Vision-Language Models with Synthetic Preference Data Samarth Mishra Kate Saenko Venkatesh Saligrama CoGe LRM 39 0 0 07 Apr 2025
On Large Multimodal Models as Open-World Image Classifiers Alessandro Conti Massimiliano Mancini Enrico Fini Yiming Wang Paolo Rota Elisa Ricci VLM Presented at ResearchTrend Connect \| VLM on 07 May 2025 91 0 0 27 Mar 2025
Fwd2Bot: LVLM Visual Token Compression with Double Forward Bottleneck Adrian Bulat Yassine Ouali Georgios Tzimiropoulos 241 0 0 27 Mar 2025
DWIM: Towards Tool-aware Visual Reasoning via Discrepancy-aware Workflow Generation & Instruct-Masking Tuning Fucai Ke Vijay Kumar B G Xingjian Leng Zhixi Cai Zaid Khan Weiqing Wang P. D. Haghighi H. Rezatofighi Manmohan Chandraker 51 0 0 25 Mar 2025
Training-Free Personalization via Retrieval and Reasoning on Fingerprints Deepayan Das Davide Talon Yiming Wang Massimiliano Mancini Elisa Ricci VLM LRM 52 0 0 24 Mar 2025
Seeing What Matters: Empowering CLIP with Patch Generation-to-Selection Gensheng Pei Tao Chen Yujia Wang Xinhao Cai Xiangbo Shu Tianfei Zhou Yazhou Yao VLM 58 1 0 21 Mar 2025
Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models Davide Berasi Matteo Farina Massimiliano Mancini Elisa Ricci Nicola Strisciuglio CoGe 70 0 0 21 Mar 2025
Is CLIP ideal? No. Can we fix it? Yes! Raphi Kang Yue Song Georgia Gkioxari Pietro Perona VLM 66 0 0 10 Mar 2025
Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data Haoxin Li Boyang Li CoGe 76 0 0 03 Mar 2025
CLIP Under the Microscope: A Fine-Grained Analysis of Multi-Object Representation Reza Abbasi Ali Nazari Aminreza Sefid Mohammadali Banayeeanzade M. Rohban M. Baghshah VLM 91 1 0 27 Feb 2025
Object-centric Binding in Contrastive Language-Image Pretraining Rim Assouel Pietro Astolfi Florian Bordes M. Drozdzal Adriana Romero Soriano OCL VLM CoGe 108 0 0 19 Feb 2025
From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models Mayank Vatsa Aparna Bharati S. Mittal Richa Singh 63 0 0 10 Feb 2025
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 96 2 0 02 Dec 2024
Sparse Attention Vectors: Generative Multimodal Model Features Are Discriminative Vision-Language Classifiers Chancharik Mitra Brandon Huang Tianning Chai Zhiqiu Lin Assaf Arbelle Rogerio Feris Leonid Karlinsky Trevor Darrell Deva Ramanan Roei Herzig VLM 137 4 0 28 Nov 2024
VidComposition: Can MLLMs Analyze Compositions in Compiled Videos? Yunlong Tang Junjia Guo Hang Hua Susan Liang Mingqian Feng ... Chao Huang Jing Bi Zeliang Zhang Pooyan Fazli Chenliang Xu CoGe 82 8 0 17 Nov 2024
TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives Maitreya Patel Abhiram Kusumba Sheng Cheng Changhoon Kim Tejas Gokhale Chitta Baral Yezhou Yang CoGe CLIP 62 7 0 04 Nov 2024
INQUIRE: A Natural World Text-to-Image Retrieval Benchmark Edward Vendrow Omiros Pantazis Alexander Shepard Gabriel J. Brostow Kate E. Jones Oisin Mac Aodha Sara Beery Grant Van Horn VLM 43 3 0 04 Nov 2024
NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples Baiqi Li Zhiqiu Lin Wenxuan Peng Jean de Dieu Nyandwi Daniel Jiang Zixian Ma Simran Khanuja Ranjay Krishna Graham Neubig Deva Ramanan AAML CoGe VLM 74 22 0 18 Oct 2024
Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad VLM 26 2 0 16 Oct 2024
CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learning Qingqing Cao Mahyar Najibi Sachin Mehta CLIP DiffM 35 1 0 15 Oct 2024
On Debiasing Text Embeddings Through Context Injection Thomas Uriot 37 0 0 14 Oct 2024
Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective Xiangru Zhu Penglei Sun Yaoxian Song Yanghua Xiao Zhixu Li Chengyu Wang Jun Huang Bei Yang Xiaoxiao Xu EGVM 272 1 0 14 Oct 2024
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models Hang Hua Yunlong Tang Ziyun Zeng Liangliang Cao Zhengyuan Yang Hangfeng He Chenliang Xu Jiebo Luo VLM CoGe 44 9 0 13 Oct 2024
Preserving Multi-Modal Capabilities of Pre-trained VLMs for Improving Vision-Linguistic Compositionality Youngtaek Oh Jae-Won Cho Dong-Jin Kim In So Kweon Junmo Kim VLM CoGe CLIP 35 4 0 07 Oct 2024
Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models Yufang Liu Tao Ji Changzhi Sun Yuanbin Wu Aimin Zhou VLM MLLM 49 2 0 04 Oct 2024
Removing Distributional Discrepancies in Captions Improves Image-Text Alignment Yuheng Li Haotian Liu Mu Cai Yijun Li Eli Shechtman Zhe Lin Yong Jae Lee Krishna Kumar Singh VLM 228 1 0 01 Oct 2024
The Hard Positive Truth about Vision-Language Compositionality Amita Kamath Cheng-Yu Hsieh Kai-Wei Chang Ranjay Krishna CLIP CoGe VLM 38 5 0 26 Sep 2024
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension Junzhuo Liu Xiaohu Yang Weiwei Li Peng Wang ObjD 61 3 0 23 Sep 2024
No Detail Left Behind: Revisiting Self-Retrieval for Fine-Grained Image Captioning Manu Gaur Darshan Singh Makarand Tapaswi 195 1 0 04 Sep 2024
ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation Wenlong Huang Chen Wang Yong Li Ruohan Zhang Li Fei-Fei 57 95 0 03 Sep 2024
Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models Gracjan Góral Alicja Ziarko Michal Nauman Maciej Wołczyk LRM 35 2 0 02 Sep 2024
UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling Haider Al-Tahan Q. Garrido Randall Balestriero Diane Bouchacourt C. Hazirbas Mark Ibrahim VLM 81 9 0 09 Aug 2024
The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communication Tom Kouwenhoven Max Peeperkorn Bram van Dijk Tessa Verhoef 34 3 0 25 Jul 2024
In-Context Learning Improves Compositional Understanding of Vision-Language Models Matteo Nulli Anesa Ibrahimi Avik Pal Hoshe Lee Ivona Najdenkoska VLM CoGe 35 0 0 22 Jul 2024
Chronologically Accurate Retrieval for Temporal Grounding of Motion-Language Models Kent Fujiwara Mikihiro Tanaka Qing Yu 59 2 0 22 Jul 2024
Weak-to-Strong Compositional Learning from Generative Models for Language-based Object Detection Kwanyong Park Kuniaki Saito Donghyun Kim VLM CoGe 58 0 0 21 Jul 2024
X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs S. Swetha Jinyu Yang T. Neiman Mamshad Nayeem Rizve Son Tran Benjamin Z. Yao Trishul Chilimbi Mubarak Shah 62 2 0 18 Jul 2024
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions Yu-Guan Hsieh Cheng-Yu Hsieh Shih-Ying Yeh Louis Béthune Hadi Pour Ansari Pavan Kumar Anasosalu Vasu Chun-Liang Li Ranjay Krishna Oncel Tuzel Marco Cuturi 66 4 0 09 Jul 2024
Deciphering the Role of Representation Disentanglement: Investigating Compositional Generalization in CLIP Models Reza Abbasi M. Rohban M. Baghshah CoGe 38 5 0 08 Jul 2024
SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Grounding Zixu Cheng Yujiang Pu Shaogang Gong Parisa Kordjamshidi Yu Kong AI4TS 38 0 0 06 Jul 2024
Automatic benchmarking of large multimodal models via iterative experiment programming Alessandro Conti Enrico Fini Paolo Rota Yiming Wang Massimiliano Mancini Elisa Ricci 48 0 0 18 Jun 2024
SUGARCREPE++ Dataset: Vision-Language Model Sensitivity to Semantic and Lexical Alterations Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad CoGe 43 9 0 17 Jun 2024