Probing Image-Language Transformers for Verb Understanding

16 June 2021

Papers citing "Probing Image-Language Transformers for Verb Understanding"

50 / 83 papers shown

Title
Compositional Image-Text Matching and Retrieval by Grounding Entities Madhukar Reddy Vongala Saurabh Srivastava Jana Kosecka CLIP CoGe VLM 36 0 0 04 May 2025
Quantifying Interpretability in CLIP Models with Concept Consistency Avinash Madasu Vasudev Lal Phillip Howard VLM 69 0 0 14 Mar 2025
MASS: Overcoming Language Bias in Image-Text Matching Jiwan Chung Seungwon Lim Sangkyu Lee Youngjae Yu VLM 32 0 0 20 Jan 2025
A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future Shilin Sun Wenbin An Feng Tian Fang Nan Qidong Liu Xiaozhong Liu N. Shah Ping Chen 96 2 0 18 Dec 2024
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey Yunkai Dang Kaichen Huang Jiahao Huo Yibo Yan S. Huang ... Kun Wang Yong Liu Jing Shao Hui Xiong Xuming Hu LRM 104 15 0 03 Dec 2024
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 85 2 0 02 Dec 2024
ACE: Action Concept Enhancement of Video-Language Models in Procedural Videos Reza Ghoddoosian Nakul Agarwal Isht Dwivedi Behzad Darisuh 68 0 0 23 Nov 2024
Multimodal Structure-Aware Quantum Data Processing Hala Hawashin Mehrnoosh Sadrzadeh 26 0 0 06 Nov 2024
Enhancing Motion in Text-to-Video Generation with Decomposed Encoding and Conditioning Penghui Ruan Pichao Wang Divya Saxena Jiannong Cao Yuhui Shi DiffM VGen 36 78 0 31 Oct 2024
Beyond Coarse-Grained Matching in Video-Text Retrieval Aozhu Chen Hazel Doughty Xirong Li Cees G. M. Snoek 36 0 0 16 Oct 2024
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models Hang Hua Yunlong Tang Ziyun Zeng Liangliang Cao Zhengyuan Yang Hangfeng He Chenliang Xu Jiebo Luo VLM CoGe 44 9 0 13 Oct 2024
Preserving Multi-Modal Capabilities of Pre-trained VLMs for Improving Vision-Linguistic Compositionality Youngtaek Oh Jae-Won Cho Dong-Jin Kim In So Kweon Junmo Kim VLM CoGe CLIP 27 4 0 07 Oct 2024
The Hard Positive Truth about Vision-Language Compositionality Amita Kamath Cheng-Yu Hsieh Kai-Wei Chang Ranjay Krishna CLIP CoGe VLM 32 5 0 26 Sep 2024
ComAlign: Compositional Alignment in Vision-Language Models Ali Abdollah Amirmohammad Izadi Armin Saghafian Reza Vahidimajd Mohammad Mozafari Amirreza Mirzaei Mohammadmahdi Samiei M. Baghshah CoGe VLM 30 0 0 12 Sep 2024
Quantifying and Enabling the Interpretability of CLIP-like Models Avinash Madasu Yossi Gandelsman Vasudev Lal Phillip Howard VLM 56 2 0 10 Sep 2024
CV-Probes: Studying the interplay of lexical and world knowledge in visually grounded verb understanding Ivana Beňová Michal Gregor Albert Gatt 40 0 0 02 Sep 2024
ActPrompt: In-Domain Feature Adaptation via Action Cues for Video Temporal Grounding Yubin Wang Xinyang Jiang De Cheng Dongsheng Li Cairong Zhao VLM 43 1 0 13 Aug 2024
How and where does CLIP process negation? Vincent Quantmeyer Pablo Mosteiro Albert Gatt CoGe 29 6 0 15 Jul 2024
What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation Michal Golovanevsky William Rudman Vedant Palit Ritambhara Singh Carsten Eickhoff 33 1 0 24 Jun 2024
FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture Wenyan Li Xinyu Crystina Zhang Jiaang Li Qiwei Peng Raphael Tang ... Guimin Hu Yifei Yuan Anders Søgaard Daniel Hershcovich Desmond Elliott CoGe 35 7 0 16 Jun 2024
Exploring the Spectrum of Visio-Linguistic Compositionality and Recognition Youngtaek Oh Pyunghwan Ahn Jinhyung Kim Gwangmo Song Soonyoung Lee In So Kweon Junmo Kim CoGe 48 2 0 13 Jun 2024
Comparison Visual Instruction Tuning Wei Lin M. Jehanzeb Mirza Sivan Doveh Rogerio Feris Raja Giryes Sepp Hochreiter Leonid Karlinsky 46 4 0 13 Jun 2024
Don't Buy it! Reassessing the Ad Understanding Abilities of Contrastive Multimodal Models A. Bavaresco A. Testoni Raquel Fernández 33 2 0 31 May 2024
Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognition Yuchen Zhou Linkai Liu Chao Gou 40 3 0 16 May 2024
Exploring Vision Transformers for 3D Human Motion-Language Models with Motion Patches Qing Yu Mikihiro Tanaka Kent Fujiwara ViT 47 2 0 08 May 2024
Q-GroundCAM: Quantifying Grounding in Vision Language Models via GradCAM Navid Rajabi Jana Kosecka 28 1 0 29 Apr 2024
Iterated Learning Improves Compositionality in Large Vision-Language Models Chenhao Zheng Jieyu Zhang Aniruddha Kembhavi Ranjay Krishna VLM CoGe 54 9 0 02 Apr 2024
If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions Reza Esfandiarpoor Cristina Menghini Stephen H. Bach CoGe VLM 40 8 0 25 Mar 2024
Can 3D Vision-Language Models Truly Understand Natural Language? Weipeng Deng Jihan Yang Runyu Ding Jiahui Liu Yijiang Li Xiaojuan Qi Edith C.H. Ngai 39 4 0 21 Mar 2024
Enhancing Conceptual Understanding in Multimodal Contrastive Learning through Hard Negative Samples Philipp J. Rösch Norbert Oswald Michaela Geierhos Jindrich Libovický 42 3 0 05 Mar 2024
CLoVe: Encoding Compositional Language in Contrastive Vision-Language Models Santiago Castro Amir Ziai Avneesh Saluja Zhuoning Yuan Rada Mihalcea MLLM CoGe VLM 34 5 0 22 Feb 2024
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 43 29 0 20 Feb 2024
Beyond Image-Text Matching: Verb Understanding in Multimodal Transformers Using Guided Masking Ivana Beňová Jana Kosecka Michal Gregor Martin Tamajka Marcel Veselý Marian Simko 30 1 0 29 Jan 2024
Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers Aleksandar Stanić Sergi Caelles Michael Tschannen LRM VLM 27 9 0 03 Jan 2024
Adventures of Trustworthy Vision-Language Models: A Survey Mayank Vatsa Anubhooti Jain Richa Singh 27 4 0 07 Dec 2023
OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition Tom Tongjia Chen Hongshan Yu Zhengeng Yang Zechuan Li Wei Sun Chen Chen 23 8 0 30 Nov 2023
Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding Wujian Peng Sicheng Xie Zuyao You Shiyi Lan Zuxuan Wu VLM CoGe MLLM 30 18 0 30 Nov 2023
ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation Yangyi Chen Xingyao Wang Manling Li Derek Hoiem Heng Ji 30 11 0 22 Nov 2023
SPOT! Revisiting Video-Language Models for Event Understanding Gengyuan Zhang Jinhe Bi Jindong Gu Yanyu Chen Volker Tresp 29 2 0 21 Nov 2023
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models .Ilker Kesen Andrea Pedrotti Mustafa Dogan Michele Cafagna Emre Can Acikgoz ... Iacer Calixto Anette Frank Albert Gatt Aykut Erdem Erkut Erdem 41 15 0 13 Nov 2023
The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models Xinyi Chen Raquel Fernández Sandro Pezzelle VLM 18 9 0 23 Oct 2023
The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models Chenwei Wu Erran L. Li Stefano Ermon Patrick Haffner Rong Ge Zaiwei Zhang VLM CoGe 26 0 0 04 Oct 2023
The Scenario Refiner: Grounding subjects in images at the morphological level Claudia Tagliaferri Sofia Axioti Albert Gatt Denis Paperno 24 1 0 20 Sep 2023
ViLTA: Enhancing Vision-Language Pre-training through Textual Augmentation Weihan Wang Zhengyuan Yang Bin Xu Juanzi Li Yankui Sun VLM 28 8 0 31 Aug 2023
Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining? Fei Wang Liang Ding Jun Rao Ye Liu Li Shen Changxing Ding 32 15 0 24 Aug 2023
Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Models Navid Rajabi Jana Kosecka VLM 34 11 0 18 Aug 2023
Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Models Zheng Ma Mianzhi Pan Wenhan Wu Ka Leong Cheng Jianbing Zhang Shujian Huang Jiajun Chen VLM CoGe 26 3 0 06 Aug 2023
Grounded Image Text Matching with Mismatched Relation Reasoning Yu Wu Yan-Tao Wei Haozhe Jasper Wang Yongfei Liu Sibei Yang Xuming He 34 6 0 02 Aug 2023
ICSVR: Investigating Compositional and Syntactic Understanding in Video Retrieval Models Avinash Madasu Vasudev Lal CoGe 42 3 0 28 Jun 2023
Towards In-context Scene Understanding Ivana Balazevic David Steiner Nikhil Parthasarathy Relja Arandjelović Olivier J. Hénaff 35 28 0 02 Jun 2023