ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph

30 June 2020

Papers citing "ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph"

50 / 208 papers shown

Title
Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks Miran Heo Min-Hung Chen De-An Huang Sifei Liu Subhashree Radhakrishnan Seon Joo Kim Yu-Chun Wang Ryo Hachiuma ObjD VLM 159 2 0 14 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 99 48 0 03 Jan 2025
A Comprehensive Survey on Visual Question Answering Datasets and Algorithms Raihan Kabir Naznin Haque Md. Saiful Islam Marium-E. Jannat CoGe 29 1 0 17 Nov 2024
Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP Chen Huang Skyler Seto Samira Abnar David Grangier Navdeep Jaitly J. Susskind VLM 51 0 0 31 Oct 2024
CMAL: A Novel Cross-Modal Associative Learning Framework for Vision-Language Pre-Training Zhiyuan Ma Jianjun Li Guohui Li Kaiyan Huang VLM 56 9 0 16 Oct 2024
Make Graph-based Referring Expression Comprehension Great Again through Expression-guided Dynamic Gating and Regression Jingcheng Ke Dele Wang Jun-Cheng Chen I-Hong Jhuo Chia-Wen Lin Yen-Yu Lin 33 0 0 05 Sep 2024
Efficient and Versatile Robust Fine-Tuning of Zero-shot Models Sungyeon Kim Boseung Jeong Donghyun Kim Suha Kwak VLM 33 2 0 11 Aug 2024
wav2graph: A Framework for Supervised Learning Knowledge Graph from Speech Khai Le-Duc Quy-Anh Dang Tan-Hanh Pham Truong Son-Hy 32 0 0 08 Aug 2024
MMCLIP: Cross-modal Attention Masked Modelling for Medical Language-Image Pre-Training Biao Wu Yutong Xie Zeyu Zhang Minh Hieu Phan Qi Chen Ling-Hao Chen Qi Wu LM&MA 37 0 0 28 Jul 2024
Unified Video-Language Pre-training with Synchronized Audio Shentong Mo Haofan Wang Huaxia Li Xu Tang 35 2 0 12 May 2024
Large Language Models for UAVs: Current State and Pathways to the Future Shumaila Javaid Nasir Saeed Bin He 40 17 0 02 May 2024
EventLens: Leveraging Event-Aware Pretraining and Cross-modal Linking Enhances Visual Commonsense Reasoning Mingjie Ma Zhihuan Yu Yichao Ma Guohui Li LRM 41 1 0 22 Apr 2024
Knowledge-enhanced Visual-Language Pretraining for Computational Pathology Xiao Zhou Xiaoman Zhang Chaoyi Wu Ya-Qin Zhang Weidi Xie Yanfeng Wang VLM 35 7 0 15 Apr 2024
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want Weifeng Lin Xinyu Wei Ruichuan An Peng Gao Bocheng Zou Yulin Luo Siyuan Huang Shanghang Zhang Hongsheng Li VLM 66 33 0 29 Mar 2024
Beyond Embeddings: The Promise of Visual Table in Visual Reasoning Yiwu Zhong Zi-Yuan Hu Michael R. Lyu Liwei Wang 29 1 0 27 Mar 2024
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World Weiyun Wang Yiming Ren Hao Luo Tiantong Li Chenxiang Yan ... Qingyun Li Lewei Lu Xizhou Zhu Yu Qiao Jifeng Dai MLLM 52 47 0 29 Feb 2024
Acquiring Linguistic Knowledge from Multimodal Input Theodor Amariucai Alexander Scott Warstadt CLL 29 2 0 27 Feb 2024
MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning Zhe Li Laurence T. Yang Bocheng Ren Xin Nie Zhangyang Gao Cheng Tan Stan Z. Li VLM 15 12 0 03 Feb 2024
Enhancing medical vision-language contrastive learning via inter-matching relation modelling Mingjian Li Mingyuan Meng M. Fulham David Dagan Feng Lei Bi Jinman Kim VLM 40 1 0 19 Jan 2024
Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection Wei Ye Chaoya Jiang Haiyang Xu Chenhao Ye Chenliang Li Mingshi Yan Shikun Zhang Songhang Huang Fei Huang VLM 29 0 0 11 Jan 2024
Few-shot Adaptation of Multi-modal Foundation Models: A Survey Fan Liu Tianshu Zhang Wenwen Dai Wenwen Cai Wenwen Cai Xiaocong Zhou Delong Chen VLM OffRL 31 23 0 03 Jan 2024
Cycle-Consistency Learning for Captioning and Grounding Ning Wang Jiajun Deng Mingbo Jia ObjD 42 7 0 23 Dec 2023
TiMix: Text-aware Image Mixing for Effective Vision-Language Pre-training Chaoya Jiang Wei Ye Haiyang Xu Qinghao Ye Mingshi Yan Ji Zhang Shikun Zhang CLIP VLM 21 4 0 14 Dec 2023
ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts Mu Cai Haotian Liu Dennis Park Siva Karthik Mustikovela Gregory P. Meyer Yuning Chai Yong Jae Lee VLM LRM MLLM 46 85 0 01 Dec 2023
Compositional Chain-of-Thought Prompting for Large Multimodal Models Chancharik Mitra Brandon Huang Trevor Darrell Roei Herzig MLLM LRM 36 80 0 27 Nov 2023
Causality is all you need Ning Xu Yifei Gao Hongshuo Tian Yongdong Zhang An-An Liu 41 0 0 21 Nov 2023
Active Prompt Learning in Vision Language Models Jihwan Bang Sumyeong Ahn Jae-Gil Lee VLM 13 9 0 18 Nov 2023
Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learning Jingbiao Mei Jinghong Chen Weizhe Lin Bill Byrne Marcus Tomalin VLM 24 4 0 14 Nov 2023
Improving Vision-and-Language Reasoning via Spatial Relations Modeling Cheng Yang Rui Xu Ye Guo Peixiang Huang Yiru Chen Wenkui Ding Zhongyuan Wang Hong Zhou LRM 21 5 0 09 Nov 2023
Augment the Pairs: Semantics-Preserving Image-Caption Pair Augmentation for Grounding-Based Vision and Language Models Jingru Yi Burak Uzkent Oana Ignat Zili Li Amanmeet Garg Xiang Yu Linda Liu VLM 38 1 0 05 Nov 2023
Open-Set Image Tagging with Multi-Grained Text Supervision Xinyu Huang Yi-Jie Huang Youcai Zhang Weiwei Tian Rui Feng Yuejie Zhang Yanchun Xie Yaqian Li Lei Zhang VLM 27 28 0 23 Oct 2023
VidChapters-7M: Video Chapters at Scale Antoine Yang Arsha Nagrani Ivan Laptev Josef Sivic Cordelia Schmid VGen 23 26 0 25 Sep 2023
Predicate Classification Using Optimal Transport Loss in Scene Graph Generation Sorachi Kurita Satoshi Oyama Itsuki Noda OT 24 0 0 19 Sep 2023
GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training Xi Deng Han Shi Runhu Huang Changlin Li Hang Xu Jianhua Han James T. Kwok Shen Zhao Wei Zhang Xiaodan Liang CLIP VLM 29 3 0 22 Aug 2023
Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memes Yosuke Miyanishi M. Nguyen 31 2 0 19 Aug 2023
BUS:Efficient and Effective Vision-language Pre-training with Bottom-Up Patch Summarization Chaoya Jiang Haiyang Xu Wei Ye Qinghao Ye Chenliang Li Mingshi Yan Bin Bi Shikun Zhang Fei Huang Songfang Huang VLM 31 9 0 17 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo VLM MLLM 85 224 0 07 Jul 2023
Efficient Token-Guided Image-Text Retrieval with Consistent Multimodal Contrastive Training Chong Liu Yuqi Zhang Hongsong Wang Weihua Chen F. Wang Yan Huang Yixing Shen Liang Wang 19 25 0 15 Jun 2023
MoviePuzzle: Visual Narrative Reasoning through Multimodal Order Learning Jianghui Wang Yuxuan Wang Dongyan Zhao Zilong Zheng 46 1 0 04 Jun 2023
DKINet: Medication Recommendation via Domain Knowledge Informed Deep Learning Sicen Liu Xiaolong Wang Xianbing Zhao Haoxing Chen 35 0 0 31 May 2023
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning Chia-Wen Kuo Z. Kira 31 21 0 25 May 2023
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining Emanuele Bugliarello Aida Nematzadeh Lisa Anne Hendricks SSL 24 5 0 23 May 2023
Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modality Jialing Yuan Ye Yu Gaurav Mittal Matthew Hall Sandra Sajeev Mei Chen 17 9 0 17 May 2023
Probing the Role of Positional Information in Vision-Language Models Philipp J. Rösch Jindrich Libovický 16 8 0 17 May 2023
Semantic Composition in Visually Grounded Language Models Rohan Pandey CoGe 23 1 0 15 May 2023
Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs Roei Herzig Alon Mendelson Leonid Karlinsky Assaf Arbelle Rogerio Feris Trevor Darrell Amir Globerson VLM 35 31 0 10 May 2023
A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues Yunxin Li Baotian Hu Xinyu Chen Yuxin Ding Lin Ma Min Zhang LRM 48 14 0 08 May 2023
Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation Chaoya Jiang Wei Ye Haiyang Xu Miang yan Shikun Zhang Jie Zhang Fei Huang VLM 34 15 0 08 May 2023
Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-modal Structured Representations Yufen Huang Jiji Tang Zhuo Chen Rongsheng Zhang Xinfeng Zhang ... Zeng Zhao Zhou Zhao Tangjie Lv Zhipeng Hu Wen Zhang VLM 20 21 0 06 May 2023
Few-shot Domain-Adaptive Visually-fused Event Detection from Text Farhad Moghimifar Fatemeh Shiri Van Nguyen Gholamreza Haffari Yuanyou Li VLM 30 2 0 04 May 2023