Supervised Multimodal Bitransformers for Classifying Images and Text

6 September 2019

Douwe Kiela

Papers citing "Supervised Multimodal Bitransformers for Classifying Images and Text"

50 / 116 papers shown

Title
Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models Minh-Hao Van Xintao Wu VLM 97 0 0 30 Apr 2025
Are you SURE? Enhancing Multimodal Pretraining with Missing Modalities through Uncertainty Estimation Duy Nguyen Quan Huu Do Khoa D. Doan Minh N. Do 42 0 0 18 Apr 2025
See-Saw Modality Balance: See Gradient, and Sew Impaired Vision-Language Balance to Mitigate Dominant Modality Bias Junehyoung Kwon Mihyeon Kim Eunju Lee Juhwan Choi Youngbin Kim 69 0 0 18 Mar 2025
MICINet: Multi-Level Inter-Class Confusing Information Removal for Reliable Multimodal Classification Tianze Zhang Shu Shen Chao Chen 87 0 0 27 Feb 2025
MATCHED: Multimodal Authorship-Attribution To Combat Human Trafficking in Escort-Advertisement Data V. Saxena Benjamin Bashpole Gijs van Dijck Gerasimos Spanakis 84 0 0 18 Dec 2024
GAMED: Knowledge Adaptive Multi-Experts Decoupling for Multimodal Fake News Detection Lingzhi Shen Yunfei Long Xiaohao Cai Imran Razzak Guanming Chen Kang Liu Shoaib Jameel MoE 88 3 0 11 Dec 2024
Approximate Fiber Product: A Preliminary Algebraic-Geometric Perspective on Multimodal Embedding Alignment Dongfang Zhao 68 0 0 30 Nov 2024
Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes Rahul Garg Trilok Padhi Hemang Jain Ugur Kursuncu Ponnurangam Kumaraguru 103 4 0 19 Nov 2024
Prompt-enhanced Network for Hateful Meme Classification Junxi Liu Yanyan Feng Jiehai Chen Yun Xue Fenghuan Li VLM 80 0 0 12 Nov 2024
Towards Low-Resource Harmful Meme Detection with LMM Agents Jianzhao Huang Hongzhan Lin Ziyan Liu Ziyang Luo Guang Chen Jing Ma 54 3 0 08 Nov 2024
M3Hop-CoT: Misogynous Meme Identification with Multimodal Multi-hop Chain-of-Thought G. Kumari Kirtan Jain Asif Ekbal 48 1 0 11 Oct 2024
Harnessing Shared Relations via Multimodal Mixup Contrastive Learning for Multimodal Classification Raja Kumar Raghav Singhal Pranamya Kulkarni Deval Mehta Kshitij Jadhav 31 0 0 26 Sep 2024
Multimodal Generalized Category Discovery Yuchang Su Renping Zhou Siyu Huang Xingjian Li Tianyang Wang Ziyue Wang Min Xu 58 0 0 18 Sep 2024
Modality Invariant Multimodal Learning to Handle Missing Modalities: A Single-Branch Approach Muhammad Saad Saeed Shah Nawaz Muhammad Zaigham Zaheer Muhammad Haris Khan Karthik Nandakumar Muhammad Haroon Yousaf Hassan Sajjad Tom De Schepper Markus Schedl 60 0 0 14 Aug 2024
Chameleon: Images Are What You Need For Multimodal Learning Robust To Missing Modalities Muhammad Irzam Liaqat Shah Nawaz Muhammad Zaigham Zaheer M. S. Saeed Hassan Sajjad Tom De Schepper Karthik Nandakumar Muhammad Haris Khan 53 1 0 23 Jul 2024
I Know About "Up"! Enhancing Spatial Reasoning in Visual Language Models Through 3D Reconstruction Zaiqiao Meng Hao Zhou Yifang Chen 47 4 0 19 Jul 2024
Robust Latent Representation Tuning for Image-text Classification Hao Sun Yu Song VLM 79 0 0 10 Jun 2024
Predictive Dynamic Fusion Bing Cao Yinan Xia Yi Ding Changqing Zhang Qinghua Hu 39 9 0 07 Jun 2024
ArMeme: Propagandistic Content in Arabic Memes Firoj Alam A. Hasnat Fatema Ahmed Md. Arid Hasan Maram Hasanain 61 7 0 06 Jun 2024
TT-BLIP: Enhancing Fake News Detection Using BLIP and Tri-Transformer Eunjee Choi Jong-Kook Kim 45 2 0 19 Mar 2024
Deciphering Hate: Identifying Hateful Memes and Their Targets E. Hossain Omar Sharif M. M. Hoque S. Preum 57 4 0 16 Mar 2024
CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning Peiyuan Liu Hang Guo Tao Dai Naiqi Li Jigang Bao Xudong Ren Yong Jiang Shu-Tao Xia AI4TS 72 18 0 12 Mar 2024
Multimodal Infusion Tuning for Large Models Hao Sun Yu Song Xinyao Yu Jiaqing Liu Yen-Wei Chen Lanfen Lin VLM 60 0 0 08 Mar 2024
MemeCraft: Contextual and Stance-Driven Multimodal Meme Generation Han Wang Roy Ka-wei Lee 39 7 0 24 Feb 2024
Can Text-to-image Model Assist Multi-modal Learning for Visual Recognition with Visual Modality Missing? Tiantian Feng Daniel Yang Digbalay Bose Shrikanth Narayanan 59 5 0 14 Feb 2024
Text Role Classification in Scientific Charts Using Multimodal Transformers Hye Jin Kim N. Lell A. Scherp 29 0 0 08 Feb 2024
Memory-Inspired Temporal Prompt Interaction for Text-Image Classification Xinyao Yu Hao Sun Ziwei Niu Rui Qin Zhenjia Bai Yen-Wei Chen Lanfen Lin VLM 63 2 0 26 Jan 2024
Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models Hongzhan Lin Ziyang Luo Wei Gao Jing Ma Bo Wang Ruichao Yang 42 14 0 24 Jan 2024
CrisisKAN: Knowledge-infused and Explainable Multimodal Attention Network for Crisis Event Classification Shubham Gupta Nandini Saini Suman Kundu Debasis Das 18 7 0 11 Jan 2024
Knowledge-enhanced Multi-perspective Video Representation Learning for Scene Recognition Xuzheng Yu Chen Jiang Wei Zhang Tian Gan Linlin Chao Jianan Zhao Yuan Cheng Qingpei Guo Wei Chu 41 0 0 09 Jan 2024
Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models Nishant Vishwamitra Keyan Guo Farhan Tajwar Romit Isabelle Ondracek Long Cheng Ziming Zhao Hongxin Hu 27 13 0 22 Dec 2023
Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models Hongzhan Lin Ziyang Luo Jing Ma Long Chen 40 9 0 09 Dec 2023
Conditional Prompt Tuning for Multimodal Fusion Ruixia Jiang Lingbo Liu Changwen Chen 44 0 0 28 Nov 2023
Detecting and Correcting Hate Speech in Multimodal Memes with Large Visual Language Model Minh-Hao Van Xintao Wu VLM MLLM 52 10 0 12 Nov 2023
BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classification Mithun Das Animesh Mukherjee 34 5 0 18 Oct 2023
Incorporating Domain Knowledge Graph into Multimodal Movie Genre Classification with Self-Supervised Attention and Contrastive Learning Jiaqi Li Guilin Qi Chuanyi Zhang Yongrui Chen Yiming Tan Chenlong Xia Ye Tian 46 3 0 12 Oct 2023
What Makes for Robust Multi-Modal Models in the Face of Missing Modalities? Siting Li Chenzhuang Du Yue Zhao Yu Huang Hang Zhao 29 4 0 10 Oct 2023
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 60 2 0 08 Oct 2023
Improving Multimodal Classification of Social Media Posts by Leveraging Image-Text Auxiliary Tasks Danae Sánchez Villegas Daniel Preoctiuc-Pietro Nikolaos Aletras 41 3 0 14 Sep 2023
A Multimodal Analysis of Influencer Content on Twitter Danae Sánchez Villegas Catalina Goanta Nikolaos Aletras 30 6 0 06 Sep 2023
Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection Rui Cao Ming Shan Hee Adriel Kuek Wen-Haw Chong Roy Ka-wei Lee Jing Jiang VLM MLLM 32 38 0 16 Aug 2023
Robust Visual Question Answering: Datasets, Methods, and Future Challenges Jie Ma Pinghui Wang Dechen Kong Zewei Wang Jun Liu Hongbin Pei Junzhou Zhao OOD 49 18 0 21 Jul 2023
Switch-BERT: Learning to Model Multimodal Interactions by Switching Attention and Input Qingpei Guo Kaisheng Yao Wei Chu MLLM 33 4 0 25 Jun 2023
Modality Influence in Multimodal Machine Learning Abdelhamid Haouhat Slimane Bellaouar A. Nehar H. Cherroun 47 2 0 10 Jun 2023
Provable Dynamic Fusion for Low-Quality Multimodal Data Qingyang Zhang Haitao Wu Changqing Zhang Qinghua Hu Huazhu Fu Qiufeng Wang Xi Peng 45 57 0 03 Jun 2023
Improving Generalization for Multimodal Fake News Detection Sahar Tahmasebi Sherzod Hakimov Ralph Ewerth Eric Müller-Budack 25 5 0 29 May 2023
MEMEX: Detecting Explanatory Evidence for Memes via Knowledge-Enriched Contextualization Shivam Sharma S Ramaneswaran Udit Arora Md. Shad Akhtar Tanmoy Chakraborty 46 9 0 25 May 2023
UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning Heqing Zou Meng Shen Chen Chen Yuchen Hu D. Rajan Chng Eng Siong SSL 49 16 0 16 May 2023
Efficient Multimodal Fusion via Interactive Prompting Yaowei Li Ruijie Quan Linchao Zhu Yezhou Yang 40 44 0 13 Apr 2023
TOT: Topology-Aware Optimal Transport For Multimodal Hate Detection Linhao Zhang Li Jin Xian Sun Guangluan Xu Zequn Zhang Xiaoyu Li Nayu Liu Qing Liu Shiyao Yan 49 8 0 27 Feb 2023