Supervised Multimodal Bitransformers for Classifying Images and Text

6 September 2019

Douwe Kiela

Papers citing "Supervised Multimodal Bitransformers for Classifying Images and Text"

50 / 116 papers shown

Title
Transformadores: Fundamentos teoricos y Aplicaciones J. D. L. Torre 78 0 0 18 Feb 2023
A dataset for Audio-Visual Sound Event Detection in Movies Rajat Hebbar Digbalay Bose Krishna Somandepalli Veena Vijai Shrikanth Narayanan 14 8 0 14 Feb 2023
Cross-Modal Fine-Tuning: Align then Refine Junhong Shen Liam Li Lucio Dery Corey Staten M. Khodak Graham Neubig Ameet Talwalkar 45 36 0 11 Feb 2023
Prompting for Multimodal Hateful Meme Classification Rui Cao Roy Ka-wei Lee Wen-Haw Chong Jing Jiang VLM 30 77 0 08 Feb 2023
Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim? Shivam Sharma Atharva Kulkarni Tharun Suresh Himanshi Mathur Preslav Nakov Md. Shad Akhtar Tanmoy Chakraborty 56 15 0 26 Jan 2023
A Concept Knowledge Graph for User Next Intent Prediction at Alipay Yacheng He Qianghuai Jia Lin Yuan Ruopeng Li Yixin Ou Ningyu Zhang 38 5 0 02 Jan 2023
Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Features Gokul Karthik Kumar Karthik Nandakumar VLM CLIP 40 59 0 12 Oct 2022
VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment Shraman Pramanick Li Jing Sayan Nag Jiachen Zhu Hardik Shah Yann LeCun Ramalingam Chellappa 42 21 0 09 Oct 2022
Domain-aware Self-supervised Pre-training for Label-Efficient Meme Analysis Shivam Sharma Mohd Khizir Siddiqui Md. Shad Akhtar Tanmoy Chakraborty SSL 36 5 0 29 Sep 2022
DM $^2$ S $^2$ : Deep Multi-Modal Sequence Sets with Hierarchical Modality Attention Shunsuke Kitada Yuki Iwazaki Riku Togashi Hitoshi Iyatomi 40 1 0 07 Sep 2022
Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework Ahmed M. Mahran C. Borella K. Perifanos 27 1 0 14 Jun 2022
DISARM: Detecting the Victims Targeted by Harmful Memes Shivam Sharma Md. Shad Akhtar Preslav Nakov Tanmoy Chakraborty 38 30 0 11 May 2022
SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalities Pengbo Hu Xingyu Li Yi Zhou 63 10 0 30 Apr 2022
Trusted Multi-View Classification with Dynamic Evidential Fusion Zongbo Han Changqing Zhang Huazhu Fu Qiufeng Wang EDL 39 223 0 25 Apr 2022
Are Multimodal Transformers Robust to Missing Modality? Mengmeng Ma Jian Ren Long Zhao Davide Testuggine Xi Peng ViT 76 149 0 12 Apr 2022
CARETS: A Consistency And Robustness Evaluative Test Suite for VQA Carlos E. Jimenez Olga Russakovsky Karthik Narasimhan CoGe 39 14 0 15 Mar 2022
HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsing Yanzhao Zheng Haibin Wang B. Dong Xingjun Wang Changshan Li 40 32 0 14 Mar 2022
Multi-Modal Attribute Extraction for E-Commerce Aloïs de La Comble Anuvabh Dutt Pablo Montalvo Aghiles Salah 39 10 0 07 Mar 2022
High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Jeffrey Tsaw Yudong Liu Shentong Mo Dani Yogatama Louis-Philippe Morency Ruslan Salakhutdinov 22 29 0 02 Mar 2022
A Survey of Vision-Language Pre-Trained Models Yifan Du Zikang Liu Junyi Li Wayne Xin Zhao VLM 64 181 0 18 Feb 2022
Indication as Prior Knowledge for Multimodal Disease Classification in Chest Radiographs with Transformers Grzegorz Jacenków Alison Q. OÑeil Sotirios A. Tsaftaris ViT MedIm 39 23 0 12 Feb 2022
MMLN: Leveraging Domain Knowledge for Multimodal Diagnosis Haodi Zhang Chenyu Xu Pei-hong Liang Ke Duan Haopan Ren Weibin Cheng Kaishun Wu 33 0 0 09 Feb 2022
Understanding and Measuring Robustness of Multimodal Learning Nishant Vishwamitra Hongxin Hu Ziming Zhao Long Cheng Feng Luo AAML 32 5 0 22 Dec 2021
Fusion of medical imaging and electronic health records with attention and multi-head machanisms Cheng Jiang Yihao Chen Jianbo Chang M. Feng Renzhi Wang Jianhua Yao 29 8 0 22 Dec 2021
Insta-VAX: A Multimodal Benchmark for Anti-Vaccine and Misinformation Posts Detection on Social Media Mingyang Zhou Mahasweta Chakraborti Sijia Qian Zhou Yu Jingwen Zhang 56 1 0 15 Dec 2021
CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classification Huidong Liu Shaoyuan Xu Jinmiao Fu Yang Liu Ning Xie Chien Wang Bryan Wang Yi Sun CLIP VLM 32 27 0 07 Dec 2021
Multimodal Learning using Optimal Transport for Sarcasm and Humor Detection Shraman Pramanick A. Roy Vishal M. Patel 40 57 0 21 Oct 2021
Understanding of Emotion Perception from Art Digbalay Bose Krishna Somandepalli Souvik Kundu Rimita Lahiri Jonathan Gratch Shrikanth Narayanan 21 4 0 13 Oct 2021
Detecting Harmful Memes and Their Targets Shraman Pramanick Dimitar Dimitrov Rituparna Mukherjee Shivam Sharma Md. Shad Akhtar Preslav Nakov Tanmoy Chakraborty 28 111 0 24 Sep 2021
MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets Shraman Pramanick Shivam Sharma Dimitar Dimitrov Md. Shad Akhtar Preslav Nakov Tanmoy Chakraborty 33 120 0 11 Sep 2021
TxT: Crossmodal End-to-End Learning with Transformers Jan-Martin O. Steitz Jonas Pfeiffer Iryna Gurevych Stefan Roth LRM 21 2 0 09 Sep 2021
TrollsWithOpinion: A Dataset for Predicting Domain-specific Opinion Manipulation in Troll Memes Shardul Suryawanshi Bharathi Raja Chakravarthi Mihael Arcan Suzanne Little P. Buitelaar 6 5 0 08 Sep 2021
Multimodal Conditionality for Natural Language Generation Michael Sollami Aashish Jain 24 10 0 02 Sep 2021
Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach Chuanbo Hu Minglei Yin Bin Liu Xin Li Yanfang Ye 24 15 0 19 Aug 2021
Detecting Propaganda Techniques in Memes Dimitar Dimitrov Bishr Bin Ali Shaden Shaar Firoj Alam Fabrizio Silvestri Hamed Firooz Preslav Nakov Giovanni Da San Martino 53 94 0 07 Aug 2021
Exceeding the Limits of Visual-Linguistic Multi-Task Learning Cameron R. Wolfe Keld T. Lundgaard VLM 50 2 0 27 Jul 2021
DRDF: Determining the Importance of Different Multimodal Information with Dual-Router Dynamic Framework Haiwen Hong Xuan Jin Yin Zhang Yunqing Hu Jingfeng Zhang Yuan He Hui Xue MoE 27 0 0 21 Jul 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 42 161 0 15 Jul 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 96 761 0 25 Jun 2021
DocFormer: End-to-End Transformer for Document Understanding Srikar Appalaraju Bhavan A. Jasani Bhargava Urala Kota Yusheng Xie R. Manmatha ViT 46 274 0 22 Jun 2021
Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate Austin Botelho Bertie Vidgen Scott A. Hale 24 8 0 10 Jun 2021
Human-Adversarial Visual Question Answering Sasha Sheng Amanpreet Singh Vedanuj Goswami Jose Alberto Lopez Magana Wojciech Galuba Devi Parikh Douwe Kiela OOD EgoV AAML 26 60 0 04 Jun 2021
Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution Yang Li Zi-xin Zhang Hutchin Huang 27 1 0 25 May 2021
Analyzing Online Political Advertisements Danae Sánchez Villegas S. Mokaram Nikolaos Aletras 30 11 0 09 May 2021
SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images Dimitar Dimitrov Bishr Bin Ali Shaden Shaar Firoj Alam Fabrizio Silvestri Hamed Firooz Preslav Nakov Giovanni Da San Martino 23 104 0 25 Apr 2021
Cross-Modal Retrieval Augmentation for Multi-Modal Classification Shir Gur Natalia Neverova C. Stauffer Ser-Nam Lim Douwe Kiela A. Reiter 52 26 0 16 Apr 2021
Multimodal Fusion Refiner Networks Sethuraman Sankaran David Yang Ser-Nam Lim OffRL 34 8 0 08 Apr 2021
A Survey on Multimodal Disinformation Detection Firoj Alam S. Cresci Tanmoy Chakraborty Fabrizio Silvestri Dimiter Dimitrov Giovanni Da San Martino Shaden Shaar Hamed Firooz Preslav Nakov 51 98 0 13 Mar 2021
Pretrained Transformers as Universal Computation Engines Kevin Lu Aditya Grover Pieter Abbeel Igor Mordatch 40 220 0 09 Mar 2021
Trusted Multi-View Classification Zongbo Han Changqing Zhang Huazhu Fu Qiufeng Wang EDL 31 166 0 03 Feb 2021