Gated Multimodal Units for Information Fusion

7 February 2017

John Arevalo

Thamar Solorio

Manuel Montes-y-Gómez

Fabio Gonzalez

ArXiv PDF HTML

Papers citing "Gated Multimodal Units for Information Fusion"

50 / 56 papers shown

Title
PREMISE: Matching-based Prediction for Accurate Review Recommendation Wei Han Hui Chen Soujanya Poria 49 0 0 02 May 2025
See-Saw Modality Balance: See Gradient, and Sew Impaired Vision-Language Balance to Mitigate Dominant Modality Bias Junehyoung Kwon Mihyeon Kim Eunju Lee Juhwan Choi Youngbin Kim 60 0 0 18 Mar 2025
Knowledge Bridger: Towards Training-free Missing Multi-modality Completion Guanzhou Ke Shengfeng He Xinyu Wang Bo Wang Guoqing Chao Yuyao Zhang Yi Xie HeXing Su 68 0 0 27 Feb 2025
Efficient Domain Adaptation of Multimodal Embeddings using Constrastive Learning Georgios Margaritis Periklis Petridis Dimitris Bertsimas 66 0 0 04 Feb 2025
MTPareto: A MultiModal Targeted Pareto Framework for Fake News Detection Kaiying Yan Moyang Liu Yukun Liu Ruibo Fu Zhengqi Wen J. Tao Xuefei Liu Guanjun Li 36 0 0 12 Jan 2025
Deep Correlated Prompting for Visual Recognition with Missing Modalities Lianyu Hu Tongkai Shi Wei Feng Fanhua Shang Liang Wan VLM 31 1 0 09 Oct 2024
What to align in multimodal contrastive learning? Benoit Dufumier J. Castillo-Navarro D. Tuia Jean-Philippe Thiran 29 3 0 11 Sep 2024
Metadata augmented deep neural networks for wild animal classification Aslak Tøn Ammar Ahmed Ali Shariq Imran Mohib Ullah R. Muhammad Atif Azad 42 1 0 07 Sep 2024
Trustworthy Enhanced Multi-view Multi-modal Alzheimer's Disease Prediction with Brain-wide Imaging Transcriptomics Data Shan Cong Zhoujie Fan Hongwei Liu Yinghan Zhang Xin Wang Haoran Luo Xiaohui Yao 30 1 0 21 Jun 2024
Robust Latent Representation Tuning for Image-text Classification Hao Sun Yu Song VLM 60 0 0 10 Jun 2024
Towards Robust Multimodal Prompting With Missing Modalities Jaehyuk Jang Yooseung Wang Changick Kim VLM 30 10 0 26 Dec 2023
Conditional Prompt Tuning for Multimodal Fusion Ruixia Jiang Lingbo Liu Changwen Chen 22 0 0 28 Nov 2023
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 39 2 0 08 Oct 2023
Demystifying Visual Features of Movie Posters for Multi-Label Genre Identification Utsav Nareti Chandranath Adak Soumiki Chattopadhyay 19 0 0 21 Sep 2023
Community-Based Hierarchical Positive-Unlabeled (PU) Model Fusion for Chronic Disease Prediction Yang Wu Xurui Li Xuhong Zhang Yangyang Kang Changlong Sun Xiaozhong Liu 32 3 0 06 Sep 2023
DCTM: Dilated Convolutional Transformer Model for Multimodal Engagement Estimation in Conversation Vu Ngoc Tu V. Huynh Hyung-Jeong Yang M. Zaheer Shah Nawaz Karthik Nandakumar Soo-Hyung Kim 19 4 0 31 Jul 2023
Efficient Multimodal Fusion via Interactive Prompting Yaowei Li Ruijie Quan Linchao Zhu Yezhou Yang 35 44 0 13 Apr 2023
Multimodal Prompting with Missing Modalities for Visual Recognition Yi-Lun Lee Yi-Hsuan Tsai Wei-Chen Chiu Chen-Yu Lee VPVLM 30 94 0 06 Mar 2023
Multimodal Tree Decoder for Table of Contents Extraction in Document Images Pengfei Hu Zhenrong Zhang Jianshu Zhang Jun Du Jiajia Wu 25 12 0 06 Dec 2022
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model Xingqian Xu Zhangyang Wang Eric Zhang Kai Wang Humphrey Shi DiffM 41 183 0 15 Nov 2022
DM $^2$ S $^2$ : Deep Multi-Modal Sequence Sets with Hierarchical Modality Attention Shunsuke Kitada Yuki Iwazaki Riku Togashi Hitoshi Iyatomi 21 1 0 07 Sep 2022
Cross-Modality Gated Attention Fusion for Multimodal Sentiment Analysis Ming-Xin Jiang Shaoxiong Ji 23 3 0 25 Aug 2022
Multimodal Crop Type Classification Fusing Multi-Spectral Satellite Time Series with Farmers Crop Rotations and Local Crop Distribution Valentin Barrière M. Claverie 21 4 0 23 Aug 2022
Learning Branched Fusion and Orthogonal Projection for Face-Voice Association M. S. Saeed Shah Nawaz M. H. Khan S. Javed Muhammad Haroon Yousaf Alessio Del Bue CVBM 27 4 0 22 Aug 2022
Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness Paul Pu Liang 27 4 0 14 Apr 2022
Are Multimodal Transformers Robust to Missing Modality? Mengmeng Ma Jian Ren Long Zhao Davide Testuggine Xi Peng ViT 26 148 0 12 Apr 2022
Dynamic Multimodal Fusion Zihui Xue R. Marculescu 39 48 0 31 Mar 2022
Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding Jiabo Ye Junfeng Tian Ming Yan Xiaoshan Yang Xuwu Wang Ji Zhang Liang He Xin Lin ObjD 11 61 0 29 Mar 2022
Self-supervised Implicit Glyph Attention for Text Recognition Tongkun Guan Chaochen Gu Jingzheng Tu Xuehang Yang Qi Feng Yudi Zhao Xiaokang Yang Wei Shen 32 25 0 07 Mar 2022
Learnable Irrelevant Modality Dropout for Multimodal Action Recognition on Modality-Specific Annotated Videos Saghir Alfasly Jian Lu C. Xu Yuru Zou 42 18 0 06 Mar 2022
A Survey of Vision-Language Pre-Trained Models Yifan Du Zikang Liu Junyi Li Wayne Xin Zhao VLM 33 179 0 18 Feb 2022
Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition Pengfei Liu Kun Li Helen Meng CVBM 25 42 0 17 Jan 2022
FaVoA: Face-Voice Association Favours Ambiguous Speaker Detection Hugo C. C. Carneiro C. Weber S. Wermter CVBM 31 7 0 01 Sep 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 32 159 0 15 Jul 2021
A Survey on Graph-Based Deep Learning for Computational Histopathology David Ahmedt-Aristizabal M. Armin Simon Denman Clinton Fookes L. Petersson GNN AI4CE 19 108 0 01 Jul 2021
AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition Yikang Shen Chun-Fu Chen Quanfu Fan Ximeng Sun Kate Saenko A. Oliva Rogerio Feris 33 47 0 11 May 2021
SMIL: Multimodal Learning with Severely Missing Modality Mengmeng Ma Jian Ren Long Zhao Sergey Tulyakov Cathy H. Wu Xi Peng 49 239 0 09 Mar 2021
Self-Augmented Multi-Modal Feature Embedding Shinnosuke Matsuo S. Uchida Brian Kenji Iwana 19 1 0 08 Mar 2021
A Case Study of Deep Learning Based Multi-Modal Methods for Predicting the Age-Suitability Rating of Movie Trailers Mahsa Shafaei C. Smailis I. Kakadiaris Thamar Solorio 138 1 0 26 Jan 2021
Modality Dropout for Improved Performance-driven Talking Faces Ahmed Hussen Abdelaziz B. Theobald Paul Dixon Reinhard Knothe N. Apostoloff Sachin Kajareker 24 36 0 27 May 2020
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes Douwe Kiela Hamed Firooz Aravind Mohan Vedanuj Goswami Amanpreet Singh Pratik Ringshia Davide Testuggine 37 580 0 10 May 2020
Are we pretraining it right? Digging deeper into visio-linguistic pretraining Amanpreet Singh Vedanuj Goswami Devi Parikh VLM 40 48 0 19 Apr 2020
Multimodal Categorization of Crisis Events in Social Media Mahdi Abavisani Liwei Wu Shengli Hu Joel R. Tetreault A. Jaimes 29 87 0 10 Apr 2020
Towards Accurate Scene Text Recognition with Semantic Reasoning Networks Deli Yu Xuan Li Chengquan Zhang Junyu Han Jingtuo Liu Errui Ding 39 285 0 27 Mar 2020
Deep Multi-Modal Sets A. Reiter Menglin Jia Pu Yang Ser-Nam Lim BDL 25 4 0 03 Mar 2020
Audiovisual SlowFast Networks for Video Recognition Fanyi Xiao Yong Jae Lee Kristen Grauman Jitendra Malik Christoph Feichtenhofer 197 207 0 23 Jan 2020
MMTM: Multimodal Transfer Module for CNN Fusion Hamid Reza Vaezi Joze Amirreza Shaban Michael L. Iuzzolino K. Koishida 18 277 0 20 Nov 2019
Fine-grained Action Segmentation using the Semi-Supervised Action GAN Harshala Gammulle Simon Denman Sridha Sridharan Clinton Fookes GAN 22 36 0 20 Sep 2019
Picture What you Read I. Gallo Shah Nawaz Alessandro Calefati Riccardo La Grassa Nicola Landro DiffM 26 0 0 09 Sep 2019
Supervised Multimodal Bitransformers for Classifying Images and Text Douwe Kiela Suvrat Bhooshan Hamed Firooz Ethan Perez Davide Testuggine 59 241 0 06 Sep 2019