Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities

19 December 2018

Hai Pham

Paul Pu Liang

Thomas Manzini

Louis-Philippe Morency

Barnabás Póczós

ArXiv PDF HTML

Papers citing "Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities"

50 / 50 papers shown

Title
TACFN: Transformer-based Adaptive Cross-modal Fusion Network for Multimodal Emotion Recognition Feng Liu Ziwang Fu Yixuan Wang Qijian Zheng 40 4 0 10 May 2025
Knowledge Bridger: Towards Training-free Missing Multi-modality Completion Guanzhou Ke Shengfeng He Xinyu Wang Bo Wang Guoqing Chao Yuyao Zhang Yi Xie HeXing Su 68 0 0 27 Feb 2025
Multi-Modality Collaborative Learning for Sentiment Analysis Shanmin Wang Chengguang Liu Qingshan Liu 42 0 0 21 Jan 2025
Global-Local Distillation Network-Based Audio-Visual Speaker Tracking with Incomplete Modalities Yidi Li Yihan Li Yixin Guo Bin Ren Zhenhuan Xu Hao Guo Hong Liu N. Sebe 47 0 0 26 Aug 2024
Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality Huy Q. Le Chu Myaet Thwal Yu Qiao Ye Lin Tun Minh N. H. Nguyen Choong Seon Hong Choong Seon Hong 65 4 0 25 Jan 2024
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition Zheng Lian Guoying Zhao Yong Ren Hao Gu Haiyang Sun Lan Chen Bin Liu Jianhua Tao 21 12 0 07 Jan 2024
Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach Weide Liu Huijing Zhan Hao Chen Fengmao Lv 18 1 0 28 Dec 2023
Modality-Collaborative Transformer with Hybrid Feature Reconstruction for Robust Emotion Recognition Chengxin Chen Pengyuan Zhang 28 5 0 26 Dec 2023
Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources Suraj Rajendran Weishen Pan M. Sabuncu Yong Chen Jiayu Zhou Fei Wang 54 14 0 10 May 2023
Decoupled Multimodal Distilling for Emotion Recognition Yong Li Yuan-Zheng Wang Zhen Cui 19 73 0 24 Mar 2023
Multimodal Prompting with Missing Modalities for Visual Recognition Yi-Lun Lee Yi-Hsuan Tsai Wei-Chen Chiu Chen-Yu Lee VPVLM 27 94 0 06 Mar 2023
CVTNet: A Cross-View Transformer Network for Place Recognition Using LiDAR Data Junyi Ma Guang-ming Xiong Jingyi Xu Xieyuanli Chen ViT 24 31 0 03 Feb 2023
InterMulti:Multi-view Multimodal Interactions with Text-dominated Hierarchical High-order Fusion for Emotion Analysis Feng Qiu Wanzeng Kong Yu-qiong Ding 34 2 0 20 Dec 2022
A Self-Adjusting Fusion Representation Learning Model for Unaligned Text-Audio Sequences Kaicheng Yang Ruxuan Zhang Hua Xu Kai Gao 23 3 0 12 Nov 2022
Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations Sijie Mai Ying Zeng Haifeng Hu 37 67 0 31 Oct 2022
A Multimodal Sensor Fusion Framework Robust to Missing Modalities for Person Recognition Vijay John Yasutomo Kawanishi 29 7 0 20 Oct 2022
Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis Teng Sun Wenjie Wang Liqiang Jing Yiran Cui Xuemeng Song Liqiang Nie OODD 24 35 0 24 Jul 2022
Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review Hao Wang Bin Guo Y. Zeng Yasan Ding Chen Qiu Ying Zhang Li Yao Zhiwen Yu 30 2 0 02 Jul 2022
COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition M. Tellamekala Shahin Amiriparian Björn W. Schuller Elisabeth André T. Giesbrecht M. Valstar 26 25 0 12 Jun 2022
Are Multimodal Transformers Robust to Missing Modality? Mengmeng Ma Jian Ren Long Zhao Davide Testuggine Xi Peng ViT 26 148 0 12 Apr 2022
GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation Zheng Lian Lang Chen Guoying Zhao B. Liu J. Tao 25 83 0 04 Mar 2022
MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis Georgios Paraskevopoulos Efthymios Georgiou Alexandros Potamianos 19 26 0 24 Jan 2022
Tailor Versatile Multi-modal Learning for Multi-label Emotion Recognition Yi Zhang Mingyuan Chen Jundong Shen Chongjun Wang 21 59 0 15 Jan 2022
LMR-CBT: Learning Modality-fused Representations with CB-Transformer for Multimodal Emotion Recognition from Unaligned Multimodal Sequences Ziwang Fu Feng Liu Hanyang Wang Siyuan Shen Jiahao Zhang Jiayin Qi Xiangling Fu Aimin Zhou 34 9 0 03 Dec 2021
Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions Huan Ma Zongbo Han Changqing Zhang Huazhu Fu Qiufeng Wang Q. Hu EDL UQCV 71 42 0 11 Nov 2021
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis Ying Zeng Sijie Mai Haifeng Hu 21 19 0 10 Nov 2021
Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism Xiaoqiang Zhang Ying Chen Guang-ying Li 24 11 0 30 Sep 2021
TEASEL: A Transformer-Based Speech-Prefixed Language Model Mehdi Arjmand M. Dousti H. Moradi 33 18 0 12 Sep 2021
Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis Sijie Mai Ying Zeng Shuangjia Zheng Haifeng Hu 21 116 0 04 Sep 2021
Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis Wei Han Hui Chen Soujanya Poria 34 315 0 01 Sep 2021
Improving Multimodal fusion via Mutual Dependency Maximisation Pierre Colombo E. Chapuis Matthieu Labeau Chloé Clavel 13 30 0 31 Aug 2021
Graph Capsule Aggregation for Unaligned Multimodal Sequences Jianfeng Wu Sijie Mai Haifeng Hu 21 32 0 17 Aug 2021
Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation Zaid Khan Y. Fu 27 131 0 03 Aug 2021
Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis Wei Han Hui Chen Alexander Gelbukh Amir Zadeh Louis-Philippe Morency Soujanya Poria 19 175 0 28 Jul 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 32 159 0 15 Jul 2021
LiRA: Learning Visual Speech Representations from Audio through Self-supervision Pingchuan Ma Rodrigo Mira Stavros Petridis Björn W. Schuller M. Pantic SSL 21 53 0 16 Jun 2021
Generalized Multimodal ELBO Thomas M. Sutter Imant Daunhawer Julia E. Vogt 13 89 0 06 May 2021
SMIL: Multimodal Learning with Severely Missing Modality Mengmeng Ma Jian Ren Long Zhao Sergey Tulyakov Cathy H. Wu Xi Peng 40 239 0 09 Mar 2021
Video Sentiment Analysis with Bimodal Information-augmented Multi-Head Attention Ting-Wei Wu Jun-jie Peng Wenqiang Zhang Huiran Zhang Chuan Ma Yansong Huang 27 84 0 03 Mar 2021
Deep Learning and the Global Workspace Theory R. V. Rullen Ryota Kanai 45 65 0 04 Dec 2020
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors Michelle A. Lee Matthew Tan Yuke Zhu Jeannette Bohg 43 25 0 01 Dec 2020
Hierachical Delta-Attention Method for Multimodal Fusion Kunjal Panchal 11 1 0 22 Nov 2020
MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences Jianing Yang Yongxin Wang Ruitao Yi Yuying Zhu Azaan Rehman Amir Zadeh Soujanya Poria Louis-Philippe Morency 34 11 0 22 Oct 2020
Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognition Jean-Benoit Delbrouck Noé Tits Stéphane Dupont 30 20 0 05 Oct 2020
Learning Speech Representations from Raw Audio by Joint Audiovisual Self-Supervision Abhinav Shukla Stavros Petridis M. Pantic SSL 32 16 0 08 Jul 2020
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis Devamanyu Hazarika Roger Zimmermann Soujanya Poria 21 669 0 07 May 2020
Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding Seonguk Park Gyubok Lee Manoj Bhat Jimin Seo Minseok Kang Jonathan M Francis Ashwin R. Jadhav Paul Pu Liang Louis-Philippe Morency 136 119 0 06 Mar 2020
Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion Sijie Mai Haifeng Hu Songlong Xing GAN 32 182 0 18 Nov 2019
Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization Paul Pu Liang Zhun Liu Yao-Hung Hubert Tsai Qibin Zhao Ruslan Salakhutdinov Louis-Philippe Morency AI4TS 30 81 0 01 Jul 2019
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor M. Hasan Wasifur Rahman Amir Zadeh Jianyuan Zhong Md. Iftekhar Tanveer Louis-Philippe Morency Mohammed Ehsan Hoque 16 179 0 14 Apr 2019