From Multimodal to Unimodal Attention in Transformers using Knowledge
Distillation

v1v2 (latest)

From Multimodal to Unimodal Attention in Transformers using Knowledge Distillation

15 October 2021

Laura M. Ferrari

Franccois Bremond

ArXiv (abs)PDF HTML

Papers citing "From Multimodal to Unimodal Attention in Transformers using Knowledge Distillation"

16 / 16 papers shown

Title
Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection Rui Dai Srijan Das Francois Bremond 74 40 0 08 Aug 2021
Multimodal End-to-End Sparse Model for Emotion Recognition Wenliang Dai Samuel Cahyawijaya Zihan Liu Pascale Fung CVBM 64 82 0 17 Mar 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 307 2,532 0 04 Jan 2021
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research Soujanya Poria Devamanyu Hazarika Navonil Majumder Rada Mihalcea 93 221 0 01 May 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 390 18,897 0 13 Feb 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 207 2,333 0 13 Jan 2020
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 168 1,054 0 23 Oct 2019
Multimodal Transformer for Unaligned Multimodal Language Sequences Yao-Hung Hubert Tsai Shaojie Bai Paul Pu Liang J. Zico Kolter Louis-Philippe Morency Ruslan Salakhutdinov 97 1,308 0 01 Jun 2019
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors Yansen Wang Ying Shen Zhun Liu Paul Pu Liang Amir Zadeh Louis-Philippe Morency 80 403 0 23 Nov 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 351 10,364 0 10 Jul 2018
Tensor Fusion Network for Multimodal Sentiment Analysis Amir Zadeh Minghai Chen Soujanya Poria Min Zhang Louis-Philippe Morency 87 1,236 0 23 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 803 132,454 0 12 Jun 2017
Multimodal Machine Learning: A Survey and Taxonomy T. Baltrušaitis Chaitanya Ahuja Louis-Philippe Morency 119 2,939 0 26 May 2017
Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysis Haohan Wang Aaksha Meghawat Louis-Philippe Morency Eric Xing 79 155 0 16 Sep 2016
FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff Dmitry Kalenichenko James Philbin 3DH 389 13,172 0 12 Mar 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 367 19,745 0 09 Mar 2015

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.