Attention Bottlenecks for Multimodal Fusion

30 June 2021

Papers citing "Attention Bottlenecks for Multimodal Fusion"

35 / 285 papers shown

Title
MultiMAE: Multi-modal Multi-task Masked Autoencoders Roman Bachmann David Mizrahi Andrei Atanov Amir Zamir 49 265 0 04 Apr 2022
Learning Audio-Video Modalities from Image Captions Arsha Nagrani Paul Hongsuck Seo Bryan Seybold Anja Hauth Santiago Manén Chen Sun Cordelia Schmid CLIP 18 83 0 01 Apr 2022
Dynamic Multimodal Fusion Zihui Xue R. Marculescu 43 48 0 31 Mar 2022
ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrieval Mengjun Cheng Yipeng Sun Long Wang Xiongwei Zhu Kun Yao ... Guoli Song Junyu Han Jingtuo Liu Errui Ding Jingdong Wang 36 60 0 31 Mar 2022
ReSTR: Convolution-free Referring Image Segmentation Using Transformers N. Kim Dongwon Kim Cuiling Lan Wenjun Zeng Suha Kwak 30 137 0 31 Mar 2022
Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding Jiabo Ye Junfeng Tian Ming Yan Xiaoshan Yang Xuwu Wang Ji Zhang Liang He Xin Lin ObjD 13 61 0 29 Mar 2022
A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition R Gnana Praveen W. Melo Nasib Ullah Haseeb Aslam Osama Zeeshan ... M. Pedersoli Alessandro Lameiras Koerich Simon L Bacon P. Cardinal Eric Granger 25 68 0 28 Mar 2022
Single-Stream Multi-Level Alignment for Vision-Language Pretraining Zaid Khan B. Vijaykumar Xiang Yu S. Schulter Manmohan Chandraker Y. Fu CLIP VLM 25 16 0 27 Mar 2022
Audio-Adaptive Activity Recognition Across Video Domains Yun C. Zhang Hazel Doughty Ling Shao Cees G. M. Snoek 17 38 0 27 Mar 2022
UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection Ye Liu Siyuan Li Yang Wu C. Chen Ying Shan Xiaohu Qie ViT 27 140 0 23 Mar 2022
Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs Jingfei Xia Mingchen Zhuge Tiantian Geng Shun Fan Yuantai Wei Zhenyu He Feng Zheng 23 14 0 08 Mar 2022
Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Language Otniel-Bogdan Mercea Lukas Riesch A. Sophia Koepke Zeynep Akata 33 48 0 07 Mar 2022
Learnable Irrelevant Modality Dropout for Multimodal Action Recognition on Modality-Specific Annotated Videos Saghir Alfasly Jian Lu C. Xu Yuru Zou 42 18 0 06 Mar 2022
HiP: Hierarchical Perceiver João Carreira Skanda Koppula Daniel Zoran Adrià Recasens Catalin Ionescu ... M. Botvinick Oriol Vinyals Karen Simonyan Andrew Zisserman Andrew Jaegle VLM 31 14 0 22 Feb 2022
OWL (Observe, Watch, Listen): Audiovisual Temporal Context for Localizing Actions in Egocentric Videos Merey Ramazanova Victor Escorcia Fabian Caba Heilbron Chen Zhao Guohao Li 28 3 0 10 Feb 2022
Omnivore: A Single Model for Many Visual Modalities Rohit Girdhar Mannat Singh Nikhil Ravi L. V. D. van der Maaten Armand Joulin Ishan Misra 229 226 0 20 Jan 2022
Pretrained Language Models for Text Generation: A Survey Junyi Li Tianyi Tang Wayne Xin Zhao J. Nie Ji-Rong Wen AI4CE 36 128 0 14 Jan 2022
Multiview Transformers for Video Recognition Shen Yan Xuehan Xiong Anurag Arnab Zhichao Lu Mi Zhang Chen Sun Cordelia Schmid ViT 26 212 0 12 Jan 2022
Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer Yanpeng Zhao Jack Hessel Youngjae Yu Ximing Lu Rowan Zellers Yejin Choi 22 27 0 16 Dec 2021
Learning Generalizable Vision-Tactile Robotic Grasping Strategy for Deformable Objects via Transformer Yunhai Han Kelin Yu Rahul Batra Nathan Boyd Chaitanya Mehta T. Zhao Y. She S. Hutchinson Ye Zhao ViT 29 43 0 13 Dec 2021
Everything at Once -- Multi-modal Fusion Transformer for Video Retrieval Nina Shvetsova Brian Chen Andrew Rouditchenko Samuel Thomas Brian Kingsbury Rogerio Feris David Harwath James R. Glass Hilde Kuehne ViT 34 128 0 08 Dec 2021
Audio-Visual Synchronisation in the wild Honglie Chen Weidi Xie Triantafyllos Afouras Arsha Nagrani Andrea Vedaldi Andrew Zisserman 26 37 0 08 Dec 2021
MutualFormer: Multi-Modality Representation Learning via Cross-Diffusion Attention Xixi Wang Tianlin Li Bo Jiang Jin Tang Bin Luo ViT 32 7 0 02 Dec 2021
SWAT: Spatial Structure Within and Among Tokens Kumara Kahatapitiya Michael S. Ryoo 25 6 0 26 Nov 2021
PolyViT: Co-training Vision Transformers on Images, Videos and Audio Valerii Likhosherstov Anurag Arnab K. Choromanski Mario Lucic Yi Tay Adrian Weller Mostafa Dehghani ViT 35 73 0 25 Nov 2021
Sparse Fusion for Multimodal Transformers Yi Ding Alex Rich Mason Wang Noah Stier M. Turk P. Sen Tobias Höllerer ViT 27 7 0 23 Nov 2021
Many Heads but One Brain: Fusion Brain -- a Competition and a Single Multimodal Multitask Architecture Daria Bakshandaeva Denis Dimitrov V.Ya. Arkhipkin Alex Shonenkov M. Potanin ... Mikhail Martynov Anton Voronov Vera Davydova E. Tutubalina Aleksandr Petiushko 37 0 0 22 Nov 2021
Masking Modalities for Cross-modal Video Retrieval Valentin Gabeur Arsha Nagrani Chen Sun Alahari Karteek Cordelia Schmid 19 29 0 01 Nov 2021
With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition Evangelos Kazakos Jaesung Huh Arsha Nagrani Andrew Zisserman Dima Damen EgoV 50 45 0 01 Nov 2021
SCENIC: A JAX Library for Computer Vision Research and Beyond Mostafa Dehghani A. Gritsenko Anurag Arnab Matthias Minderer Yi Tay 49 68 0 18 Oct 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 251 577 0 22 Apr 2021
Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers Lisa Anne Hendricks John F. J. Mellor R. Schneider Jean-Baptiste Alayrac Aida Nematzadeh 79 110 0 31 Jan 2021
Multi-modal Transformer for Video Retrieval Valentin Gabeur Chen Sun Alahari Karteek Cordelia Schmid ViT 430 596 0 21 Jul 2020
Neural Entity Linking: A Survey of Models Based on Deep Learning Ozge Sevgili Artem Shelmanov Mikhail V. Arkhipov Alexander Panchenko Christian Biemann VLM 3DV AI4TS 23 118 0 31 May 2020
Audiovisual SlowFast Networks for Video Recognition Fanyi Xiao Yong Jae Lee Kristen Grauman Jitendra Malik Christoph Feichtenhofer 197 207 0 23 Jan 2020