Deep Audio-Visual Learning: A Survey

14 January 2020

Papers citing "Deep Audio-Visual Learning: A Survey"

50 / 97 papers shown

Title
Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning Sangyeon Cho Jangyeong Jeon Mingi Kim Junyeong Kim CLIP VLM 124 0 0 30 Apr 2025
MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AI Arya Tschand Arun Tejusve Raghunath Rajan S. Idgunji Anirban Ghosh J. Holleman ... Rowan Taubitz Sean Zhan Scott Wasson David Kanter Vijay Janapa Reddi 78 3 0 15 Oct 2024
Progressive Confident Masking Attention Network for Audio-Visual Segmentation Yuxuan Wang Feng Dong Jinchao Zhu Shuyue Zhu VOS 79 0 0 04 Jun 2024
Unveiling the frontiers of deep learning: innovations shaping diverse domains Shams Forruque Ahmed Md. Sakib Bin Alam Maliha Kabir Shaila Afrin Sabiha Jannat Rafa Aanushka Mehjabin Amir H. Gandomi AI4CE 51 2 0 06 Sep 2023
Lightweight Vision Transformer with Bidirectional Interaction Qihang Fan Huaibo Huang Xiaoqiang Zhou Ran He ViT 69 28 0 01 Jun 2023
Multimodal Deep Learning Cem Akkus Jiquan Ngiam Vladana Djakovic Steffen Jauch-Walser A. Khosla ... Jann Goschenhofer Honglak Lee A. Ng Daniel Schalk Matthias Aßenmacher 33 3,161 0 12 Jan 2023
Speech Driven Talking Face Generation from a Single Image and an Emotion Condition Sefik Emre Eskimez You Zhang Z. Duan EGVM CVBM 30 87 0 08 Aug 2020
Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation Hang Zhou Xudong Xu Dahua Lin Xiaogang Wang Ziwei Liu DiffM 49 81 0 20 Jul 2020
Leveraging Category Information for Single-Frame Visual Sound Source Separation Lingyu Zhu Esa Rahtu 33 9 0 15 Jul 2020
Direct Speech-to-image Translation Jiguo Li Xinfeng Zhang Chuanmin Jia Jizheng Xu Li Zhang Y. Wang Siwei Ma Wen Gao 43 29 0 07 Apr 2020
The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification Dongliang Chang Yifeng Ding Jiyang Xie A. Bhunia Xiaoxu Li Zhanyu Ma Ming Wu Jun Guo Yi-Zhe Song 49 305 0 11 Feb 2020
Recurrent Neural Network Transducer for Audio-Visual Speech Recognition Takaki Makino H. Liao Yannis Assael Brendan Shillingford Basi García Otavio Braga Olivier Siohan 34 129 0 08 Nov 2019
Sound Localization and Separation in Three-dimensional Space Using a Single Microphone with a Metamaterial Enclosure Xuecong Sun Han Jia Zhe Zhang Yuzhen Yang Zhaoyong Sun Jun Yang 14 3 0 22 Aug 2019
A Short Note on the Kinetics-700 Human Action Dataset João Carreira Eric Noland Chloe Hillier Andrew Zisserman 33 446 0 15 Jul 2019
Speech2Face: Learning the Face Behind a Voice Tae-Hyun Oh Tali Dekel Changil Kim Inbar Mosseri William T. Freeman Michael Rubinstein Wojciech Matusik SSL CVBM 89 163 0 23 May 2019
Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss Lele Chen R. Maddox Z. Duan Chenliang Xu CVBM 53 395 0 09 May 2019
Self-Supervised Audio-Visual Co-Segmentation Andrew Rouditchenko Hang Zhao Chuang Gan Josh H. McDermott Antonio Torralba VLM SSL 26 103 0 18 Apr 2019
The Sound of Motions Hang Zhao Chuang Gan Wei-Chiu Ma Antonio Torralba 40 252 0 11 Apr 2019
Dual Variational Generation for Low-Shot Heterogeneous Face Recognition Chaoyou Fu Xiang Wu Yibo Hu Huaibo Huang Ran He CVBM 55 73 0 25 Mar 2019
Wav2Pix: Speech-conditioned Face Generation using Generative Adversarial Networks A. Duarte Francisco Roldan Miquel Tubau Janna Escur Santiago Pascual Amaia Salvador Eva Mohedano Kevin McGuinness Jordi Torres Xavier Giró-i-Nieto GAN CVBM 39 79 0 25 Mar 2019
Speech Recognition with no speech or with noisy speech G. Krishna Co Tran Jianguo Yu Ahmed H. Tewfik 13 40 0 02 Mar 2019
Deep Fusion: An Attention Guided Factorized Bilinear Pooling for Audio-video Emotion Recognition Yuanyuan Zhang Zirui Wang Jun Du 21 31 0 15 Jan 2019
SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild Jean Kossaifi R. Walecki Yannis Panagakis Jie Shen Maximilian Schmitt ... Antoine Toisoul Bjorn Schuller Kam Star Elnar Hajiyev Maja Pantic 42 198 0 09 Jan 2019
AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detection Joseph Roth Sourish Chaudhuri Ondˇrej Klejch Radhika Marvin Andrew C. Gallagher ... S. Ramaswamy Arkadiusz Stopczynski Cordelia Schmid Zhonghua Xi C. Pantofaru 30 144 0 05 Jan 2019
Arbitrary Talking Face Generation via Attentional Audio-Visual Coherence Learning Hao Zhu Huaibo Huang Yi Li Aihua Zheng Ran He CVBM 19 58 0 17 Dec 2018
A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras S. Laine Timo Aila 468 10,466 0 12 Dec 2018
Modality Attention for End-to-End Audio-visual Speech Recognition Pan Zhou Wenwen Yang Wei Chen Yanfeng Wang Jia Jia 33 69 0 13 Nov 2018
Identify, locate and separate: Audio-visual object extraction in large video collections using weak supervision Sanjeel Parekh A. Ozerov S. Essid Ngoc Q. K. Duong P. Pérez G. Richard 35 16 0 09 Nov 2018
Face Landmark-based Speaker-Independent Audio-Visual Speech Enhancement in Multi-Talker Environments Yufei Wang Luca Pasa Lantao Yu Rohit Singh Luciano Fadiga L. Joppa CVBM 28 60 0 06 Nov 2018
Perfect match: Improved cross-modal embeddings for audio-visual synchronisation Soo-Whan Chung Joon Son Chung Hong-Goo Kang 20 117 0 21 Sep 2018
Self-Supervised Generation of Spatial Audio for 360 Video Pedro Morgado Nuno Vasconcelos Timothy R. Langlois Oliver Wang MDE 33 171 0 07 Sep 2018
Deep Audio-Visual Speech Recognition Triantafyllos Afouras Joon Son Chung A. Senior Oriol Vinyals Andrew Zisserman 38 696 0 06 Sep 2018
Towards Audio to Scene Image Synthesis using Generative Adversarial Network Chia-Hung Wan Shun-Po Chuang Hung-yi Lee GAN 40 61 0 13 Aug 2018
A Short Note about Kinetics-600 João Carreira Eric Noland Andras Banki-Horvath Chloe Hillier Andrew Zisserman 50 520 0 03 Aug 2018
X2Face: A network for controlling face generation by using images, audio, and pose codes Olivia Wiles A. Sophia Koepke Andrew Zisserman CVBM 50 414 0 27 Jul 2018
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation Hang Zhou Yu Liu Ziwei Liu Ping Luo Xiaogang Wang CVBM 47 438 0 20 Jul 2018
Disjoint Mapping Network for Cross-modal Matching of Voices and Faces Yandong Wen Mahmoud Al Ismail Weiyang Liu Bhiksha Raj Rita Singh FedML 24 71 0 12 Jul 2018
Deep Multimodal Clustering for Unsupervised Audiovisual Learning Di Hu Feiping Nie Xuelong Li SSL 11 8 0 09 Jul 2018
Weakly Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation Nelson Yalta Shinji Watanabe K. Nakadai T. Ogata 29 62 0 03 Jul 2018
Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization Bruno Korbar Du Tran Lorenzo Torresani 62 473 0 30 Jun 2018
VoxCeleb2: Deep Speaker Recognition Joon Son Chung Arsha Nagrani Andrew Zisserman 334 2,261 0 14 Jun 2018
End-to-End Speech-Driven Facial Animation with Temporal GANs Konstantinos Vougioukas Stavros Petridis Maja Pantic CVBM 44 103 0 23 May 2018
Learnable PINs: Cross-Modal Embeddings for Person Identity Arsha Nagrani Samuel Albanie Andrew Zisserman SSL 92 141 0 02 May 2018
Weakly Supervised Representation Learning for Unsynchronized Audio-Visual Events Sanjeel Parekh S. Essid A. Ozerov Ngoc Q. K. Duong P. Pérez G. Richard SSL 33 19 0 19 Apr 2018
The Conversation: Deep Audio-Visual Speech Enhancement Triantafyllos Afouras Joon Son Chung Andrew Zisserman 50 360 0 11 Apr 2018
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features Andrew Owens Alexei A. Efros SSL 63 747 0 10 Apr 2018
The Sound of Pixels Hang Zhao Chuang Gan Andrew Rouditchenko Carl Vondrick Josh H. McDermott Antonio Torralba VLM 42 532 0 09 Apr 2018
Learning to Separate Object Sounds by Watching Unlabeled Video Ruohan Gao Rogerio Feris Kristen Grauman SSL 33 284 0 05 Apr 2018
Seeing Voices and Hearing Faces: Cross-modal biometric matching Arsha Nagrani Samuel Albanie Andrew Zisserman CVBM 39 220 0 01 Apr 2018
Lip Movements Generation at a Glance Lele Chen Zhiheng Li R. Maddox Z. Duan Chenliang Xu 80 263 0 28 Mar 2018