Multimodal Self-Supervised Learning of General Audio Representations

26 April 2021

Luyu Wang

Pauline Luc

Adrià Recasens

Jean-Baptiste Alayrac

Aaron van den Oord

SSL

ArXiv PDF HTML

Papers citing "Multimodal Self-Supervised Learning of General Audio Representations"

32 / 32 papers shown

Title
CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment Edson Araujo Andrew Rouditchenko Yuan Gong Saurabhchand Bhati Samuel Thomas Brian Kingsbury Leonid Karlinsky Rogerio Feris James Glass Hilde Kuehne 81 0 0 02 May 2025
Multi-Format Contrastive Learning of Audio Representations Luyu Wang Aaron van den Oord 57 59 0 11 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 909 29,372 0 26 Feb 2021
LEAF: A Learnable Frontend for Audio Classification Neil Zeghidour O. Teboul Félix de Chaumont Quitry Marco Tagliasacchi VLM AAML 105 147 0 21 Jan 2021
Unsupervised Contrastive Learning of Sound Event Representations Eduardo Fonseca Diego Ortego Kevin McGuinness Noel E. O'Connor Xavier Serra SSL 63 65 0 15 Nov 2020
Contrastive Learning of General-Purpose Audio Representations Aaqib Saeed David Grangier Neil Zeghidour VLM SSL 68 269 0 21 Oct 2020
Self-Supervised MultiModal Versatile Networks Jean-Baptiste Alayrac Adrià Recasens R. Schneider Relja Arandjelović Jason Ramapuram J. Fauw Lucas Smaira Sander Dieleman Andrew Zisserman SSL 121 373 0 29 Jun 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 282 5,790 0 20 Jun 2020
Towards Learning a Universal Non-Semantic Representation of Speech Joel Shor A. Jansen Ronnie Maor Oran Lang Omry Tuval Félix de Chaumont Quitry Marco Tagliasacchi Ira Shavitt Dotan Emanuel Yinnon A. Haviv SSL 126 157 0 25 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 358 18,752 0 13 Feb 2020
Learning Robust and Multilingual Speech Representations Kazuya Kawakami Luyu Wang Chris Dyer Phil Blunsom Aaron van den Oord SSL 71 100 0 29 Jan 2020
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition Qiuqiang Kong Yin Cao Turab Iqbal Yuxuan Wang Wenwu Wang Mark D. Plumbley VLM SSL 184 1,076 0 21 Dec 2019
End-to-End Learning of Visual Representations from Uncurated Instructional Videos Antoine Miech Jean-Baptiste Alayrac Lucas Smaira Ivan Laptev Josef Sivic Andrew Zisserman VGen SSL 114 711 0 13 Dec 2019
Coincidence, Categorization, and Consolidation: Learning to Recognize Sounds with Minimal Supervision A. Jansen D. Ellis Shawn Hershey R. C. Moore Manoj Plakal Ashok Popat Rif A. Saurous SSL 54 26 0 14 Nov 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 199 12,074 0 13 Nov 2019
Learning Representations by Maximizing Mutual Information Across Views Philip Bachman R. Devon Hjelm William Buchwalter SSL 189 1,472 0 03 Jun 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 137 18,115 0 28 May 2019
Data-Efficient Image Recognition with Contrastive Predictive Coding Olivier J. Hénaff A. Srinivas J. Fauw Ali Razavi Carl Doersch S. M. Ali Eslami Aaron van den Oord SSL 122 1,427 0 22 May 2019
TSM: Temporal Shift Module for Efficient Video Understanding Ji Lin Chuang Gan Song Han 98 1,689 0 20 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,770 0 11 Oct 2018
Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge D. Stowell Y. Stylianou Mike Wood H. Pamula H. Glotin 74 310 0 16 Jul 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 309 10,284 0 10 Jul 2018
Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition Pete Warden 76 1,616 0 09 Apr 2018
Unsupervised Learning of Semantic Audio Representations A. Jansen Manoj Plakal R. Pandya D. Ellis Shawn Hershey Jiayang Liu R. C. Moore Rif A. Saurous SSL 82 131 0 06 Nov 2017
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 276 9,760 0 25 Oct 2017
VoxCeleb: a large-scale speaker identification dataset Arsha Nagrani Joon Son Chung Andrew Zisserman 122 2,273 0 26 Jun 2017
Look, Listen and Learn Relja Arandjelović Andrew Zisserman SSL 111 903 0 23 May 2017
Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders Jesse Engel Cinjon Resnick Adam Roberts Sander Dieleman Douglas Eck Karen Simonyan Mohammad Norouzi 109 624 0 05 Apr 2017
CNN Architectures for Large-Scale Audio Classification Shawn Hershey Sourish Chaudhuri D. Ellis J. Gemmeke A. Jansen ... Rif A. Saurous Bryan Seybold M. Slaney Ron J. Weiss K. Wilson 120 2,498 0 29 Sep 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 327 8,116 0 13 Aug 2016
MUSAN: A Music, Speech, and Noise Corpus David Snyder Guoguo Chen Daniel Povey 78 1,347 0 28 Oct 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.8K 150,039 0 22 Dec 2014