Understanding Self-Attention of Self-Supervised Audio Transformers

5 June 2020

Papers citing "Understanding Self-Attention of Self-Supervised Audio Transformers"

10 / 10 papers shown

Title
Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budget Andy T. Liu Yi-Cheng Lin Haibin Wu Stefan Winkler Hung-yi Lee 31 1 0 09 Sep 2024
Knowledge Distillation from Non-streaming to Streaming ASR Encoder using Auxiliary Non-streaming Layer Kyuhong Shim Jinkyu Lee Simyoung Chang Kyuwoong Hwang 40 2 0 31 Aug 2023
Exploring Attention Map Reuse for Efficient Transformer Neural Networks Kyuhong Shim Jungwook Choi Wonyong Sung ViT 24 3 0 29 Jan 2023
A Comparison of Transformer, Convolutional, and Recurrent Neural Networks on Phoneme Recognition Kyuhong Shim Wonyong Sung 25 2 0 01 Oct 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 134 350 0 21 May 2022
Similarity and Content-based Phonetic Self Attention for Speech Recognition Kyuhong Shim Wonyong Sung 18 7 0 19 Mar 2022
Audio Self-supervised Learning: A Survey Shuo Liu Adria Mallol-Ragolta Emilia Parada-Cabeleiro Kun Qian Xingshuo Jing Alexander Kathan Bin Hu Bjoern W. Schuller SSL 35 106 0 02 Mar 2022
Don't speak too fast: The impact of data bias on self-supervised speech models Yen Meng Yi-Hui Chou Andy T. Liu Hung-yi Lee 34 26 0 15 Oct 2021
CLAR: Contrastive Learning of Auditory Representations Haider Al-Tahan Y. Mohsenzadeh SSL 118 56 0 19 Oct 2020
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech Andy T. Liu Shang-Wen Li Hung-yi Lee SSL 62 356 0 12 Jul 2020