Full-Rank No More: Low-Rank Weight Training for Modern Speech
Recognition Models

Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models

10 October 2024

Adriana Fernandez-Lopez

Stavros Petridis

Maja Pantic

Papers citing "Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models"

15 / 15 papers shown

Title
Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients Zhenyu Zhang Ajay Jaiswal L. Yin Shiwei Liu Jiawei Zhao Yuandong Tian Zhangyang Wang VLM 56 21 0 11 Jul 2024
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection Jiawei Zhao Zhenyu Zhang Beidi Chen Zhangyang Wang A. Anandkumar Yuandong Tian 81 209 0 06 Mar 2024
ReLoRA: High-Rank Training Through Low-Rank Updates Vladislav Lialin Namrata Shivagunde Sherin Muckatira Anna Rumshisky BDL 76 110 0 11 Jul 2023
Cuttlefish: Low-Rank Model Training without All the Tuning Hongyi Wang Saurabh Agarwal Pongsakorn U-chupala Yoshiki Tanaka Eric P. Xing Dimitris Papailiopoulos OffRL 92 23 0 04 May 2023
Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels Pingchuan Ma A. Haliassos Adriana Fernandez-Lopez Honglie Chen Stavros Petridis Maja Pantic 54 114 0 25 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.2K 14,179 0 15 Mar 2023
On the Initialisation of Wide Low-Rank Feedforward Neural Networks Thiziri Nait Saada Jared Tanner 28 1 0 31 Jan 2023
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 371 10,273 0 17 Jun 2021
End-to-end Audio-visual Speech Recognition with Conformers Pingchuan Ma Stavros Petridis Maja Pantic 110 231 0 12 Feb 2021
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 212 3,119 0 16 May 2020
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.6K 94,511 0 11 Oct 2018
LRS3-TED: a large-scale dataset for visual speech recognition Triantafyllos Afouras Joon Son Chung Andrew Zisserman 62 441 0 03 Sep 2018
Personalized Speech recognition on mobile devices Ian McGraw Rohit Prabhavalkar R. Álvarez Montse Gonzalez Arenas Kanishka Rao ... O. Alsharif Hasim Sak A. Gruenstein F. Beaufays Carolina Parada 67 183 0 10 Mar 2016
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 243 8,821 0 01 Oct 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 292 18,587 0 06 Feb 2015