Maestro: Uncovering Low-Rank Structures via Trainable Decomposition

28 August 2023

Papers citing "Maestro: Uncovering Low-Rank Structures via Trainable Decomposition"

50 / 55 papers shown

Title
Forget the Data and Fine-Tuning! Just Fold the Network to Compress Dong Wang Haris Šikić Lothar Thiele O. Saukh 115 1 0 17 Feb 2025
MELTing point: Mobile Evaluation of Language Transformers Stefanos Laskaridis Kleomenis Katevas Lorenzo Minto Hamed Haddadi 57 23 0 19 Mar 2024
Efficient Large Language Models: A Survey Zhongwei Wan Xin Wang Che Liu Samiul Alam Yu Zheng ... Shen Yan Yi Zhu Quanlu Zhang Mosharaf Chowdhury Mi Zhang LM&MA 37 130 0 06 Dec 2023
Cuttlefish: Low-Rank Model Training without All the Tuning Hongyi Wang Saurabh Agarwal Pongsakorn U-chupala Yoshiki Tanaka Eric P. Xing Dimitris Papailiopoulos OffRL 102 23 0 04 May 2023
Federated Learning for Inference at Anytime and Anywhere Zicheng Liu Da Li Javier Fernandez-Marques Stefanos Laskaridis Yan Gao Łukasz Dudziak Stan Z. Li S. Hu Timothy M. Hospedales FedML 74 5 0 08 Dec 2022
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 183 3,655 0 06 Dec 2022
FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction Samiul Alam Luyang Liu Ming Yan Mi Zhang 157 150 0 03 Dec 2022
The Future of Consumer Edge-AI Computing Stefanos Laskaridis Stylianos I. Venieris Alexandros Kouris Rui Li Nicholas D. Lane 85 10 0 19 Oct 2022
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? Mansheej Paul F. Chen Brett W. Larsen Jonathan Frankle Surya Ganguli Gintare Karolina Dziugaite UQCV 96 38 0 06 Oct 2022
On-Device Training Under 256KB Memory Ji Lin Ligeng Zhu Wei-Ming Chen Wei-Chen Wang Chuang Gan Song Han MQ 85 206 0 30 Jun 2022
The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink David A. Patterson Joseph E. Gonzalez Urs Holzle Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 76 247 0 11 Apr 2022
Rare Gems: Finding Lottery Tickets at Initialization Kartik K. Sreenivasan Jy-yong Sohn Liu Yang Matthew Grinde Alliot Nagle Hongyi Wang Eric P. Xing Kangwook Lee Dimitris Papailiopoulos 48 42 0 24 Feb 2022
Sustainable AI: Environmental Implications, Challenges and Opportunities Carole-Jean Wu Ramya Raghavendra Udit Gupta Bilge Acun Newsha Ardalani ... Maximilian Balandat Joe Spisak R. Jain Michael G. Rabbat K. Hazelwood 104 405 0 30 Oct 2021
Efficient and Private Federated Learning with Partially Trainable Networks Hakim Sidahmed Zheng Xu Ankush Garg Yuan Cao Mingqing Chen FedML 99 13 0 06 Oct 2021
Smart at what cost? Characterising Mobile Deep Neural Networks in the wild Mario Almeida Stefanos Laskaridis Abhinav Mehrotra Łukasz Dudziak Ilias Leontiadis Nicholas D. Lane HAI 141 46 0 28 Sep 2021
Only Train Once: A One-Shot Neural Network Training And Pruning Framework Tianyi Chen Bo Ji Tianyu Ding Biyi Fang Guanyi Wang Zhihui Zhu Luming Liang Yixin Shi Sheng Yi Xiao Tu 85 105 0 15 Jul 2021
Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization Tian-Chun Ye S. Du 49 48 0 27 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 433 10,328 0 17 Jun 2021
Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions Stefanos Laskaridis Alexandros Kouris Nicholas D. Lane TPM 99 117 0 09 Jun 2021
Initialization and Regularization of Factorized Neural Layers M. Khodak Neil A. Tenenholtz Lester W. Mackey Nicolò Fusi 117 57 0 03 May 2021
Pufferfish: Communication-efficient Models At No Extra Cost Hongyi Wang Saurabh Agarwal Dimitris Papailiopoulos 60 59 0 05 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 923 29,372 0 26 Feb 2021
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout Samuel Horváth Stefanos Laskaridis Mario Almeida Ilias Leondiadis Stylianos I. Venieris Nicholas D. Lane 216 273 0 26 Feb 2021
Hyperparameter Transfer Learning with Adaptive Complexity Samuel Horváth Aaron Klein Peter Richtárik Cédric Archambeau 48 18 0 25 Feb 2021
HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients Enmao Diao Jie Ding Vahid Tarokh FedML 96 556 0 03 Oct 2020
ShrinkML: End-to-End ASR Model Compression Using Reinforcement Learning Łukasz Dudziak Mohamed S. Abdelfattah Ravichander Vipperla Stefanos Laskaridis Nicholas D. Lane OffRL 74 18 0 08 Jul 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 139 18,115 0 28 May 2019
Universally Slimmable Networks and Improved Training Techniques Jiahui Yu Thomas Huang 74 385 0 12 Mar 2019
Parameter-Efficient Transfer Learning for NLP N. Houlsby A. Giurgiu Stanislaw Jastrzebski Bruna Morrone Quentin de Laroussilhe Andrea Gesmundo Mona Attariyan Sylvain Gelly 210 4,451 0 02 Feb 2019
Slimmable Neural Networks Jiahui Yu L. Yang N. Xu Jianchao Yang Thomas Huang 78 553 0 21 Dec 2018
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements S. Caldas Jakub Konecný H. B. McMahan Ameet Talwalkar 64 450 0 18 Dec 2018
Rethinking the Value of Network Pruning Zhuang Liu Mingjie Sun Tinghui Zhou Gao Huang Trevor Darrell 36 1,471 0 11 Oct 2018
The Convergence of Sparsified Gradient Methods Dan Alistarh Torsten Hoefler M. Johansson Sarit Khirirat Nikola Konstantinov Cédric Renggli 165 493 0 27 Sep 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 230 3,463 0 09 Mar 2018
The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning Siyuan Ma Raef Bassily M. Belkin 76 288 0 18 Dec 2017
BlockDrop: Dynamic Inference Paths in Residual Networks Zuxuan Wu Tushar Nagarajan Abhishek Kumar Steven J. Rennie L. Davis Kristen Grauman Rogerio Feris 89 466 0 22 Nov 2017
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 194 1,276 0 05 Oct 2017
Channel Pruning for Accelerating Very Deep Neural Networks Yihui He Xiangyu Zhang Jian Sun 201 2,524 0 19 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 698 131,526 0 12 Jun 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.1K 20,837 0 17 Apr 2017
Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning Tien-Ju Yang Yu-hsin Chen Vivienne Sze 3DV 86 740 0 16 Nov 2016
Distributed Mean Estimation with Limited Communication A. Suresh Felix X. Yu Sanjiv Kumar H. B. McMahan FedML 101 364 0 02 Nov 2016
Pruning Filters for Efficient ConvNets Hao Li Asim Kadav Igor Durdanovic H. Samet H. Graf 3DPC 193 3,696 0 31 Aug 2016
Learning Structured Sparsity in Deep Neural Networks W. Wen Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 176 2,337 0 12 Aug 2016
Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures Hengyuan Hu Rui Peng Yu-Wing Tai Chi-Keung Tang 66 891 0 12 Jul 2016
Multi30K: Multilingual English-German Image Descriptions Desmond Elliott Stella Frank K. Simaán Lucia Specia VLM 128 588 0 02 May 2016
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks Mohammad Rastegari Vicente Ordonez Joseph Redmon Ali Farhadi MQ 167 4,356 0 16 Mar 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 401 17,468 0 17 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 193,878 0 10 Dec 2015
All you need is a good init Dmytro Mishkin Jirí Matas ODL 86 610 0 19 Nov 2015