ACDC: A Structured Efficient Linear Layer

18 November 2015

Papers citing "ACDC: A Structured Efficient Linear Layer"

32 / 32 papers shown

Title
Block Circulant Adapter for Large Language Models Xinyu Ding Meiqi Wang Siyu Liao Zhongfeng Wang 38 0 0 01 May 2025
Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling Mahdi Karami Ali Ghodsi VLM 48 6 0 28 Feb 2024
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 32 87 0 01 Apr 2022
Is the Number of Trainable Parameters All That Actually Matters? A. Chatelain Amine Djeghri Daniel Hesslow Julien Launay Iacopo Poli 51 7 0 24 Sep 2021
FNet: Mixing Tokens with Fourier Transforms James Lee-Thorp Joshua Ainslie Ilya Eckstein Santiago Ontanon 47 518 0 09 May 2021
Initialization and Regularization of Factorized Neural Layers M. Khodak Neil A. Tenenholtz Lester W. Mackey Nicolò Fusi 65 56 0 03 May 2021
Rethinking Neural Operations for Diverse Tasks Nicholas Roberts M. Khodak Tri Dao Liam Li Christopher Ré Ameet Talwalkar AI4CE 38 22 0 29 Mar 2021
GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning Juhyoung Lee Sangyeob Kim Sangjin Kim Wooyoung Jo H. Yoo OffRL 21 9 0 24 Jan 2021
Sparse Linear Networks with a Fixed Butterfly Structure: Theory and Practice Nir Ailon Omer Leibovitch Vineet Nair 15 14 0 17 Jul 2020
Reservoir Computing meets Recurrent Kernels and Structured Transforms Jonathan Dong Ruben Ohana M. Rafayelyan Florent Krzakala TPM 31 18 0 12 Jun 2020
Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification Huanrui Yang Minxue Tang W. Wen Feng Yan Daniel Hu Ang Li H. Li Yiran Chen 36 63 0 20 Apr 2020
Generalisation error in learning with random features and the hidden manifold model Federica Gerace Bruno Loureiro Florent Krzakala M. Mézard Lenka Zdeborová 25 166 0 21 Feb 2020
Iteratively Training Look-Up Tables for Network Quantization Fabien Cardinaux Stefan Uhlich K. Yoshiyama Javier Alonso García Lukas Mauch Stephen Tiedemann Thomas Kemp Akira Nakamura MQ 27 16 0 12 Nov 2019
Principled Training of Neural Networks with Direct Feedback Alignment Julien Launay Iacopo Poli Florent Krzakala 21 35 0 11 Jun 2019
Butterfly Transform: An Efficient FFT Based Neural Architecture Design Keivan Alizadeh-Vahid Anish K. Prabhu Ali Farhadi Mohammad Rastegari 32 50 0 05 Jun 2019
CircConv: A Structured Convolution with Low Complexity Siyu Liao Zhe Li Liang Zhao Qinru Qiu Yanzhi Wang Bo Yuan 27 18 0 28 Feb 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 37 307 0 15 Feb 2019
Understanding and Training Deep Diagonal Circulant Neural Networks Alexandre Araujo Benjamin Négrevergne Y. Chevaleyre Jamal Atif 27 4 0 29 Jan 2019
Building Efficient Deep Neural Networks with Unitary Group Convolutions Ritchie Zhao Yuwei Hu Jordan Dotzel Christopher De Sa Zhiru Zhang 29 28 0 19 Nov 2018
Learning Compressed Transforms with Low Displacement Rank Anna T. Thomas Albert Gu Tri Dao Atri Rudra Christopher Ré 27 40 0 04 Oct 2018
Entropy and mutual information in models of deep neural networks Marylou Gabrié Andre Manoel Clément Luneau Jean Barbier N. Macris Florent Krzakala Lenka Zdeborová 30 179 0 24 May 2018
Hybrid Binary Networks: Optimizing for Accuracy, Efficiency and Memory Ameya Prabhu Vishal Batchu Rohit Gajawada Sri Aurobindo Munagala A. Namboodiri MQ 33 18 0 11 Apr 2018
Moonshine: Distilling with Cheap Convolutions Elliot J. Crowley Gavia Gray Amos Storkey 27 120 0 07 Nov 2017
A Survey of Model Compression and Acceleration for Deep Neural Networks Yu Cheng Duo Wang Pan Zhou Zhang Tao 40 1,087 0 23 Oct 2017
Fine-Pruning: Joint Fine-Tuning and Compression of a Convolutional Network with Bayesian Optimization Frederick Tung S. Muralidharan Greg Mori 27 35 0 28 Jul 2017
Training Sparse Neural Networks Suraj Srinivas Akshayvarun Subramanya R. Venkatesh Babu 21 204 0 21 Nov 2016
Small-footprint Highway Deep Neural Networks for Speech Recognition Liang Lu Steve Renals 32 15 0 18 Oct 2016
HyperNetworks David R Ha Andrew M. Dai Quoc V. Le 84 1,589 0 27 Sep 2016
Multiplicative LSTM for sequence modelling Ben Krause Liang Lu Iain Murray Steve Renals 35 208 0 26 Sep 2016
Knowledge Distillation for Small-footprint Highway Networks Liang Lu Michelle Guo Steve Renals 24 73 0 02 Aug 2016
Structured Convolution Matrices for Energy-efficient Deep learning R. Appuswamy T. Nayak John V. Arthur S. K. Esser P. Merolla J. McKinstry T. Melano M. Flickner D. Modha 38 11 0 08 Jun 2016
Strongly-Typed Recurrent Neural Networks David Balduzzi Muhammad Ghifary PINN 16 60 0 06 Feb 2016