Structured Transforms for Small-Footprint Deep Learning

6 October 2015

Sanjiv Kumar

Papers citing "Structured Transforms for Small-Footprint Deep Learning"

50 / 53 papers shown

Title
NdLinear Is All You Need for Representation Learning Alex Reneau Jerry Yao-Chieh Hu Zhongfang Zhuang Ting-Chun Liu HAI 44 0 0 21 Mar 2025
Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models Xubin Wang Zhiqing Tang Jianxiong Guo Tianhui Meng Chenhao Wang Tian-sheng Wang Weijia Jia 65 1 0 08 Mar 2025
Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks Ashwin Samudre Mircea Petrache Brian D. Nord Shubhendu Trivedi 55 2 0 18 Sep 2024
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 22 52 0 13 Feb 2023
Arithmetic Circuits, Structured Matrices and (not so) Deep Learning Atri Rudra 21 1 0 24 Jun 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 116 2,055 0 27 May 2022
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 32 87 0 01 Apr 2022
Mixed Precision Low-bit Quantization of Neural Network Language Models for Speech Recognition Junhao Xu Jianwei Yu Shoukang Hu Xunying Liu Helen Meng MQ 30 13 0 29 Nov 2021
Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers Junhao Xu Xie Chen Shoukang Hu Jianwei Yu Xunying Liu Helen Meng MQ 28 9 0 29 Nov 2021
CHIP: CHannel Independence-based Pruning for Compact Neural Networks Yang Sui Miao Yin Yi Xie Huy Phan S. Zonouz Bo Yuan VLM 35 129 0 26 Oct 2021
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression Weiyue Su Xuyi Chen Shi Feng Jiaxiang Liu Weixin Liu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 13 0 04 Jun 2021
FNet: Mixing Tokens with Fourier Transforms James Lee-Thorp Joshua Ainslie Ilya Eckstein Santiago Ontanon 47 520 0 09 May 2021
GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning Juhyoung Lee Sangyeob Kim Sangjin Kim Wooyoung Jo H. Yoo OffRL 21 9 0 24 Jan 2021
Sparse Linear Networks with a Fixed Butterfly Structure: Theory and Practice Nir Ailon Omer Leibovitch Vineet Nair 15 14 0 17 Jul 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 23 2,857 0 09 Jun 2020
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects Zewen Li Wenjie Yang Shouheng Peng Fan Liu HAI 3DV 64 2,608 0 01 Apr 2020
Communication-Efficient Edge AI: Algorithms and Systems Yuanming Shi Kai Yang Tao Jiang Jun Zhang Khaled B. Letaief GNN 29 327 0 22 Feb 2020
Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread Haofeng Li Guanbin Li Binbin Yang Guanqi Chen Liang Lin Yizhou Yu ObjD 46 28 0 22 Jan 2020
Discrimination-aware Network Pruning for Deep Model Compression Jing Liu Bohan Zhuang Zhuangwei Zhuang Yong Guo Junzhou Huang Jin-Hui Zhu Mingkui Tan CVBM 19 119 0 04 Jan 2020
Iteratively Training Look-Up Tables for Network Quantization Fabien Cardinaux Stefan Uhlich K. Yoshiyama Javier Alonso García Lukas Mauch Stephen Tiedemann Thomas Kemp Akira Nakamura MQ 27 16 0 12 Nov 2019
Extremely Small BERT Models from Mixed-Vocabulary Training Sanqiang Zhao Raghav Gupta Yang Song Denny Zhou VLM 14 53 0 25 Sep 2019
Survey on Deep Neural Networks in Speech and Vision Systems M. Alam Manar D. Samad Lasitha Vidyaratne Alexander M. Glandon Khan M. Iftekharuddin 3DV VLM AI4TS 34 205 0 16 Aug 2019
Compressing RNNs for IoT devices by 15-38x using Kronecker Products Urmish Thakker Jesse G. Beu Dibakar Gope Chu Zhou Igor Fedorov Ganesh S. Dasika Matthew Mattina 27 36 0 07 Jun 2019
Butterfly Transform: An Efficient FFT Based Neural Architecture Design Keivan Alizadeh-Vahid Anish K. Prabhu Ali Farhadi Mohammad Rastegari 32 50 0 05 Jun 2019
Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations Tri Dao Albert Gu Matthew Eichhorn Atri Rudra Christopher Ré 24 102 0 14 Mar 2019
CircConv: A Structured Convolution with Low Complexity Siyu Liao Zhe Li Liang Zhao Qinru Qiu Yanzhi Wang Bo Yuan 27 18 0 28 Feb 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 37 307 0 15 Feb 2019
Understanding and Training Deep Diagonal Circulant Neural Networks Alexandre Araujo Benjamin Négrevergne Y. Chevaleyre Jamal Atif 27 4 0 29 Jan 2019
Pre-Defined Sparse Neural Networks with Hardware Acceleration Sourya Dey Kuan-Wen Huang P. Beerel K. Chugg 41 24 0 04 Dec 2018
Building Efficient Deep Neural Networks with Unitary Group Convolutions Ritchie Zhao Yuwei Hu Jordan Dotzel Christopher De Sa Zhiru Zhang 32 28 0 19 Nov 2018
Learning Compressed Transforms with Low Displacement Rank Anna T. Thomas Albert Gu Tri Dao Atri Rudra Christopher Ré 27 40 0 04 Oct 2018
Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition Chun-Fu Chen Quanfu Fan Neil Rohit Mallinar Tom Sercu Rogerio Feris 20 96 0 10 Jul 2018
Resource-Efficient Neural Architect Yanqi Zhou S. Ebrahimi Sercan Ö. Arik Haonan Yu Hairong Liu G. Diamos 22 64 0 12 Jun 2018
Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression Jiahao Su Jingling Li Bobby Bhattacharjee Furong Huang 16 20 0 25 May 2018
Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds Cenk Baykal Lucas Liebenwein Igor Gilitschenski Dan Feldman Daniela Rus 25 79 0 15 Apr 2018
Structured Evolution with Compact Architectures for Scalable Policy Optimization K. Choromanski Mark Rowland Vikas Sindhwani Richard Turner Adrian Weller 27 147 0 06 Apr 2018
FFT-Based Deep Learning Deployment in Embedded Systems Sheng Lin Ning Liu M. Nazemi Hongjia Li Caiwen Ding Yanzhi Wang Massoud Pedram 41 52 0 13 Dec 2017
BlockDrop: Dynamic Inference Paths in Residual Networks Zuxuan Wu Tushar Nagarajan Abhishek Kumar Steven J. Rennie L. Davis Kristen Grauman Rogerio Feris 42 462 0 22 Nov 2017
Trace norm regularization and faster inference for embedded speech recognition RNNs Markus Kliegl Siddharth Goyal Kexin Zhao Kavya Srinet M. Shoeybi 40 8 0 25 Oct 2017
A Survey of Model Compression and Acceleration for Deep Neural Networks Yu Cheng Duo Wang Pan Zhou Zhang Tao 40 1,087 0 23 Oct 2017
Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting Sercan Ö. Arik Markus Kliegl R. Child Joel Hestness Andrew Gibiansky Christopher Fougner R. Prenger Adam Coates 35 180 0 15 Mar 2017
Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank Liang Zhao Siyu Liao Yanzhi Wang Zhe Li Jian Tang Victor Pan Bo Yuan 31 61 0 01 Mar 2017
Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification Y. Lu Abhishek Kumar Shuangfei Zhai Yu Cheng T. Javidi Rogerio Feris 3DH 21 384 0 16 Nov 2016
Doubly Convolutional Neural Networks Shuangfei Zhai Yu Cheng Weining Lu Zhongfei Zhang OOD 3DV 22 63 0 30 Oct 2016
Small-footprint Highway Deep Neural Networks for Speech Recognition Liang Lu Steve Renals 38 15 0 18 Oct 2016
Knowledge Distillation for Small-footprint Highway Networks Liang Lu Michelle Guo Steve Renals 26 73 0 02 Aug 2016
Structured Convolution Matrices for Energy-efficient Deep learning R. Appuswamy T. Nayak John V. Arthur S. K. Esser P. Merolla J. McKinstry T. Melano M. Flickner D. Modha 38 11 0 08 Jun 2016
TripleSpin - a generic compact paradigm for fast machine learning computations K. Choromanski Francois Fagan Cédric Gouy-Pailler Anne Morvan Vikas Sindhwani Jamal Atif 42 7 0 29 May 2016
Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding Milos Cernak Alexandros Lazaridis Afsaneh Asaei Philip N. Garner 21 29 0 15 Apr 2016
Learning Compact Recurrent Neural Networks Zhiyun Lu Vikas Sindhwani Tara N. Sainath 25 86 0 09 Apr 2016