Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps

29 December 2020

Papers citing "Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps"

20 / 20 papers shown

Title
Geometry is All You Need: A Unified Taxonomy of Matrix and Tensor Factorization for Compression of Generative Language Models Mingxue Xu Sadia Sharmin Danilo Mandic 32 2 0 03 Oct 2024
Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks Ashwin Samudre Mircea Petrache Brian D. Nord Shubhendu Trivedi 50 2 0 18 Sep 2024
MoRe Fine-Tuning with 10x Fewer Parameters Wenxuan Tan Nicholas Roberts Tzu-Heng Huang Jitian Zhao John Cooper Samuel Guo Chengyu Duan Frederic Sala 34 0 0 30 Aug 2024
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation Shen Yuan Haotian Liu Hongteng Xu 44 2 0 24 May 2024
Black Box Lie Group Preconditioners for SGD Xi-Lin Li 18 8 0 08 Nov 2022
Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design Hongxiang Fan Thomas C. P. Chau Stylianos I. Venieris Royson Lee Alexandros Kouris Wayne Luk Nicholas D. Lane Mohamed S. Abdelfattah 40 58 0 20 Sep 2022
Arithmetic Circuits, Structured Matrices and (not so) Deep Learning Atri Rudra 18 1 0 24 Jun 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 93 2,045 0 27 May 2022
AANG: Automating Auxiliary Learning Lucio Dery Paul Michel M. Khodak Graham Neubig Ameet Talwalkar 41 9 0 27 May 2022
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 26 87 0 01 Apr 2022
Low-Rank Constraints for Fast Inference in Structured Models Justin T. Chiu Yuntian Deng Alexander M. Rush BDL 32 13 0 08 Jan 2022
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models Tri Dao Beidi Chen Kaizhao Liang Jiaming Yang Zhao Song Atri Rudra Christopher Ré 33 75 0 30 Nov 2021
Efficient Identification of Butterfly Sparse Matrix Factorizations Léon Zheng E. Riccietti Rémi Gribonval 44 6 0 04 Oct 2021
Is the Number of Trainable Parameters All That Actually Matters? A. Chatelain Amine Djeghri Daniel Hesslow Julien Launay Iacopo Poli 51 7 0 24 Sep 2021
Initialization and Regularization of Factorized Neural Layers M. Khodak Neil A. Tenenholtz Lester W. Mackey Nicolò Fusi 65 56 0 03 May 2021
Rethinking Neural Operations for Diverse Tasks Nicholas Roberts M. Khodak Tri Dao Liam Li Christopher Ré Ameet Talwalkar AI4CE 36 22 0 29 Mar 2021
Physics-Informed Neural State Space Models via Learning and Evolution Elliott Skomski Ján Drgoňa Aaron Tuor PINN AI4CE 27 9 0 26 Nov 2020
HiPPO: Recurrent Memory with Optimal Polynomial Projections Albert Gu Tri Dao Stefano Ermon Atri Rudra Christopher Ré 33 489 0 17 Aug 2020
Sparse Linear Networks with a Fixed Butterfly Structure: Theory and Practice Nir Ailon Omer Leibovitch Vineet Nair 15 14 0 17 Jul 2020
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 300 10,225 0 16 Nov 2016