Faster Neural Network Training with Approximate Tensor Operations

Faster Neural Network Training with Approximate Tensor Operations

21 May 2018

Menachem Adelman

Papers citing "Faster Neural Network Training with Approximate Tensor Operations"

15 / 15 papers shown

Title
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang ALM MQ 90 0 0 18 Feb 2025
TinyProp -- Adaptive Sparse Backpropagation for Efficient TinyML On-device Learning Marcus Rüb Daniel Maier Daniel Mueller-Gritschneder Axel Sikora 26 3 0 17 Aug 2023
Training Transformers with 4-bit Integers Haocheng Xi Changhao Li Jianfei Chen Jun Zhu MQ 25 47 0 21 Jun 2023
Fine-Tuning Language Models with Just Forward Passes Sadhika Malladi Tianyu Gao Eshaan Nichani Alexandru Damian Jason D. Lee Danqi Chen Sanjeev Arora 22 177 0 27 May 2023
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model Zirui Liu Guanchu Wang Shaochen Zhong Zhaozhuo Xu Daochen Zha ... Zhimeng Jiang Kaixiong Zhou V. Chaudhary Shuai Xu Xia Hu 36 11 0 24 May 2023
RSC: Accelerating Graph Neural Networks Training via Randomized Sparse Computations Zirui Liu Sheng-Wei Chen Kaixiong Zhou Daochen Zha Xiao Huang Xia Hu 32 14 0 19 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 31 47 0 13 Oct 2022
Memory-Efficient Backpropagation through Large Linear Layers Daniel Bershatsky A. Mikhalev A. Katrutsa Julia Gusak D. Merkulov Ivan V. Oseledets 11 4 0 31 Jan 2022
L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization Jiaqi Gu Hanqing Zhu Chenghao Feng Zixuan Jiang Ray T. Chen David Z. Pan 18 28 0 27 Oct 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 50 110 0 16 Feb 2021
Randomized Automatic Differentiation Deniz Oktay N. McGreivy Joshua Aduol Alex Beatson Ryan P. Adams ODL 14 26 0 20 Jul 2020
Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights Shail Dave Riyadh Baghdadi Tony Nowatzki Sasikanth Avancha Aviral Shrivastava Baoxin Li 56 81 0 02 Jul 2020
Dithered backprop: A sparse and quantized backpropagation algorithm for more efficient deep neural network training Simon Wiedemann Temesgen Mehari Kevin Kepp Wojciech Samek 17 18 0 09 Apr 2020
Improved Subsampled Randomized Hadamard Transform for Linear SVM Zijian Lei Liang Lan 13 8 0 05 Feb 2020
Accelerated CNN Training Through Gradient Approximation Ziheng Wang Sree Harsha Nelaturu 75 5 0 15 Aug 2019