ResearchTrend.AI
Matrix Compression via Randomized Low Rank and Low Precision Factorization

Matrix Compression via Randomized Low Rank and Low Precision Factorization

R. Saha
Varun Srivastava
Mert Pilanci
ArXivPDFHTML

Papers citing "Matrix Compression via Randomized Low Rank and Low Precision Factorization"

13 / 13 papers shown
Title
Clustering-based Low-Rank Matrix Approximation: An Adaptive Theoretical Analysis with Application to Data Compression
Clustering-based Low-Rank Matrix Approximation: An Adaptive Theoretical Analysis with Application to Data Compression
Sisipho Hamlomo
M. Atemkeng
34
0
0
13 May 2025
Pushing the Envelope of Low-Bit LLM via Dynamic Error Compensation
Pushing the Envelope of Low-Bit LLM via Dynamic Error Compensation
Y. Park
Jake Hyun
Hojoon Kim
Jae W. Lee
MQ
59
0
0
31 Dec 2024
Reassessing Layer Pruning in LLMs: New Insights and Methods
Reassessing Layer Pruning in LLMs: New Insights and Methods
Yao Lu
Hao Cheng
Yujie Fang
Zeyu Wang
Jiaheng Wei
Dongwei Xu
Qi Xuan
Xiaoniu Yang
Zhaowei Zhu
73
0
0
23 Nov 2024
FiRST: Finetuning Router-Selective Transformers for Input-Adaptive Latency Reduction
FiRST: Finetuning Router-Selective Transformers for Input-Adaptive Latency Reduction
Akriti Jain
Saransh Sharma
Koyel Mukherjee
Soumyabrata Pal
36
1
0
16 Oct 2024
Getting Free Bits Back from Rotational Symmetries in LLMs
Getting Free Bits Back from Rotational Symmetries in LLMs
Jiajun He
Gergely Flamich
José Miguel Hernández-Lobato
MQ
23
0
0
02 Oct 2024
In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting
In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting
Constantin Philippenko
Kevin Scaman
Laurent Massoulié
FedML
42
1
0
13 Sep 2024
On-Device Language Models: A Comprehensive Review
On-Device Language Models: A Comprehensive Review
Jiajun Xu
Zhiyuan Li
Wei Chen
Qun Wang
Xin Gao
Qi Cai
Ziyuan Ling
59
31
0
26 Aug 2024
From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients
From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients
Ajay Jaiswal
Lu Yin
Zhenyu Zhang
Shiwei Liu
Jiawei Zhao
Yuandong Tian
Zhangyang Wang
43
14
0
15 Jul 2024
Compressing Large Language Models using Low Rank and Low Precision Decomposition
Compressing Large Language Models using Low Rank and Low Precision Decomposition
R. Saha
Naomi Sagan
Varun Srivastava
Andrea J. Goldsmith
Mert Pilanci
MQ
30
11
0
29 May 2024
A Survey on Efficient Inference for Large Language Models
A Survey on Efficient Inference for Large Language Models
Zixuan Zhou
Xuefei Ning
Ke Hong
Tianyu Fu
Jiaming Xu
...
Shengen Yan
Guohao Dai
Xiao-Ping Zhang
Yuhan Dong
Yu Wang
51
87
0
22 Apr 2024
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
Han Guo
P. Greengard
Eric P. Xing
Yoon Kim
MQ
43
44
0
20 Nov 2023
A Survey on Model Compression for Large Language Models
A Survey on Model Compression for Large Language Models
Xunyu Zhu
Jian Li
Yong Liu
Can Ma
Weiping Wang
45
200
0
15 Aug 2023
DRIVE: One-bit Distributed Mean Estimation
DRIVE: One-bit Distributed Mean Estimation
S. Vargaftik
Ran Ben-Basat
Amit Portnoy
Gal Mendelson
Y. Ben-Itzhak
Michael Mitzenmacher
OOD
FedML
86
52
0
18 May 2021