LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning

24 May 2025

Papers citing "LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning"

26 / 26 papers shown

Title
Scaling Laws for Precision Tanishq Kumar Zachary Ankner Benjamin Spector Blake Bordelon Niklas Muennighoff Mansheej Paul Cengiz Pehlevan Christopher Ré Aditi Raghunathan AIFin MoMe 96 29 0 07 Nov 2024
IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models Hang Guo Yawei Li Tao Dai Shu-Tao Xia Luca Benini MQ 119 2 0 29 Oct 2024
RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization Xijie Huang Zechun Liu Shih-yang Liu Kwang-Ting Cheng MQ 84 9 0 10 Jul 2024
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Mengzhao Chen Wenqi Shao Peng Xu Jiahao Wang Peng Gao Kaipeng Zhang Ping Luo MQ 143 35 0 10 Jul 2024
Low-Rank Quantization-Aware Training for LLMs Yelysei Bondarenko Riccardo Del Chiaro Markus Nagel MQ 77 14 0 10 Jun 2024
A Survey on Large Language Models for Code Generation Juyong Jiang Fan Wang Jiasi Shen Sungju Kim Sunghun Kim 128 201 0 01 Jun 2024
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs Saleh Ashkboos Amirkeivan Mohtashami Maximilian L. Croci Bo Li Martin Jaggi Dan Alistarh Torsten Hoefler James Hensman MQ 122 184 0 30 Mar 2024
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models Hyesung Jeon Yulhwa Kim Jae-Joon Kim MQ 50 5 0 07 Feb 2024
Efficient Large Language Models: A Survey Zhongwei Wan Xin Wang Che Liu Samiul Alam Yu Zheng ... Shen Yan Yi Zhu Quanlu Zhang Mosharaf Chowdhury Mi Zhang LM&MA 70 137 0 06 Dec 2023
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models Yixiao Li Yifan Yu Chen Liang Pengcheng He Nikos Karampatziakis Weizhu Chen Tuo Zhao MQ 123 148 0 12 Oct 2023
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models Yuhui Xu Lingxi Xie Xiaotao Gu Xin Chen Heng Chang Hengheng Zhang Zhensu Chen Xiaopeng Zhang Qi Tian MQ 73 107 0 26 Sep 2023
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs Wenhua Cheng Weiwei Zhang Haihao Shen Yiyang Cai Xin He Kaokao Lv Yi. Liu MQ 143 25 0 11 Sep 2023
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration Ji Lin Jiaming Tang Haotian Tang Shang Yang Wei-Ming Chen Wei-Chen Wang Guangxuan Xiao Xingyu Dang Chuang Gan Song Han EDL MQ 112 582 0 01 Jun 2023
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models Zechun Liu Barlas Oğuz Changsheng Zhao Ernie Chang Pierre Stock Yashar Mehdad Yangyang Shi Raghuraman Krishnamoorthi Vikas Chandra MQ 111 208 0 29 May 2023
QLoRA: Efficient Finetuning of Quantized LLMs Tim Dettmers Artidoro Pagnoni Ari Holtzman Luke Zettlemoyer ALM 157 2,615 0 23 May 2023
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers Elias Frantar Saleh Ashkboos Torsten Hoefler Dan Alistarh MQ 157 1,012 0 31 Oct 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 380 4,601 0 27 Oct 2021
On Faster Convergence of Scaled Sign Gradient Descent Xiuxian Li Kuo-Yi Lin Li Li Yiguang Hong Jie-bin Chen ODL 61 11 0 04 Sep 2021
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 191 4,580 0 07 Sep 2020
The Geometry of Sign Gradient Descent Lukas Balles Fabian Pedregosa Nicolas Le Roux ODL 72 27 0 19 Feb 2020
ViGGO: A Video Game Corpus for Data-To-Text Generation in Open-Domain Conversation Juraj Juraska Kevin K. Bowden M. Walker 60 44 0 26 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 530 20,378 0 23 Oct 2019
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task Tao Yu Rui Zhang Kai-Chou Yang Michihiro Yasunaga Dongxu Wang ... Irene Li Qingning Yao Shanelle Roman Zilin Zhang Dragomir R. Radev RALM 137 1,247 0 24 Sep 2018
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning Victor Zhong Caiming Xiong R. Socher RALM 141 1,209 0 31 Aug 2017
Ternary Weight Networks Fengfu Li Bin Liu Xiaoxing Wang Bo Zhang Junchi Yan MQ 89 526 0 16 May 2016
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 358 18,664 0 06 Feb 2015