Training Neural Networks with Fixed Sparse Masks

18 November 2021

Papers citing "Training Neural Networks with Fixed Sparse Masks"

50 / 143 papers shown

Title
TeleSparse: Practical Privacy-Preserving Verification of Deep Neural Networks Mohammad Maheri Hamed Haddadi Alex Davidson 71 0 0 27 Apr 2025
Sparsity Outperforms Low-Rank Projections in Few-Shot Adaptation Nairouz Mrabah Nicolas Richet Ismail Ben Ayed Eric Granger BDL VLM 53 0 0 16 Apr 2025
FISH-Tuning: Enhancing PEFT Methods with Fisher Information Kang Xue Ming Dong Xinhui Tu Tingting He 34 0 0 05 Apr 2025
Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models Yuheng Wu Wentao Guo Zirui Liu Heng Ji Zhaozhuo Xu Denghui Zhang 33 0 0 05 Apr 2025
Efficient Model Editing with Task-Localized Sparse Fine-tuning Leonardo Iurada Marco Ciccone Tatiana Tommasi KELM MoMe 47 1 0 03 Apr 2025
An Efficient Training Algorithm for Models with Block-wise Sparsity Ding Zhu Zhiqun Zuo Mohammad Mahdi Khalili 37 0 0 27 Mar 2025
FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation Dong Zhao Jinlong Li Shuang Wang Mengyao Wu Qi Zang N. Sebe Zhun Zhong 147 0 0 23 Mar 2025
Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflection Romain Thoreau Valerio Marsocci Dawa Derksen AI4CE 71 2 0 12 Mar 2025
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model Wenke Huang Jian Liang Xianda Guo Yiyang Fang Guancheng Wan ... Bin Yang He Li Jiawei Shao Mang Ye Bo Du OffRL LRM MLLM KELM VLM 65 1 0 06 Mar 2025
Personalized Federated Fine-tuning for Heterogeneous Data: An Automatic Rank Learning Approach via Two-Level LoRA Jie Hao Yuman Wu Ali Payani Myungjin Lee Mingrui Liu 37 1 0 05 Mar 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 77 2 0 24 Feb 2025
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 56 0 0 17 Feb 2025
The impact of allocation strategies in subset learning on the expressive power of neural networks Ofir Schlisselberg Ran Darshan 93 0 0 10 Feb 2025
Memory-Efficient Fine-Tuning of Transformers via Token Selection Antoine Simoulin Namyong Park Xiaoyi Liu Grey Yang 112 0 0 31 Jan 2025
Sparse High Rank Adapters K. Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... P. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MQ 43 4 0 28 Jan 2025
Parameter-Efficient Fine-Tuning for Foundation Models Dan Zhang Tao Feng Lilong Xue Yuandong Wang Yuxiao Dong J. Tang 46 8 0 23 Jan 2025
EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition Hamid Nasiri Peter Garraghan 41 1 0 21 Jan 2025
Toward Realistic Camouflaged Object Detection: Benchmarks and Method Zhimeng Xin Tianxu Wu Shiming Chen Shuo Ye Zijing Xie Yixiong Zou Xinge You Yufei Guo 31 0 0 13 Jan 2025
CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasets Tanay Agrawal Mohammed Guermal Michal Balazia François Brémond 29 0 0 08 Jan 2025
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 94 1 0 08 Dec 2024
FisherMask: Enhancing Neural Network Labeling Efficiency in Image Classification Using Fisher Information Shreen Gul Mohamed Elmahallawy S. Madria Ardhendu Tripathy 28 1 0 08 Nov 2024
Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach Yequan Zhao Hai Li Ian Young Zheng-Wei Zhang MQ 37 2 0 07 Nov 2024
Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation Ayan Sengupta Vaibhav Seth Arinjay Pathak Natraj Raman Sriram Gopalakrishnan Tanmoy Chakraborty BDL 28 2 0 07 Nov 2024
Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envision Xiangzhong Luo Di Liu Hao Kong Shuo Huai Hui Chen Guochu Xiong Weichen Liu 31 2 0 03 Nov 2024
CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning Yeachan Kim Junho Kim SangKeun Lee NoLa AAML 35 2 0 31 Oct 2024
Ensembling Finetuned Language Models for Text Classification Sebastian Pineda Arango Maciej Janowski Lennart Purucker Arber Zela Frank Hutter Josif Grabocka 33 0 0 25 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning Zhiwei Hao Jianyuan Guo Li Shen Yong Luo Han Hu Yonggang Wen VLM 23 0 0 23 Oct 2024
LoLDU: Low-Rank Adaptation via Lower-Diag-Upper Decomposition for Parameter-Efficient Fine-Tuning Yiming Shi Jiwei Wei Yujia Wu Ran Ran Chengwei Sun Shiyuan He Yang Yang ALM 43 1 0 17 Oct 2024
LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models Hossein Abdi Mingfei Sun Andi Zhang Samuel Kaski Wei Pan 23 0 0 15 Oct 2024
Differentiable Weightless Neural Networks Alan T. L. Bacellar Zachary Susskind Mauricio Breternitz Jr. E. John L. John P. Lima F. M. G. França 22 3 0 14 Oct 2024
One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation Fabian Paischer Lukas Hauzenberger Thomas Schmied Benedikt Alkin Marc Peter Deisenroth Sepp Hochreiter 29 4 0 09 Oct 2024
WAPITI: A Watermark for Finetuned Open-Source LLMs Lingjie Chen Ruizhong Qiu Siyu Yuan Zhining Liu Tianxin Wei Hyunsik Yoo Zhichen Zeng Deqing Yang Hanghang Tong WaLM 39 4 0 09 Oct 2024
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models Cong Guo Feng Cheng Zhixu Du James Kiessling Jonathan Ku ... Qilin Zheng Guanglei Zhou Hai Li-Wei Li Yiran Chen 31 7 0 08 Oct 2024
OD-Stega: LLM-Based Near-Imperceptible Steganography via Optimized Distributions Yu-Shin Huang Peter Just Krishna Narayanan Chao Tian 34 1 0 06 Oct 2024
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 52 3 0 02 Oct 2024
Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models Aradhye Agarwal Suhas K Ramesh Ayan Sengupta Tanmoy Chakraborty 25 1 0 26 Aug 2024
Multi-scale Contrastive Adaptor Learning for Segmenting Anything in Underperformed Scenes Ke Zhou Zhongwei Qiu Dongmei Fu VLM 35 1 0 12 Aug 2024
Rapid Switching and Multi-Adapter Fusion via Sparse High Rank Adapters Kartikeya Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... Paul N. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MoMe 28 0 0 22 Jul 2024
Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach Yuzhu Mao Siqi Ping Zihao Zhao Yang Liu Wenbo Ding 37 1 0 16 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 34 1 0 11 Jul 2024
ROSA: Random Subspace Adaptation for Efficient Fine-Tuning Marawan Gamal Abdel Hameed Aristides Milios Siva Reddy Guillaume Rabusseau CLL 24 3 0 10 Jul 2024
SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning Haiwen Diao Bo Wan Xu Jia Yunzhi Zhuge Ying Zhang Huchuan Lu Long Chen VLM 47 4 0 10 Jul 2024
See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition Chongjie Si Xiaokang Yang Wei Shen 38 5 0 07 Jul 2024
SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning Bac Nguyen Stefan Uhlich Fabien Cardinaux Lukas Mauch Marzieh Edraki Aaron Courville OODD CLL VLM 54 3 0 03 Jul 2024
Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models Zihan Wang Deli Chen Damai Dai Runxin Xu Zhuoshu Li Y. Wu MoE ALM 40 2 0 02 Jul 2024
Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning Haobo Song Hao Zhao Soumajit Majumder Tao Lin 30 3 0 01 Jul 2024
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 38 7 0 07 Jun 2024
MEFT: Memory-Efficient Fine-Tuning through Sparse Adapter Jitai Hao Weiwei Sun Xin Xin Qi Meng Zhumin Chen Pengjie Ren Zhaochun Ren MoE 42 2 0 07 Jun 2024
Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity Wentao Guo Jikai Long Yimeng Zeng Zirui Liu Xinyu Yang ... Osbert Bastani Christopher De Sa Xiaodong Yu Beidi Chen Zhaozhuo Xu 28 14 0 05 Jun 2024