The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

9 March 2018

Papers citing "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks"

50 / 2,030 papers shown

Title
A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems Hung Vinh Tran Tong Chen Quoc Viet Hung Nguyen Zi-Rui Huang Lizhen Cui Hongzhi Yin 141 2 0 25 Jun 2024
Unlocking Continual Learning Abilities in Language Models Wenyu Du Shuang Cheng Tongxu Luo Zihan Qiu Zeyu Huang Ka Chun Cheung Reynold Cheng Jie Fu KELM CLL 109 10 0 25 Jun 2024
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs Ashwinee Panda Berivan Isik Xiangyu Qi Sanmi Koyejo Tsachy Weissman Prateek Mittal MoMe 132 16 0 24 Jun 2024
ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models Yash Akhauri Ahmed F. AbouElhamayed Jordan Dotzel Zhiru Zhang Alexander M Rush Safeen Huda Mohamed S. Abdelfattah 49 5 0 24 Jun 2024
Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers Xiuying Wei Skander Moalla Razvan Pascanu Çağlar Gülçehre 81 2 0 24 Jun 2024
Towards Lightweight Graph Neural Network Search with Curriculum Graph Sparsification Beini Xie Heng Chang Ziwei Zhang Zeyang Zhang Simin Wu Xin Wang Yuan Meng Wenwu Zhu 78 3 0 24 Jun 2024
Synergizing Foundation Models and Federated Learning: A Survey Shenghui Li Fanghua Ye Meng Fang Jiaxu Zhao Yun-Hin Chan Edith C. -H. Ngai Thiemo Voigt AI4CE 109 7 0 18 Jun 2024
Sparsifying dimensionality reduction of PDE solution data with Bregman learning T. J. Heeringa Christoph Brune Mengwu Guo 62 0 0 18 Jun 2024
Low-Redundant Optimization for Large Language Model Alignment Zhipeng Chen Kun Zhou Wayne Xin Zhao Jingyuan Wang Ji-Rong Wen 78 3 0 18 Jun 2024
MOYU: A Theoretical Study on Massive Over-activation Yielded Uplifts in LLMs Chi Ma Mincong Huang Chao Wang Yujie Wang Lei Yu AI4CE 57 0 0 18 Jun 2024
Finding Task-specific Subnetworks in Multi-task Spoken Language Understanding Model Hayato Futami Siddhant Arora Yosuke Kashiwagi E. Tsunoo Shinji Watanabe 81 1 0 18 Jun 2024
Endor: Hardware-Friendly Sparse Format for Offloaded LLM Inference Donghyeon Joo Ramyad Hadidi Soheil Feizi Bahar Asgari MQ 45 0 0 17 Jun 2024
DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling Pala Tej Deep Rishabh Bhardwaj Soujanya Poria MoMe 81 31 0 17 Jun 2024
Scale Equivariant Graph Metanetworks Ioannis Kalogeropoulos Giorgos Bouritsas Yannis Panagakis 122 11 0 15 Jun 2024
DIET: Customized Slimming for Incompatible Networks in Sequential Recommendation Kairui Fu Shengyu Zhang Zheqi Lv Jingyuan Chen Jiwei Li 70 3 0 13 Jun 2024
Pre-Training Identification of Graph Winning Tickets in Adaptive Spatial-Temporal Graph Neural Networks Wenying Duan Tianxiang Fang Hong Rao Xiaoxi He 80 0 0 12 Jun 2024
MoreauPruner: Robust Pruning of Large Language Models against Weight Perturbations Zixiao Wang Jingwei Zhang Wenqian Zhao Farzan Farnia Bei Yu AAML 71 3 0 11 Jun 2024
Sparse Bayesian Networks: Efficient Uncertainty Quantification in Medical Image Analysis Zeinab Abboud Herve Lombaert Samuel Kadoury UQCV 131 1 0 11 Jun 2024
Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustness Dingrong Wang Hitesh Sapkota Zhiqiang Tao Qi Yu AAML 63 1 0 10 Jun 2024
Forget Sharpness: Perturbed Forgetting of Model Biases Within SAM Dynamics Ankit Vani Frederick Tung Gabriel L. Oliveira Hossein Sharifi-Noghabi AAML 76 0 0 10 Jun 2024
Interpretability of Language Models via Task Spaces Lucas Weber Jaap Jumelet Elia Bruni Dieuwke Hupkes 54 4 0 10 Jun 2024
Geometric sparsification in recurrent neural networks Wyatt Mackey Ioannis Schizas Jared Deighton David L. Boothe, Jr. Vasileios Maroulas 59 0 0 10 Jun 2024
Compute Better Spent: Replacing Dense Layers with Structured Matrices Shikai Qiu Andres Potapczynski Marc Finzi Micah Goldblum Andrew Gordon Wilson 84 18 0 10 Jun 2024
Evaluating Zero-Shot Long-Context LLM Compression Chenyu Wang Yihan Wang Kai Li 110 0 0 10 Jun 2024
Optimal Eye Surgeon: Finding Image Priors through Sparse Generators at Initialization Avrajit Ghosh Xitong Zhang Kenneth K. Sun Qing Qu S. Ravishankar Rongrong Wang MedIm 74 5 0 07 Jun 2024
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 104 11 0 07 Jun 2024
Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems Rohan R. Paleja Michael Munje K. Chang Reed Jensen Matthew C. Gombolay 86 2 0 07 Jun 2024
Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction Christoph Jürgen Hemmer Manuel Brenner Florian Hess Daniel Durstewitz 101 4 0 07 Jun 2024
MeGA: Merging Multiple Independently Trained Neural Networks Based on Genetic Algorithm Daniel Yun FedML MoMe 46 1 0 07 Jun 2024
Concurrent Training and Layer Pruning of Deep Neural Networks Valentin Frank Ingmar Guenter Athanasios Sideris 3DPC 79 3 0 06 Jun 2024
Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning Naibin Gu Peng Fu Xiyu Liu Bowen Shen Zheng Lin Weiping Wang 69 10 0 06 Jun 2024
Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity Wentao Guo Jikai Long Yimeng Zeng Zirui Liu Xinyu Yang ... Osbert Bastani Christopher De Sa Xiaodong Yu Beidi Chen Zhaozhuo Xu 90 21 0 05 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 181 6 0 05 Jun 2024
Cyclic Sparse Training: Is it Enough? Advait Gadhikar Sree Harsha Nelaturu R. Burkholz CLL 91 0 0 04 Jun 2024
Linguistic Fingerprint in Transformer Models: How Language Variation Influences Parameter Selection in Irony Detection Michele Mastromattei Fabio Massimo Zanzotto 64 0 0 04 Jun 2024
SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining Andi Han Jiaxiang Li Wei Huang Mingyi Hong Akiko Takeda Pratik Jawanpuria Bamdev Mishra 102 16 0 04 Jun 2024
Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning Leonardo Iurada Marco Ciccone Tatiana Tommasi 80 3 0 03 Jun 2024
Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization A. Nowak Lukasz Gniecki Filip Szatkowski Jacek Tabor 75 1 0 03 Jun 2024
Tilting the Odds at the Lottery: the Interplay of Overparameterisation and Curricula in Neural Networks Stefano Sarao Mannelli Yaraslau Ivashinka Andrew M. Saxe Luca Saglietti 67 2 0 03 Jun 2024
What makes unlearning hard and what to do about it Kairan Zhao M. Kurmanji George-Octavian Barbulescu Eleni Triantafillou Peter Triantafillou MU 117 25 0 03 Jun 2024
Effective Subset Selection Through The Lens of Neural Network Pruning Noga Bar Raja Giryes CVBM 67 0 0 03 Jun 2024
Efficient Neural Light Fields (ENeLF) for Mobile Devices Austin Peng 57 1 0 02 Jun 2024
Effective Interplay between Sparsity and Quantization: From Theory to Practice Simla Burcu Harma Ayan Chakraborty Elizaveta Kostenok Danila Mishin Dongho Ha ... Martin Jaggi Ming Liu Yunho Oh Suvinay Subramanian Amir Yazdanbakhsh MQ 96 10 0 31 May 2024
Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Study Pallavi Mitra Gesina Schwalbe Nadja Klein AAML 69 1 0 31 May 2024
The EarlyBird Gets the WORM: Heuristically Accelerating EarlyBird Convergence Adithya Vasudev 78 0 0 31 May 2024
Enhancing Generative Molecular Design via Uncertainty-guided Fine-tuning of Variational Autoencoders Naﬁz Abeer Sanket Jantre Nathan M. Urban Byung-Jun Yoon 80 0 0 31 May 2024
Occam Gradient Descent B. N. Kausik ODL VLM 56 0 0 30 May 2024
$Dual sparse training framework: inducing activation map sparsity via Transformed $\ell1$ regularization$ Dual sparse training framework: inducing activation map sparsity via Transformed $\ell1$ regularization Xiaolong Yu Cong Tian 77 0 0 30 May 2024
ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning Ruchika Chavhan Da Li Timothy M. Hospedales 91 16 0 29 May 2024
Survival of the Fittest Representation: A Case Study with Modular Addition Xiaoman Delores Ding Zifan Carl Guo Eric J. Michaud Ziming Liu Max Tegmark 116 4 0 27 May 2024