NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for
Large Language Models

NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models

28 February 2024

Amit Dhurandhar

Tejaswini Pedapati

Aurélie C. Lozano

Georgios Kollias

Papers citing "NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models"

12 / 12 papers shown

Title
Identifying Sub-networks in Neural Networks via Functionally Similar Representations Tian Gao Amit Dhurandhar Karthikeyan N. Ramamurthy Dennis L. Wei 64 0 0 21 Oct 2024
Intriguing Properties of Quantization at Scale Arash Ahmadian Saurabh Dash Hongyu Chen Bharat Venkitesh Stephen Gou Phil Blunsom Ahmet Üstün Sara Hooker MQ 61 38 0 30 May 2023
Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability Ziming Liu Eric Gan Max Tegmark 54 38 0 04 May 2023
EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets Xiaohan Chen Yu Cheng Shuohang Wang Zhe Gan Zhangyang Wang Jingjing Liu 58 100 0 31 Dec 2020
Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers Junjie Liu Zhe Xu Runbin Shi R. Cheung Hayden Kwok-Hay So 26 120 0 14 May 2020
Reducing Transformer Depth on Demand with Structured Dropout Angela Fan Edouard Grave Armand Joulin 88 586 0 25 Sep 2019
Are Sixteen Heads Really Better than One? Paul Michel Omer Levy Graham Neubig MoE 64 1,049 0 25 May 2019
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned Elena Voita David Talbot F. Moiseev Rico Sennrich Ivan Titov 76 1,120 0 23 May 2019
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 200 1,190 0 04 Oct 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 162 3,433 0 09 Mar 2018
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 189 8,793 0 01 Oct 2015
Structured sparsity through convex optimization Francis R. Bach Rodolphe Jenatton Julien Mairal G. Obozinski 165 324 0 12 Sep 2011