Convex Distillation: Efficient Compression of Deep Networks via Convex Optimization

9 October 2024

Papers citing "Convex Distillation: Efficient Compression of Deep Networks via Convex Optimization"

21 / 21 papers shown

Title
Convexifying Transformers: Improving optimization and understanding of transformer networks Tolga Ergen Behnam Neyshabur Harsh Mehta MLT 81 15 0 20 Nov 2022
FitHuBERT: Going Thinner and Deeper for Knowledge Distillation of Speech Self-Supervised Learning Yeonghyeon Lee Kangwook Jang Jahyun Goo Youngmoon Jung Hoi-Rim Kim 95 33 0 01 Jul 2022
Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions Aaron Mishkin Arda Sahiner Mert Pilanci OffRL 116 30 0 02 Feb 2022
Distilling Knowledge via Knowledge Review Pengguang Chen Shu Liu Hengshuang Zhao Jiaya Jia 186 440 0 19 Apr 2021
The Pitfalls of Simplicity Bias in Neural Networks Harshay Shah Kaustav Tamuly Aditi Raghunathan Prateek Jain Praneeth Netrapalli AAML 65 359 0 13 Jun 2020
Natural Language Processing Advancements By Deep Learning: A Survey A. Torfi Rouzbeh A. Shirvani Yaser Keneshloo Nader Tavvaf Edward A. Fox AI4CE VLM 125 220 0 02 Mar 2020
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks Mert Pilanci Tolga Ergen 77 118 0 24 Feb 2020
Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss Lénaïc Chizat Francis R. Bach MLT 114 339 0 11 Feb 2020
Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey F. Sultana Abu Sufian P. Dutta SSeg 62 256 0 13 Jan 2020
AI Benchmark: All About Deep Learning on Smartphones in 2019 Andrey D. Ignatov Radu Timofte Andrei Kulik Seungsoo Yang Ke Wang Felix Baum Max Wu Lirong Xu Luc Van Gool ELM 43 220 0 15 Oct 2019
Autonomous Navigation via Deep Reinforcement Learning for Resource Constraint Edge Nodes using Transfer Learning Aqeel Anwar A. Raychowdhury 55 73 0 12 Oct 2019
Survey on Deep Neural Networks in Speech and Vision Systems M. Alam Manar D. Samad Lasitha Vidyaratne Alexander M. Glandon Khan M. Iftekharuddin 3DV VLM AI4TS 68 210 0 16 Aug 2019
Similarity-Preserving Knowledge Distillation Frederick Tung Greg Mori 118 977 0 23 Jul 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 137 18,115 0 28 May 2019
Searching for MobileNetV3 Andrew G. Howard Mark Sandler Grace Chu Liang-Chieh Chen Bo Chen ... Yukun Zhu Ruoming Pang Vijay Vasudevan Quoc V. Le Hartwig Adam 345 6,778 0 06 May 2019
Benchmark Analysis of Representative Deep Neural Network Architectures Simone Bianco Rémi Cadène Luigi Celona Paolo Napoletano BDL 66 677 0 01 Oct 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 225 3,463 0 09 Mar 2018
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 310 6,672 0 08 Jun 2015
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 303 3,883 0 19 Dec 2014
Compressing Deep Convolutional Networks using Vector Quantization Yunchao Gong Liu Liu Ming Yang Lubomir D. Bourdev MQ 155 1,170 0 18 Dec 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 413 43,638 0 01 May 2014