Distilling the Knowledge in a Neural Network

9 March 2015

Papers citing "Distilling the Knowledge in a Neural Network"

50 / 327 papers shown

Title
Distilling Long-tailed Datasets Zhenghao Zhao Haoxuan Wang Yuzhang Shang Kai Wang Yan Yan DD 67 3 0 24 Aug 2024
The Evolution of Reinforcement Learning in Quantitative Finance: A Survey Nikolaos Pippas Cagatay Turkay Elliot A. Ludvig AIFin 129 3 0 20 Aug 2024
Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models Aviv Bick Kevin Y. Li Eric P. Xing J. Zico Kolter Albert Gu Mamba 78 27 0 19 Aug 2024
MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU Yan Li So-Eon Kim Seong-Bae Park S. Han 56 1 0 15 Aug 2024
FedQUIT: On-Device Federated Unlearning via a Quasi-Competent Virtual Teacher Alessio Mora Lorenzo Valerio Paolo Bellavista A. Passarella FedML MU 73 2 0 14 Aug 2024
PEARL: Parallel Speculative Decoding with Adaptive Draft Length Tianyu Liu Yun Li Qitan Lv Kai Liu Jianchen Zhu Winston Hu Xingwu Sun 79 15 0 13 Aug 2024
A Review of Pseudo-Labeling for Computer Vision Patrick Kage Jay C. Rothenberger Pavlos Andreadis Dimitrios I. Diochnos VLM 73 5 0 13 Aug 2024
Content-decoupled Contrastive Learning-based Implicit Degradation Modeling for Blind Image Super-Resolution Jiang Yuan Ji Ma Bo Wang Weiming Hu 63 0 0 10 Aug 2024
Overcoming Uncertain Incompleteness for Robust Multimodal Sequential Diagnosis Prediction via Curriculum Data Erasing Guided Knowledge Distillation Heejoon Koo 78 0 0 28 Jul 2024
Accelerating the Low-Rank Decomposed Models Habib Hajimolahoseini Walid Ahmed Austin Wen Yang Liu 64 0 0 24 Jul 2024
Continual Distillation Learning: Knowledge Distillation in Prompt-based Continual Learning Qifan Zhang Yunhui Guo Yu Xiang CLL VLM 76 0 0 18 Jul 2024
Relational Representation Distillation Nikolaos Giakoumoglou Tania Stathaki 55 0 0 16 Jul 2024
Partner in Crime: Boosting Targeted Poisoning Attacks against Federated Learning Shihua Sun Shridatt Sugrim Angelos Stavrou Haining Wang AAML 88 1 0 13 Jul 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 69 49 0 09 Jul 2024
Gaussian Eigen Models for Human Heads Wojciech Zielonka Timo Bolkart Thabo Beeler Justus Thies 3DGS 75 5 0 05 Jul 2024
Direct Preference Knowledge Distillation for Large Language Models Yixing Li Yuxian Gu Li Dong Dequan Wang Yu Cheng Furu Wei 62 6 0 28 Jun 2024
From Distributional to Overton Pluralism: Investigating Large Language Model Alignment Thom Lake Eunsol Choi Greg Durrett 70 9 0 25 Jun 2024
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging Deyuan Liu Zhan Qin Han Wang Zhao Yang Zecheng Wang ... Zhao Lv Zhiying Tu Dianhui Chu Bo Li Dianbo Sui 48 2 0 24 Jun 2024
From Instance Training to Instruction Learning: Task Adapters Generation from Instructions Huanxuan Liao Yao Xu Shizhu He Yuanzhe Zhang Yanchao Hao Shengping Liu Kang Liu Jun Zhao 92 1 0 18 Jun 2024
Retraining with Predicted Hard Labels Provably Increases Model Accuracy Rudrajit Das Inderjit S Dhillon Alessandro Epasto Adel Javanmard Jieming Mao Vahab Mirrokni Sujay Sanghavi Peilin Zhong 71 2 0 17 Jun 2024
Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Models Alireza Ganjdanesh Reza Shirkavand Shangqian Gao Heng Huang DiffM VLM 96 4 0 17 Jun 2024
A Label is Worth a Thousand Images in Dataset Distillation Tian Qin Zhiwei Deng David Alvarez-Melis DD 140 12 0 15 Jun 2024
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications Jordy Van Landeghem Subhajit Maity Ayan Banerjee Matthew Blaschko Marie-Francine Moens Josep Lladós Sanket Biswas 83 2 0 12 Jun 2024
GENIE: Watermarking Graph Neural Networks for Link Prediction Venkata Sai Pranav Bachina Ankit Gangwal Aaryan Ajay Sharma Charu Sharma 69 1 0 07 Jun 2024
ReDistill: Residual Encoded Distillation for Peak Memory Reduction of CNNs Fang Chen Gourav Datta Mujahid Al Rafi Hyeran Jeon Meng Tang 123 1 0 06 Jun 2024
OCCAM: Towards Cost-Efficient and Accuracy-Aware Classification Inference Dujian Ding Bicheng Xu L. Lakshmanan VLM 75 1 0 06 Jun 2024
Distilling Aggregated Knowledge for Weakly-Supervised Video Anomaly Detection Jash Dalvi Ali Dabouei Gunjan Dhanuka Min Xu 43 0 0 05 Jun 2024
Tiny models from tiny data: Textual and null-text inversion for few-shot distillation Erik Landolsi Fredrik Kahl DiffM 76 1 0 05 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 142 4 0 05 Jun 2024
Harnessing Increased Client Participation with Cohort-Parallel Federated Learning Akash Dhasade Anne-Marie Kermarrec Tuan-Anh Nguyen Rafael Pires M. Vos FedML 79 0 0 24 May 2024
Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detection Jia Guo Shuai Lu Weihang Zhang Huiqi Li Huiqi Li Hongen Liao ViT 99 10 0 23 May 2024
Combining Relevance and Magnitude for Resource-Aware DNN Pruning C. Chiasserini F. Malandrino Nuria Molner Zhiqiang Zhao 52 0 0 21 May 2024
Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors Chun-Yin Huang Kartik Srinivas Xin Zhang Xiaoxiao Li DD 91 6 0 19 May 2024
Multicenter Privacy-Preserving Model Training for Deep Learning Brain Metastases Autosegmentation Yixing Huang Zahra Khodabakhshi A. Gomaa Manuel Schmidt R. Fietkau Matthias Guckenberger N. Andratschke Christoph Bert S. Tanadini-Lang F. Putz 51 5 0 17 May 2024
S3Former: Self-supervised High-resolution Transformer for Solar PV Profiling Minh-Triet Tran Adrian de Luis Haitao Liao Ying Huang Roy McCann Alan Mantooth Jack Cothren Ngan Le 157 0 0 07 May 2024
ATOM: Attention Mixer for Efficient Dataset Distillation Samir Khaki A. Sajedi Kai Wang Lucy Z. Liu Y. Lawryshyn Konstantinos N. Plataniotis 100 3 0 02 May 2024
CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective Wencheng Zhu Xin Zhou Pengfei Zhu Yu Wang Qinghua Hu VLM 86 1 0 22 Apr 2024
LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Models Dingkun Zhang Sijia Li Chen Chen Qingsong Xie H. Lu 56 25 0 17 Apr 2024
Faster Diffusion via Temporal Attention Decomposition Haozhe Liu Wentian Zhang Jinheng Xie Francesco Faccio Mengmeng Xu Tao Xiang Mike Zheng Shou Juan-Manuel Perez-Rua Jürgen Schmidhuber DiffM 102 21 0 03 Apr 2024
TSCM: A Teacher-Student Model for Vision Place Recognition Using Cross-Metric Knowledge Distillation Yehui Shen Mingmin Liu Huimin Lu Xieyuanli Chen 51 1 0 02 Apr 2024
The Unreasonable Ineffectiveness of the Deeper Layers Andrey Gromov Kushal Tirumala Hassan Shapourian Paolo Glorioso Daniel A. Roberts 71 93 0 26 Mar 2024
Improving Forward Compatibility in Class Incremental Learning by Increasing Representation Rank and Feature Richness Jaeill Kim Wonseok Lee Moonjung Eo Wonjong Rhee CLL 80 0 0 22 Mar 2024
REAL: Representation Enhanced Analytic Learning for Exemplar-free Class-incremental Learning Run He Huiping Zhuang Di Fang Yizhu Chen Kai Tong Cen Chen Lap-Pui Chau Huiping Zhuang 86 1 0 20 Mar 2024
Don't Half-listen: Capturing Key-part Information in Continual Instruction Tuning Yongquan He Wenyuan Zhang Xuancheng Huang Peng Zhang Lingxun Meng Wei Lin Wenyuan Zhang Yifu Gao CLL ALM 86 5 0 15 Mar 2024
LIX: Implicitly Infusing Spatial Geometric Prior Knowledge into Visual Semantic Segmentation for Autonomous Driving Sicen Guo Zhiyuan Wu Qijun Chen Ioannis Pitas Rui Fan Rui Fan 69 1 0 13 Mar 2024
Uncertainty in Graph Neural Networks: A Survey Fangxin Wang Yuqing Liu Kay Liu Yibo Wang Sourav Medya Philip S. Yu AI4CE 92 10 0 11 Mar 2024
Attention-guided Feature Distillation for Semantic Segmentation Amir M. Mansourian Arya Jalali Rozhan Ahmadi S. Kasaei 118 0 0 08 Mar 2024
RadarDistill: Boosting Radar-based Object Detection Performance via Knowledge Distillation from LiDAR Features Geonho Bang Kwangjin Choi Jisong Kim Dongsuk Kum Jun Won Choi 56 12 0 08 Mar 2024
Personalized Negative Reservoir for Incremental Learning in Recommender Systems Antonios Valkanas Yuening Wang Yingxue Zhang Mark Coates CLL 60 1 0 06 Mar 2024
A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods Hanlei Jin Yang Zhang Dan Meng Jun Wang Jinghua Tan 144 87 0 05 Mar 2024