Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion

18 December 2019

Papers citing "Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion"

50 / 112 papers shown

Title
$Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization$ Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ ual- $\mathbf{\texttt{H}}$ ead $\mathbf{\texttt{O}}$ ptimization Seongjae Kang Dong Bok Lee Hyungjoon Jang Sung Ju Hwang VLM 58 0 0 12 May 2025
Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possible Chaoyi Lu Yiding Sun Pengbo Li Zhichuan Yang FedML 41 0 0 05 Apr 2025
Forget the Data and Fine-Tuning! Just Fold the Network to Compress Dong Wang Haris Šikić Lothar Thiele O. Saukh 68 0 0 17 Feb 2025
CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling Kaiyuan Zhang Siyuan Cheng Guangyu Shen Bruno Ribeiro Shengwei An Pin-Yu Chen Xinming Zhang Ninghui Li 154 1 0 28 Jan 2025
Semantics Prompting Data-Free Quantization for Low-Bit Vision Transformers Mingliang Xu Yuyao Zhou Yuxin Zhang Shen Li Yong Li Rongrong Ji Zhanpeng Zeng Rongrong Ji MQ 102 0 0 31 Dec 2024
Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models Xiao Cui Mo Zhu Yulei Qin Liang Xie Wengang Zhou Yiming Li 94 4 0 19 Dec 2024
FEDLAD: Federated Evaluation of Deep Leakage Attacks and Defenses Isaac Baglin Xiatian Zhu Simon Hadfield FedML 32 1 0 05 Nov 2024
One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering Deepayan Das Davide Talon Massimiliano Mancini Yiming Wang Elisa Ricci 52 0 0 04 Nov 2024
Data Generation for Hardware-Friendly Post-Training Quantization Lior Dikstein Ariel Lapid Arnon Netzer H. Habi MQ 232 0 0 29 Oct 2024
A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning Jun Bai Yiliao Song Di Wu Atul Sajjanhar Yong Xiang Wei Zhou Xiaohui Tao Yan Li Yuante Li FedML 55 0 0 28 Oct 2024
Few-Shot Class-Incremental Learning with Non-IID Decentralized Data Cuiwei Liu Siang Xu Huaijun Qiu Jing Zhang Zhi Liu Liang Zhao CLL 36 0 0 18 Sep 2024
DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture Qianlong Xiang Miao Zhang Yuzhang Shang Jianlong Wu Yan Yan Liqiang Nie DiffM 66 10 0 05 Sep 2024
MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity Kanghyun Choi Hyeyoon Lee Dain Kwon Sunjong Park Kyuyeun Kim Noseong Park Jinho Lee Jinho Lee MQ 52 1 0 29 Jul 2024
Distilling Vision-Language Foundation Models: A Data-Free Approach via Prompt Diversification Yunyi Xuan Weijie Chen Shicai Yang Di Xie Luojun Lin Yueting Zhuang VLM 40 4 0 21 Jul 2024
A Label is Worth a Thousand Images in Dataset Distillation Tian Qin Zhiwei Deng David Alvarez-Melis DD 94 12 0 15 Jun 2024
Data Reconstruction: When You See It and When You Don't Edith Cohen Haim Kaplan Yishay Mansour Shay Moran Kobbi Nissim Uri Stemmer Eliad Tsfadia AAML 44 2 0 24 May 2024
FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning Yinlin Zhu Xunkai Li Zhengyu Wu Di Wu Miao Hu Ronghua Li FedML 29 6 0 22 Apr 2024
Data-free Knowledge Distillation for Fine-grained Visual Categorization Renrong Shao Wei Zhang Jianhua Yin Jun Wang 39 2 0 18 Apr 2024
Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal Jianheng Huang Leyang Cui Ante Wang Chengyi Yang Xinting Liao Linfeng Song Junfeng Yao Jinsong Su KELM CLL 37 35 0 02 Mar 2024
Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning Chenyang Wang Junjun Jiang Xingyu Hu Xianming Liu Xiangyang Ji 47 3 0 12 Jan 2024
Continual Learning through Networks Splitting and Merging with Dreaming-Meta-Weighted Model Fusion Yi Sun Xin Xu Jian Li Guanglei Xie Yifei Shi Qiang Fang CLL MoMe 36 1 0 12 Dec 2023
FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation Xiang Liu Liangxi Liu Feiyang Ye Yunheng Shen Xia Li Linshan Jiang Jialin Li 36 4 0 30 Sep 2023
Towards Few-Call Model Stealing via Active Self-Paced Knowledge Distillation and Diffusion-Based Image Generation Vlad Hondru Radu Tudor Ionescu DiffM 50 1 0 29 Sep 2023
Feature Matching Data Synthesis for Non-IID Federated Learning Zijian Li Yuchang Sun Jiawei Shao Yuyi Mao Jessie Hui Wang Jun Zhang 30 20 0 09 Aug 2023
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants Yibo Yang Haobo Yuan Hefei Ling Jianlong Wu Lefei Zhang Zhouchen Lin Philip Torr Dacheng Tao Guohao Li CLL 33 8 0 03 Aug 2023
Sampling to Distill: Knowledge Transfer from Open-World Data Yuzheng Wang Zhaoyu Chen Jie M. Zhang Dingkang Yang Zuhao Ge Yang Liu Siao Liu Yunquan Sun Wenqiang Zhang Lizhe Qi 34 9 0 31 Jul 2023
f-Divergence Minimization for Sequence-Level Knowledge Distillation Yuqiao Wen Zichao Li Wenyu Du Lili Mou 34 53 0 27 Jul 2023
Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses G. Buzaglo Niv Haim Gilad Yehudai Gal Vardi Yakir Oz Yaniv Nikankin Michal Irani 34 10 0 04 Jul 2023
DHBE: Data-free Holistic Backdoor Erasing in Deep Neural Networks via Restricted Adversarial Distillation Zhicong Yan Shenghong Li Ruijie Zhao Yuan Tian Yuanyuan Zhao AAML 39 11 0 13 Jun 2023
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models Zechun Liu Barlas Oğuz Changsheng Zhao Ernie Chang Pierre Stock Yashar Mehdad Yangyang Shi Raghuraman Krishnamoorthi Vikas Chandra MQ 60 191 0 29 May 2023
Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation? Zheng Li Yuxuan Li Penghai Zhao Renjie Song Xiang Li Jian Yang 34 19 0 22 May 2023
Lion: Adversarial Distillation of Proprietary Large Language Models Yuxin Jiang Chunkit Chan Mingyang Chen Wei Wang ALM 33 23 0 22 May 2023
Remind of the Past: Incremental Learning with Analogical Prompts Zhiheng Ma Xiaopeng Hong Beinan Liu Yabin Wang Pinyue Guo Huiyun Li CLL 34 1 0 24 Mar 2023
Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning Zixuan Hu Li Shen Zhenyi Wang Tongliang Liu Chun Yuan Dacheng Tao 49 4 0 20 Mar 2023
Feature-Rich Audio Model Inversion for Data-Free Knowledge Distillation Towards General Sound Classification Zuheng Kang Yayun He Jianzong Wang Junqing Peng Xiaoyang Qu Jing Xiao 16 2 0 14 Mar 2023
A Comprehensive Survey on Source-free Domain Adaptation Zhiqi Yu Jingjing Li Zhekai Du Lei Zhu H. Shen TTA 39 96 0 23 Feb 2023
CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models Hossein Hajipour Keno Hassler Thorsten Holz Lea Schonherr Mario Fritz ELM 40 20 0 08 Feb 2023
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning Yibo Yang Haobo Yuan Hefei Ling Zhouchen Lin Philip Torr Dacheng Tao CLL 41 98 0 06 Feb 2023
Task-Adaptive Saliency Guidance for Exemplar-free Class Incremental Learning Xialei Liu Jiang-Tian Zhai Andrew D. Bagdanov Ke Li Mingg-Ming Cheng CLL 33 4 0 16 Dec 2022
What do Vision Transformers Learn? A Visual Exploration Amin Ghiasi Hamid Kazemi Eitan Borgnia Steven Reich Manli Shu Micah Goldblum A. Wilson Tom Goldstein ViT 34 60 0 13 Dec 2022
Expanding Small-Scale Datasets with Guided Imagination Yifan Zhang Daquan Zhou Bryan Hooi Kaixin Wang Jiashi Feng 49 46 0 25 Nov 2022
Video Test-Time Adaptation for Action Recognition Wei Lin M. Jehanzeb Mirza Mateusz Koziñski Horst Possegger Hilde Kuehne Horst Bischof TTA 47 31 0 24 Nov 2022
ConStruct-VL: Data-Free Continual Structured VL Concepts Learning James Smith Paola Cascante-Bonilla Assaf Arbelle Donghyun Kim Yikang Shen David D. Cox Diyi Yang Z. Kira Rogerio Feris Leonid Karlinsky VLM 47 20 0 17 Nov 2022
A Survey on Computer Vision based Human Analysis in the COVID-19 Era Fevziye Irem Eyiokur Alperen Kantarci M. Erakin Naser Damer Ferda Ofli ... Janez Križaj A. Salah Alexander Waibel Vitomir Štruc H. K. Ekenel 39 20 0 07 Nov 2022
Decompiling x86 Deep Neural Network Executables Zhibo Liu Yuanyuan Yuan Shuai Wang Xiaofei Xie Lei Ma AAML 45 13 0 03 Oct 2022
MAGIC: Mask-Guided Image Synthesis by Inverting a Quasi-Robust Classifier Mozhdeh Rouhsedaghat Masoud Monajatipoor C.-C. Jay Kuo I. Masi 45 6 0 23 Sep 2022
Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillation Kien Do Hung Le D. Nguyen Dang Nguyen Haripriya Harikumar T. Tran Santu Rana Svetha Venkatesh 18 32 0 21 Sep 2022
PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers Zhikai Li Mengjuan Chen Junrui Xiao Qingyi Gu ViT MQ 55 33 0 13 Sep 2022
Cocktail Party Attack: Breaking Aggregation-Based Privacy in Federated Learning using Independent Component Analysis Sanjay Kariyappa Chuan Guo Kiwan Maeng Wenjie Xiong G. E. Suh Moinuddin K. Qureshi Hsien-Hsin S. Lee FedML 26 29 0 12 Sep 2022
Dynamic Data-Free Knowledge Distillation by Easy-to-Hard Learning Strategy Jingru Li Sheng Zhou Liangcheng Li Haishuai Wang Zhi Yu Jiajun Bu 39 14 0 29 Aug 2022