Prototype Augmented Hypernetworks for Continual Learning

Continual learning (CL) aims to learn a sequence of tasks without forgetting prior knowledge, but gradient updates for a new task often overwrite the weights learned earlier, causing catastrophic forgetting (CF). We propose Prototype-Augmented Hypernetworks (PAH), a framework where a single hypernetwork, conditioned on learnable task prototypes, dynamically generates task-specific classifier heads on demand. To mitigate forgetting, PAH combines cross-entropy with dual distillation losses, one to align logits and another to align prototypes, ensuring stable feature representations across tasks. Evaluations on Split-CIFAR100 and TinyImageNet demonstrate that PAH achieves state-of-the-art performance, reaching 74.5 % and 63.7 % accuracy with only 1.7 % and 4.4 % forgetting, respectively, surpassing prior methods without storing samples or heads.

@article{fuente2025_2505.07450,
  title={ Prototype Augmented Hypernetworks for Continual Learning },
  author={ Neil De La Fuente and Maria Pilligua and Daniel Vidal and Albin Soutiff and Cecilia Curreli and Daniel Cremers and Andrey Barsky },
  journal={arXiv preprint arXiv:2505.07450},
  year={ 2025 }
}