The Tunnel Effect: Building Data Representations in Deep Neural Networks

31 May 2023

Papers citing "The Tunnel Effect: Building Data Representations in Deep Neural Networks"

20 / 20 papers shown

Title
Register and CLS tokens yield a decoupling of local and global features in large ViTs Alexander Lappe M. Giese 24 0 0 09 May 2025
Intermediate Layer Classifiers for OOD generalization Arnas Uselis Seong Joon Oh OOD 53 0 0 07 Apr 2025
Parameter Symmetry Breaking and Restoration Determines the Hierarchical Learning in AI Systems Liu Ziyin Yizhou Xu T. Poggio Isaac Chuang 50 4 0 07 Feb 2025
Understanding How Nonlinear Layers Create Linearly Separable Features for Low-Dimensional Data Alec S. Xu Can Yaras Peng Wang Q. Qu 30 0 0 04 Jan 2025
Parseval Regularization for Continual Reinforcement Learning Wesley Chung Lynn Cherif D. Meger Doina Precup CLL 78 2 0 10 Dec 2024
Layer Pruning with Consensus: A Triple-Win Solution Leandro Giusti Mugnaini Carolina Tavares Duarte Anna H. Reali Costa Artur Jordao 73 0 0 21 Nov 2024
Towards Optimal Adapter Placement for Efficient Transfer Learning Aleksandra I. Nowak Otniel-Bogdan Mercea Anurag Arnab Jonas Pfeiffer Yann N. Dauphin Utku Evci 25 0 0 21 Oct 2024
Formation of Representations in Neural Networks Liu Ziyin Isaac Chuang Tomer Galanti T. Poggio 37 4 0 03 Oct 2024
DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations Guogang Zhu Xuefeng Liu Jianwei Niu Shaojie Tang Xinghao Wu Jiayuan Zhang AI4CE 52 1 0 25 Jul 2024
A Global Geometric Analysis of Maximal Coding Rate Reduction Peng Wang Huikang Liu Druv Pai Yaodong Yu Zhihui Zhu Q. Qu Yi Ma 34 6 0 04 Jun 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu V. Papyan 48 12 0 28 May 2024
Effective Layer Pruning Through Similarity Metric Perspective Ian Pons Bruno Yamamoto Anna H. Reali Costa Artur Jordao 46 2 0 27 May 2024
What Variables Affect Out-Of-Distribution Generalization in Pretrained Models? Md Yousuf Harun Kyungbok Lee Jhair Gallardo Giri Krishnan Christopher Kanan 39 3 0 23 May 2024
Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal? Peter Súkeník Marco Mondelli Christoph H. Lampert AI4CE 34 7 0 23 May 2024
Neural Collapse Meets Differential Privacy: Curious Behaviors of NoisyGD with Near-perfect Representation Learning Chendi Wang Yuqing Zhu Weijie J. Su Yu-Xiang Wang AAML 58 4 0 14 May 2024
End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training Keitaro Sakamoto Issei Sato 24 4 0 14 Feb 2024
Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation Sebastian Lee Stefano Sarao Mannelli Claudia Clopath Sebastian Goldt Andrew M. Saxe CLL 50 12 0 18 May 2022
Architecture Matters in Continual Learning Seyed Iman Mirzadeh Arslan Chaudhry Dong Yin Timothy Nguyen Razvan Pascanu Dilan Görür Mehrdad Farajtabar OOD KELM 116 58 0 01 Feb 2022
Adversarial Continual Learning Sayna Ebrahimi Franziska Meier Roberto Calandra Trevor Darrell Marcus Rohrbach CLL VLM 152 198 0 21 Mar 2020
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 159 234 0 04 Mar 2020