Visualizing the Loss Landscape of Neural Nets

28 December 2017

Hao Li

Papers citing "Visualizing the Loss Landscape of Neural Nets"

50 / 1,039 papers shown

Title
Friendly Sharpness-Aware Minimization Tao Li Pan Zhou Zhengbao He Xinwen Cheng Xiaolin Huang AAML 54 15 0 19 Mar 2024
Towards Faster Training of Diffusion Models: An Inspiration of A Consistency Phenomenon Tianshuo Xu Peng Mi Ruilin Wang Yingcong Chen DiffM 46 6 0 14 Mar 2024
Unveiling the Significance of Toddler-Inspired Reward Transition in Goal-Oriented Reinforcement Learning Junseok Park Yoonsung Kim Hee Bin Yoo Min Whoo Lee Kibeom Kim Won-Seok Choi Minsu Lee Byoung-Tak Zhang OffRL 43 1 0 11 Mar 2024
PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor Jaewon Jung Hongsun Jang Jaeyong Song Jinho Lee OOD AAML 176 4 0 11 Mar 2024
Improve Generalization Ability of Deep Wide Residual Network with A Suitable Scaling Factor Songtao Tian Zixiong Yu 23 1 0 07 Mar 2024
T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformers Mariano V. Ntrougkas Nikolaos Gkalelis Vasileios Mezaris FAtt ViT 33 5 0 07 Mar 2024
On a Neural Implementation of Brenier's Polar Factorization Nina Vesseron Marco Cuturi 45 2 0 05 Mar 2024
Sensitivity Analysis On Loss Landscape Salman Faroz 21 0 0 02 Mar 2024
Merging Text Transformer Models from Different Initializations Neha Verma Maha Elbayad MoMe 59 7 0 01 Mar 2024
Beyond Single-Model Views for Deep Learning: Optimization versus Generalizability of Stochastic Optimization Algorithms Toki Tahmid Inan Mingrui Liu Amarda Shehu 32 0 0 01 Mar 2024
Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes Xiaomeng Hu Pin-Yu Chen Tsung-Yi Ho AAML 26 26 0 01 Mar 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 44 1 0 01 Mar 2024
Improving Group Connectivity for Generalization of Federated Deep Learning Zexi Li Jie Lin Zhiqi Li Didi Zhu Chao Wu AI4CE FedML 43 0 0 29 Feb 2024
Gradient Alignment for Cross-Domain Face Anti-Spoofing B. Le Simon S. Woo CVBM 43 18 0 29 Feb 2024
Helen: Optimizing CTR Prediction Models with Frequency-wise Hessian Eigenvalue Regularization Zirui Zhu Yong Liu Zangwei Zheng Huifeng Guo Yang You 35 0 0 23 Feb 2024
Investigating the Histogram Loss in Regression Ehsan Imani Kai Luedemann Sam Scholnick-Hughes Esraa Elelimy Martha White UQCV 34 5 0 20 Feb 2024
Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima Shan Zhong Zhongzhan Huang Daifeng Li Wushao Wen Jinghui Qin Liang Lin 22 12 0 17 Feb 2024
Bridging the Empirical-Theoretical Gap in Neural Network Formal Language Learning Using Minimum Description Length N. Lan Emmanuel Chemla Roni Katzir 15 2 0 15 Feb 2024
Switch EMA: A Free Lunch for Better Flatness and Sharpness Siyuan Li Zicheng Liu Juanxi Tian Ge Wang Zedong Wang ... Cheng Tan Tao Lin Yang Liu Baigui Sun Stan Z. Li 30 6 0 14 Feb 2024
On Differentially Private Subspace Estimation in a Distribution-Free Setting Eliad Tsfadia 25 1 0 09 Feb 2024
Tradeoffs of Diagonal Fisher Information Matrix Estimators Alexander Soen Ke Sun 19 1 0 08 Feb 2024
Strong convexity-guided hyper-parameter optimization for flatter losses Rahul Yedida Snehanshu Saha 24 0 0 07 Feb 2024
Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time Sungyoon Kim Mert Pilanci 48 4 0 06 Feb 2024
Careful with that Scalpel: Improving Gradient Surgery with an EMA Yu-Guan Hsieh James Thornton Eugène Ndiaye Michal Klein Marco Cuturi Pierre Ablin MedIm 39 0 0 05 Feb 2024
Balanced Resonate-and-Fire Neurons Saya Higuchi Sebastian Kairat S. Bohté Sebastian Otte 24 6 0 02 Feb 2024
Training-time Neuron Alignment through Permutation Subspace for Improving Linear Mode Connectivity and Model Fusion Zexi Li Zhiqi Li Jie Lin Tao Shen Tao Lin Chao Wu 41 4 0 02 Feb 2024
LTAU-FF: Loss Trajectory Analysis for Uncertainty in Atomistic Force Fields Joshua A. Vita Amit Samanta Fei Zhou Vincenzo Lordi 25 2 0 01 Feb 2024
EPSD: Early Pruning with Self-Distillation for Efficient Model Compression Dong Chen Ning Liu Yichen Zhu Zhengping Che Rui Ma Fachao Zhang Xiaofeng Mou Yi Chang Jian Tang 31 3 0 31 Jan 2024
Towards Assessing the Synthetic-to-Measured Adversarial Vulnerability of SAR ATR Bowen Peng Bo Peng Jingyuan Xia Tianpeng Liu Yongxiang Liu Li Liu AAML 32 4 0 30 Jan 2024
Speeding up and reducing memory usage for scientific machine learning via mixed precision Joel Hayford Jacob Goldman-Wetzler Eric Wang Lu Lu 49 8 0 30 Jan 2024
Accelerating superconductor discovery through tempered deep learning of the electron-phonon spectral function Jason B. Gibson A. Hire P. M. Dee Oscar Barrera Benjamin Geisler P. Hirschfeld R. G. Hennig 13 4 0 29 Jan 2024
Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators Yaniv Blumenfeld Itay Hubara Daniel Soudry 39 3 0 25 Jan 2024
Towards Effective and General Graph Unlearning via Mutual Evolution Xunkai Li Yulin Zhao Zhengyu Wu Wentao Zhang Ronghua Li Guoren Wang MU 33 14 0 22 Jan 2024
Momentum-SAM: Sharpness Aware Minimization without Computational Overhead Marlon Becker Frederick Altrock Benjamin Risse 79 5 0 22 Jan 2024
Understanding the Generalization Benefits of Late Learning Rate Decay Yinuo Ren Chao Ma Lexing Ying AI4CE 32 6 0 21 Jan 2024
Bag of Tricks to Boost Adversarial Transferability Zeliang Zhang Rongyi Zhu Wei Yao Xiaosen Wang Chenliang Xu AAML 47 9 0 16 Jan 2024
A topological description of loss surfaces based on Betti Numbers Maria Sofia Bucarelli Giuseppe Alessio D’Inverno Monica Bianchini F. Scarselli Fabrizio Silvestri 27 1 0 08 Jan 2024
Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspective Sunwoong Yang Hojin Kim Y. Hong K. Yee R. Maulik Namwoo Kang PINN AI4CE 28 17 0 05 Jan 2024
$f$ -Divergence Based Classification: Beyond the Use of Cross-Entropy Nicola Novello Andrea M. Tonello 22 7 0 02 Jan 2024
On the Necessity of Metalearning: Learning Suitable Parameterizations for Learning Processes Massinissa Hamidi A. Osmani 35 0 0 31 Dec 2023
Universal Pyramid Adversarial Training for Improved ViT Performance Ping Yeh-Chiang Yipin Zhou Omid Poursaeed S. Narayan Shukla Tom Goldstein Ser-Nam Lim AAML ViT 16 0 0 26 Dec 2023
CR-SAM: Curvature Regularized Sharpness-Aware Minimization Tao Wu Tie Luo D. C. Wunsch 18 3 0 21 Dec 2023
Enhancing Neural Training via a Correlated Dynamics Model Jonathan Brokman Roy Betser Rotem Turjeman Tom Berkov I. Cohen Guy Gilboa 24 3 0 20 Dec 2023
Sparse is Enough in Fine-tuning Pre-trained Large Language Models Weixi Song Z. Li Lefei Zhang Hai Zhao Bo Du VLM 23 7 0 19 Dec 2023
NAC-TCN: Temporal Convolutional Networks with Causal Dilated Neighborhood Attention for Emotion Understanding Alexander Mehta William Yang ViT 44 0 0 12 Dec 2023
Continual Learning through Networks Splitting and Merging with Dreaming-Meta-Weighted Model Fusion Yi Sun Xin Xu Jian Li Guanglei Xie Yifei Shi Qiang Fang CLL MoMe 31 1 0 12 Dec 2023
Measurement-driven neural-network training for integrated magnetic tunnel junction arrays W. A. Borders A. Madhavan M. Daniels Vasileia Georgiou Martin Lueker-Boden Tiffany S. Santos Patrick M. Braganca M. D. Stiles Jabez J. McClelland Brian D. Hoskins 27 3 0 11 Dec 2023
MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness Xiaoyun Xu Shujian Yu Jingzheng Wu S. Picek AAML 35 0 0 08 Dec 2023
On The Fairness Impacts of Hardware Selection in Machine Learning Sree Harsha Nelaturu Nishaanth Kanna Ravichandran Cuong Tran Sara Hooker Ferdinando Fioretto 53 2 0 06 Dec 2023
Generalisable Agents for Neural Network Optimisation Kale-ab Tessera C. Tilbury Sasha Abramowitz Ruan de Kock Omayma Mahjoub Benjamin Rosman Sara Hooker Arnu Pretorius AI4CE 20 0 0 30 Nov 2023