Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?

22 October 2018

Papers citing "Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?"

37 / 37 papers shown

Title
Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold S. Schechtman D. Tiapkin Michael Muehlebach Eric Moulines 27 6 0 16 Mar 2023
A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold Yanhong Fei Xian Wei Yingjie Liu Zhengyu Li Mingsong Chen 28 6 0 16 Feb 2023
Look Beyond Bias with Entropic Adversarial Data Augmentation Thomas Duboudin Emmanuel Dellandrea Corentin Abgrall Gilles Hénaff Liming Chen CML 35 4 0 10 Jan 2023
Orthogonal SVD Covariance Conditioning and Latent Disentanglement Yue Song N. Sebe Wei Wang 26 6 0 11 Dec 2022
Batch Normalization Explained Randall Balestriero Richard G. Baraniuk AAML 33 16 0 29 Sep 2022
Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality Yue Song N. Sebe Wei Wang 19 8 0 05 Jul 2022
Maximum Class Separation as Inductive Bias in One Matrix Tejaswi Kasarla Gertjan J. Burghouts Max van Spengler Elise van der Pol Rita Cucchiara Pascal Mettes 26 22 0 17 Jun 2022
Entangled Residual Mappings Mathias Lechner Ramin Hasani Z. Babaiee Radu Grosu Daniela Rus T. Henzinger Sepp Hochreiter 11 4 0 02 Jun 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 28 6 0 12 May 2022
Riemannian Hamiltonian methods for min-max optimization on manifolds Andi Han Bamdev Mishra Pratik Jawanpuria Pawan Kumar Junbin Gao 35 17 0 25 Apr 2022
Learning Decoupling Features Through Orthogonality Regularization Li Wang Rongzhi Gu Weiji Zhuang Peng Gao Yujun Wang Yuexian Zou 22 1 0 31 Mar 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu (Allen) Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 41 37 0 12 Mar 2022
projUNN: efficient method for training deep networks with unitary matrices B. Kiani Randall Balestriero Yann LeCun S. Lloyd 43 32 0 10 Mar 2022
Exploiting a Zoo of Checkpoints for Unseen Tasks Jiaji Huang Qiang Qiu Kenneth Ward Church 27 4 0 05 Nov 2021
Orthogonal Graph Neural Networks Kai Guo Kaixiong Zhou Xia Hu Yu Li Yi Chang Xin Wang 43 34 0 23 Sep 2021
Acceleration Method for Learning Fine-Layered Optical Neural Networks K. Aoyama H. Sawada 27 1 0 01 Sep 2021
Existence, Stability and Scalability of Orthogonal Convolutional Neural Networks E. M. Achour Franccois Malgouyres Franck Mamalet 16 20 0 12 Aug 2021
Coordinate descent on the orthogonal group for recurrent neural network training E. Massart V. Abrol 34 10 0 30 Jul 2021
Taxonomy of Machine Learning Safety: A Survey and Primer Sina Mohseni Haotao Wang Zhiding Yu Chaowei Xiao Zhangyang Wang J. Yadawa 21 31 0 09 Jun 2021
Dual Metric Learning for Effective and Efficient Cross-Domain Recommendations Pan Li Alexander Tuzhilin 19 50 0 17 Apr 2021
Self-Domain Adaptation for Face Anti-Spoofing Jingjing Wang Jingyi Zhang Ying Bian Youyi Cai Chunmao Wang Shiliang Pu OOD TTA 27 95 0 24 Feb 2021
Self domain adapted network Yufan He A. Carass Lianrui Zuo Blake E Dewey Jerry L. Prince OOD 17 29 0 07 Jul 2020
Deep Isometric Learning for Visual Recognition Haozhi Qi Chong You Xinyu Wang Yi Ma Jitendra Malik VLM 30 53 0 30 Jun 2020
Orthogonal Over-Parameterized Training Weiyang Liu Rongmei Lin Zhen Liu James M. Rehg Liam Paull Li Xiong Le Song Adrian Weller 32 41 0 09 Apr 2020
Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform Jun Li Fuxin Li S. Todorovic 24 102 0 04 Feb 2020
Orthogonal Wasserstein GANs J. Müller Reinhard Klein Michael Weinmann 48 9 0 29 Nov 2019
Learning to Optimize in Swarms Yue Cao Tianlong Chen Zhangyang Wang Yang Shen 20 55 0 09 Nov 2019
HUBERT Untangles BERT to Improve Transfer across NLP Tasks M. Moradshahi Hamid Palangi M. Lam P. Smolensky Jianfeng Gao 23 16 0 25 Oct 2019
ABD-Net: Attentive but Diverse Person Re-Identification Tianlong Chen Shaojin Ding Jingyi Xie Ye Yuan Wuyang Chen Yang Yang Zhou Ren Zhangyang Wang 27 479 0 03 Aug 2019
Orthogonal Deep Neural Networks Kui Jia Shuai Li Yuxin Wen Tongliang Liu Dacheng Tao 34 132 0 15 May 2019
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers Ernest K. Ryu Jialin Liu Sicheng Wang Xiaohan Chen Zhangyang Wang W. Yin AI4CE 22 347 0 14 May 2019
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group Mario Lezcano Casado David Martínez-Rubio 21 193 0 24 Jan 2019
Deep Asymmetric Networks with a Set of Node-wise Variant Activation Functions Jinhyeok Jang Hyunjoong Cho Jaehong Kim Jaeyeon Lee Seungjoon Yang 18 2 0 11 Sep 2018
NEU: A Meta-Algorithm for Universal UAP-Invariant Feature Representation Anastasis Kratsios Cody B. Hyndman OOD 28 17 0 31 Aug 2018
Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding Dong Liu Ke Sun Zhangyang Wang Runsheng Liu Zhengjun Zha 24 12 0 28 Feb 2018
Optimization on Submanifolds of Convolution Kernels in CNNs Mete Ozay Takayuki Okatani 51 46 0 22 Oct 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016