Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks

16 September 2017

Xianglong Liu

Papers citing "Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks"

44 / 44 papers shown

Title
Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities Rahul Parhi Michael Unser 47 5 0 05 Oct 2023
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold Jun Chen Haishan Ye Mengmeng Wang Tianxin Huang Guangwen Dai Ivor W.Tsang Yong-Jin Liu 29 11 0 21 Aug 2023
Group Orthogonalization Regularization For Vision Models Adaptation and Robustness Yoav Kurtz Noga Bar Raja Giryes 32 0 0 16 Jun 2023
A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold Yanhong Fei Xian Wei Yingjie Liu Zhengyu Li Mingsong Chen 28 6 0 16 Feb 2023
Decentralized Riemannian Algorithm for Nonconvex Minimax Problems Xidong Wu Zhengmian Hu Heng Huang 27 15 0 08 Feb 2023
Auto-Encoding Goodness of Fit A. Palmer Zhiyi Chi Derek Aguiar J. Bi 46 1 0 12 Oct 2022
AdaWCT: Adaptive Whitening and Coloring Style Injection Antoine Dufour Yohan Poirier-Ginter Alexandre Lessard Ryan Smith Michael Lockyer Jean-François Lalonde 32 1 0 01 Aug 2022
Entangled Residual Mappings Mathias Lechner Ramin Hasani Z. Babaiee Radu Grosu Daniela Rus T. Henzinger Sepp Hochreiter 14 5 0 02 Jun 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 28 6 0 12 May 2022
Riemannian Hamiltonian methods for min-max optimization on manifolds Andi Han Bamdev Mishra Pratik Jawanpuria Pawan Kumar Junbin Gao 38 17 0 25 Apr 2022
Approximation of Lipschitz Functions using Deep Spline Neural Networks Sebastian Neumayer Alexis Goujon Pakshal Bohra M. Unser 37 16 0 13 Apr 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 41 37 0 12 Mar 2022
Learning Connectivity with Graph Convolutional Networks for Skeleton-based Action Recognition H. Sahbi GNN 38 27 0 06 Dec 2021
Orthogonal Graph Neural Networks Kai Guo Kaixiong Zhou Xia Hu Yu Li Yi Chang Xin Wang 48 34 0 23 Sep 2021
Existence, Stability and Scalability of Orthogonal Convolutional Neural Networks El Mehdi Achour Franccois Malgouyres Franck Mamalet 16 20 0 12 Aug 2021
Coordinate descent on the orthogonal group for recurrent neural network training E. Massart V. Abrol 37 10 0 30 Jul 2021
R-Drop: Regularized Dropout for Neural Networks Xiaobo Liang Lijun Wu Juntao Li Yue Wang Qi Meng Tao Qin Wei Chen Hao Fei Tie-Yan Liu 47 424 0 28 Jun 2021
Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity Ryan Henderson Djork-Arné Clevert F. Montanari 33 26 0 11 May 2021
Skeleton-based Hand-Gesture Recognition with Lightweight Graph Convolutional Networks H. Sahbi 3DH GNN 18 3 0 09 Apr 2021
Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse YuXiang Liu Jidong Ge Chuanyi Li Jie Gui 21 2 0 22 Mar 2021
Learning with Hyperspherical Uniformity Weiyang Liu Rongmei Lin Zhen Liu Li Xiong Bernhard Schölkopf Adrian Weller 37 35 0 02 Mar 2021
Improving Unsupervised Domain Adaptation by Reducing Bi-level Feature Redundancy Mengzhu Wang Xiang Zhang L. Lan Wei Wang Huibin Tan Zhigang Luo AI4CE 39 1 0 28 Dec 2020
Efficient On-Chip Learning for Optical Neural Networks Through Power-Aware Sparse Zeroth-Order Optimization Jiaqi Gu Chenghao Feng Zheng Zhao Zhoufeng Ying Ray T. Chen David Z. Pan 27 28 0 21 Dec 2020
Convolutional Proximal Neural Networks and Plug-and-Play Algorithms J. Hertrich Sebastian Neumayer Gabriele Steidl 19 57 0 04 Nov 2020
Continual Learning in Low-rank Orthogonal Subspaces Arslan Chaudhry Naeemullah Khan P. Dokania Philip Torr CLL 33 115 0 22 Oct 2020
Deep Isometric Learning for Visual Recognition Haozhi Qi Chong You Xueliang Wang Yi Ma Jitendra Malik VLM 35 54 0 30 Jun 2020
OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer Xiaoxu Li Dongliang Chang Zhanyu Ma Zheng-Hua Tan Jing-Hao Xue Jie Cao Jingyi Yu Jun Guo 28 31 0 20 Apr 2020
Orthogonal Over-Parameterized Training Weiyang Liu Rongmei Lin Zhen Liu James M. Rehg Liam Paull Li Xiong Le Song Adrian Weller 32 41 0 09 Apr 2020
Concept Whitening for Interpretable Image Recognition Zhi Chen Yijie Bei Cynthia Rudin FAtt 31 314 0 05 Feb 2020
Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform Jun Li Fuxin Li S. Todorovic 24 102 0 04 Feb 2020
Orthogonal Wasserstein GANs J. Müller Reinhard Klein Michael Weinmann 48 9 0 29 Nov 2019
On Generalization Bounds of a Family of Recurrent Neural Networks Minshuo Chen Xingguo Li T. Zhao 19 70 0 28 Oct 2019
Attentive Normalization Xilai Li Wei Sun Tianfu Wu OOD ViT 28 31 0 04 Aug 2019
ABD-Net: Attentive but Diverse Person Re-Identification Tianlong Chen Shaojin Ding Jingyi Xie Ye Yuan Wuyang Chen Yang Yang Zhou Ren Zhangyang Wang 27 478 0 03 Aug 2019
OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization Bingchen Liu Yizhe Zhu Zuohui Fu Gerard de Melo Ahmed Elgammal CML 9 41 0 26 May 2019
Cascaded Projection: End-to-End Network Compression and Acceleration Breton L. Minnehan Andreas E. Savakis 18 26 0 12 Mar 2019
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group Mario Lezcano Casado David Martínez-Rubio 27 194 0 24 Jan 2019
Hamiltonian Monte-Carlo for Orthogonal Matrices V. Yanush D. Kropotov 19 1 0 23 Jan 2019
McTorch, a manifold optimization library for deep learning Mayank Meghwanshi Pratik Jawanpuria Anoop Kunchukuttan Hiroyuki Kasai Bamdev Mishra AI4CE 28 41 0 03 Oct 2018
Deep Asymmetric Networks with a Set of Node-wise Variant Activation Functions Jinhyeok Jang Hyunjoong Cho Jaehong Kim Jaeyeon Lee Seungjoon Yang 18 2 0 11 Sep 2018
On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyond Xingguo Li Junwei Lu Zhaoran Wang Jarvis Haupt T. Zhao 27 78 0 13 Jun 2018
geomstats: a Python Package for Riemannian Geometry in Machine Learning Nina Miolane Johan Mathe Claire Donnat Mikael Jorda Xavier Pennec AI4CE 37 123 0 21 May 2018
Decorrelated Batch Normalization Lei Huang Dawei Yang B. Lang Jia Deng 16 190 0 23 Apr 2018
Optimization on Submanifolds of Convolution Kernels in CNNs Mete Ozay Takayuki Okatani 51 46 0 22 Oct 2016