Orthogonalizing Convolutional Layers with the Cayley Transform

14 April 2021

J. Zico Kolter

Papers citing "Orthogonalizing Convolutional Layers with the Cayley Transform"

50 / 51 papers shown

Title
Approximation theory for 1-Lipschitz ResNets Davide Murari Takashi Furuya Carola-Bibiane Schönlieb 55 0 0 17 May 2025
Bridging the Theoretical Gap in Randomized Smoothing Blaise Delattre Paul Caillon Quentin Barthélemy Erwan Fagnou Alexandre Allauzen AAML 131 0 0 03 Apr 2025
Fast and scalable Wasserstein-1 neural optimal transport solver for single-cell perturbation prediction Yanshuo Chen Zhengmian Hu Wei Chen Heng Huang OT 82 2 0 01 Nov 2024
Certified Causal Defense with Generalizable Robustness Yiran Qiao Yu Yin Chen Chen Jing Ma AAML OOD CML 137 0 0 28 Aug 2024
On Robust Reinforcement Learning with Lipschitz-Bounded Policy Networks Nicholas H. Barbara Ruigang Wang I. Manchester 82 4 0 19 May 2024
Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularization Mahyar Fazlyab Taha Entesari Aniket Roy Ramalingam Chellappa AAML 59 11 0 29 Sep 2023
Deep Isometric Learning for Visual Recognition Haozhi Qi Chong You Xinyu Wang Yi-An Ma Jitendra Malik VLM 56 55 0 30 Jun 2020
Adversarial Robustness on In- and Out-Distribution Improves Explainability Maximilian Augustin Alexander Meinke Matthias Hein OOD 154 102 0 20 Mar 2020
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks Francesco Croce Matthias Hein AAML 211 1,837 0 03 Mar 2020
Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform Jun Li Fuxin Li S. Todorovic 50 104 0 04 Feb 2020
Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks Wei Hu Lechao Xiao Jeffrey Pennington 57 113 0 16 Jan 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 396 42,299 0 03 Dec 2019
Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networks Qiyang Li Saminul Haque Cem Anil James Lucas Roger C. Grosse Joern-Henrik Jacobsen 80 116 0 03 Nov 2019
Trivializations for Gradient-Based Optimization on Manifolds Mario Lezcano Casado 102 126 0 20 Sep 2019
Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks Mahyar Fazlyab Alexander Robey Hamed Hassani M. Morari George J. Pappas 87 456 0 12 Jun 2019
Certified Adversarial Robustness via Randomized Smoothing Jeremy M. Cohen Elan Rosenfeld J. Zico Kolter AAML 130 2,036 0 08 Feb 2019
Fixup Initialization: Residual Learning Without Normalization Hongyi Zhang Yann N. Dauphin Tengyu Ma ODL AI4CE 85 349 0 27 Jan 2019
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group Mario Lezcano Casado David Martínez-Rubio 46 198 0 24 Jan 2019
Sorting out Lipschitz function approximation Cem Anil James Lucas Roger C. Grosse 86 321 0 13 Nov 2018
Complex Unitary Recurrent Neural Networks using Scaled Cayley Transform K. D. G. Maduranga Kyle E. Helfrich Qiang Ye 50 32 0 09 Nov 2018
Can We Gain More from Orthogonality Regularizations in Training Deep CNNs? Nitin Bansal Xiaohan Chen Zhangyang Wang OOD 79 188 0 22 Oct 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 298 353 0 14 Jun 2018
Scaling provable adversarial defenses Eric Wong Frank R. Schmidt J. H. Metzen J. Zico Kolter AAML 70 446 0 31 May 2018
The Singular Values of Convolutional Layers Hanie Sedghi Vineet Gupta Philip M. Long FAtt 81 202 0 26 May 2018
Regularisation of Neural Networks by Enforcing Lipschitz Continuity Henry Gouk E. Frank Bernhard Pfahringer M. Cree 164 477 0 12 Apr 2018
i-RevNet: Deep Invertible Networks J. Jacobsen A. Smeulders Edouard Oyallon 83 333 0 20 Feb 2018
Spectral Normalization for Generative Adversarial Networks Takeru Miyato Toshiki Kataoka Masanori Koyama Yuichi Yoshida ODL 155 4,433 0 16 Feb 2018
Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks Yusuke Tsuzuku Issei Sato Masashi Sugiyama AAML 103 307 0 12 Feb 2018
Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach Tsui-Wei Weng Huan Zhang Pin-Yu Chen Jinfeng Yi D. Su Yupeng Gao Cho-Jui Hsieh Luca Daniel AAML 80 467 0 31 Jan 2018
Certified Defenses against Adversarial Examples Aditi Raghunathan Jacob Steinhardt Percy Liang AAML 105 967 0 29 Jan 2018
Evaluating Robustness of Neural Networks with Mixed Integer Programming Vincent Tjeng Kai Y. Xiao Russ Tedrake AAML 75 117 0 20 Nov 2017
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice Jeffrey Pennington S. Schoenholz Surya Ganguli ODL 43 252 0 13 Nov 2017
Provable defenses against adversarial examples via the convex outer adversarial polytope Eric Wong J. Zico Kolter AAML 104 1,498 0 02 Nov 2017
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform Kyle E. Helfrich Devin Willmott Qiang Ye 78 131 0 29 Jul 2017
Spectrally-normalized margin bounds for neural networks Peter L. Bartlett Dylan J. Foster Matus Telgarsky ODL 191 1,216 0 26 Jun 2017
An approach to reachability analysis for feed-forward ReLU neural networks A. Lomuscio Lalit Maganti 52 357 0 22 Jun 2017
Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation Matthias Hein Maksym Andriushchenko AAML 97 511 0 23 May 2017
Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks Rüdiger Ehlers 91 624 0 03 May 2017
Maximum Resilience of Artificial Neural Networks Chih-Hong Cheng Georg Nührenberg Harald Ruess AAML 88 283 0 28 Apr 2017
Parseval Networks: Improving Robustness to Adversarial Examples Moustapha Cissé Piotr Bojanowski Edouard Grave Yann N. Dauphin Nicolas Usunier AAML 136 807 0 28 Apr 2017
On orthogonality and learning recurrent networks with long term dependencies Eugene Vorontsov C. Trabelsi Samuel Kadoury C. Pal ODL 75 241 0 31 Jan 2017
Regularizing CNNs with Locally Constrained Decorrelations Pau Rodríguez López Jordi Gonzalez Guillem Cucurull J. M. Gonfaus F. X. Roca 68 133 0 07 Nov 2016
Safety Verification of Deep Neural Networks Xiaowei Huang Marta Kwiatkowska Sen Wang Min Wu AAML 200 938 0 21 Oct 2016
Towards Evaluating the Robustness of Neural Networks Nicholas Carlini D. Wagner OOD AAML 239 8,548 0 16 Aug 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 324 7,971 0 23 May 2016
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks Tim Salimans Diederik P. Kingma ODL 186 1,940 0 25 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.1K 193,426 0 10 Dec 2015
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 73 769 0 20 Nov 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.6K 149,842 0 22 Dec 2014
Intriguing properties of neural networks Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna D. Erhan Ian Goodfellow Rob Fergus AAML 251 14,912 1 21 Dec 2013