Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networks

8 March 2021

Papers citing "Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networks"

34 / 34 papers shown

Title
Approximation theory for 1-Lipschitz ResNets Davide Murari Takashi Furuya Carola-Bibiane Schönlieb 34 0 0 17 May 2025
Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification Yunsheng Shi Zhengjie Huang Shikun Feng Hui Zhong Wenjin Wang Yu Sun AI4CE 51 770 0 08 Sep 2020
Simple and Deep Graph Convolutional Networks Ming Chen Zhewei Wei Zengfeng Huang Bolin Ding Yaliang Li GNN 79 1,473 0 04 Jul 2020
DeeperGCN: All You Need to Train Deeper GCNs Guohao Li Chenxin Xiong Ali K. Thabet Guohao Li GNN 119 438 0 13 Jun 2020
On the Bottleneck of Graph Neural Networks and its Practical Implications Uri Alon Eran Yahav GNN 77 675 0 09 Jun 2020
The Lipschitz Constant of Self-Attention Hyunjik Kim George Papamakarios A. Mnih 40 143 0 08 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 498 41,106 0 28 May 2020
Open Graph Benchmark: Datasets for Machine Learning on Graphs Weihua Hu Matthias Fey Marinka Zitnik Yuxiao Dong Hongyu Ren Bowen Liu Michele Catasta J. Leskovec 194 2,701 0 02 May 2020
Coloring graph neural networks for node disambiguation George Dasoulas Ludovic Dos Santos Kevin Scaman Aladin Virmaux 56 82 0 12 Dec 2019
On the Relationship between Self-Attention and Convolutional Layers Jean-Baptiste Cordonnier Andreas Loukas Martin Jaggi 89 530 0 08 Nov 2019
Graph Transformer Networks Seongjun Yun Minbyul Jeong Raehyun Kim Jaewoo Kang Hyunwoo J. Kim 116 958 0 06 Nov 2019
PairNorm: Tackling Oversmoothing in GNNs Lingxiao Zhao Leman Akoglu 48 505 0 26 Sep 2019
What graph neural networks cannot learn: depth vs width Andreas Loukas GNN 72 298 0 06 Jul 2019
DeepGCNs: Can GCNs Go as Deep as CNNs? Ge Li Matthias Muller Ali K. Thabet Guohao Li 3DPC GNN 108 1,333 0 07 Apr 2019
On the Turing Completeness of Modern Neural Network Architectures Jorge A. Pérez Javier Marinkovic Pablo Barceló BDL 48 144 0 10 Jan 2019
Invariant and Equivariant Graph Networks Haggai Maron Heli Ben-Hamu Nadav Shamir Y. Lipman 69 498 0 24 Dec 2018
How Powerful are Graph Neural Networks? Keyulu Xu Weihua Hu J. Leskovec Stefanie Jegelka GNN 150 7,554 0 01 Oct 2018
Lipschitz regularity of deep neural networks: analysis and efficient estimation Kevin Scaman Aladin Virmaux 62 523 0 28 May 2018
Regularisation of Neural Networks by Enforcing Lipschitz Continuity Henry Gouk E. Frank Bernhard Pfahringer M. Cree 108 473 0 12 Apr 2018
Spectral Normalization for Generative Adversarial Networks Takeru Miyato Toshiki Kataoka Masanori Koyama Yuichi Yoshida ODL 135 4,421 0 16 Feb 2018
Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning Qimai Li Zhichao Han Xiao-Ming Wu GNN SSL 156 2,796 0 22 Jan 2018
Graph Attention Networks Petar Velickovic Guillem Cucurull Arantxa Casanova Adriana Romero Pietro Lio Yoshua Bengio GNN 314 19,991 0 30 Oct 2017
Representation Learning on Graphs: Methods and Applications William L. Hamilton Rex Ying J. Leskovec GNN 118 1,970 0 17 Sep 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 443 129,831 0 12 Jun 2017
Inductive Representation Learning on Large Graphs William L. Hamilton Z. Ying J. Leskovec 394 15,066 0 07 Jun 2017
A Convolutional Encoder Model for Neural Machine Translation Jonas Gehring Michael Auli David Grangier Yann N. Dauphin 58 450 0 07 Nov 2016
Semi-Supervised Classification with Graph Convolutional Networks Thomas Kipf Max Welling GNN SSL 439 28,901 0 09 Sep 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 246 10,412 0 21 Jul 2016
Gated Graph Sequence Neural Networks Yujia Li Daniel Tarlow Marc Brockschmidt R. Zemel GNN 262 3,271 0 17 Nov 2015
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 313 7,951 0 17 Aug 2015
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention Ke Xu Jimmy Ba Ryan Kiros Kyunghyun Cho Aaron Courville Ruslan Salakhutdinov R. Zemel Yoshua Bengio DiffM 281 10,034 0 10 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 808 149,474 0 22 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 928 99,991 0 04 Sep 2014
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 388 27,205 0 01 Sep 2014