Demystifying Oversmoothing in Attention-Based Graph Neural Networks

Demystifying Oversmoothing in Attention-Based Graph Neural Networks

25 May 2023

Papers citing "Demystifying Oversmoothing in Attention-Based Graph Neural Networks"

14 / 14 papers shown

Title
Transformer Meets Twicing: Harnessing Unattended Residual Information Laziz U. Abdullaev Tan M. Nguyen 41 2 0 02 Mar 2025
Mamba-Based Graph Convolutional Networks: Tackling Over-smoothing with Selective State Space Xin He Yixuan Wang Wenqi Fan Xu Shen Xin Juan Rui Miao Xin Wang 68 0 0 26 Jan 2025
The Geometry of Tokens in Internal Representations of Large Language Models Karthik Viswanathan Yuri Gardinazzi Giada Panerai Alberto Cazzaniga Matteo Biagetti AIFin 94 4 0 17 Jan 2025
Lambda-Skip Connections: the architectural component that prevents Rank Collapse Federico Arangath Joseph Jerome Sieber M. Zeilinger Carmen Amo Alonso 33 0 0 14 Oct 2024
NetDiff: Deep Graph Denoising Diffusion for Ad Hoc Network Topology Generation Félix Marcoccia C. Adjih P. Mühlethaler DiffM 16 0 0 09 Oct 2024
Revisiting Random Walks for Learning on Graphs Jinwoo Kim Olga Zaghen Ayhan Suleymanzade Youngmin Ryou Seunghoon Hong 62 0 0 01 Jul 2024
Bundle Neural Networks for message diffusion on graphs Jacob Bamberger Federico Barbero Xiaowen Dong Michael M. Bronstein 39 1 0 24 May 2024
GRASP: GRAph-Structured Pyramidal Whole Slide Image Representation Ali Khajegili Mirabadi Graham Archibald Amirali Darbandsari A. Contreras-Sanz Ramin Nakhli ... C. Gilks Peter C Black Gang Wang H. Farahani A. Bashashati 32 5 0 06 Feb 2024
Graph Convolutions Enrich the Self-Attention in Transformers! Jeongwhan Choi Hyowon Wi Jayoung Kim Yehjin Shin Kookjin Lee Nathaniel Trask Noseong Park 30 4 0 07 Dec 2023
Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networks Andreas Roth Thomas Liebig 43 12 0 31 Aug 2023
Geom-GCN: Geometric Graph Convolutional Networks Hongbin Pei Bingzhen Wei Kevin Chen-Chuan Chang Yu Lei Bo Yang GNN 169 1,078 0 13 Feb 2020
Representation Learning on Graphs with Jumping Knowledge Networks Keyulu Xu Chengtao Li Yonglong Tian Tomohiro Sonobe Ken-ichi Kawarabayashi Stefanie Jegelka GNN 267 1,945 0 09 Jun 2018
Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation Jiaxuan You Bowen Liu Rex Ying Vijay S. Pande J. Leskovec GNN 203 885 0 07 Jun 2018
Interaction Networks for Learning about Objects, Relations and Physics Peter W. Battaglia Razvan Pascanu Matthew Lai Danilo Jimenez Rezende Koray Kavukcuoglu AI4CE OCL PINN GNN 280 1,400 0 01 Dec 2016