Graph Convolutions Enrich the Self-Attention in Transformers!

Graph Convolutions Enrich the Self-Attention in Transformers!

7 December 2023

Jeongwhan Choi

Nathaniel Trask

Papers citing "Graph Convolutions Enrich the Self-Attention in Transformers!"

9 / 9 papers shown

Title
Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain Hyowon Wi Jeongwhan Choi Noseong Park 33 0 0 13 May 2025
Setting the Record Straight on Transformer Oversmoothing G. Dovonon M. Bronstein Matt J. Kusner 28 5 0 09 Jan 2024
RDGCL: Reaction-Diffusion Graph Contrastive Learning for Recommendation Jeongwhan Choi Hyowon Wi C. Lee Sung-Bae Cho Dongha Lee Noseong Park DiffM 44 2 0 27 Dec 2023
Stabilizing Transformer Training by Preventing Attention Entropy Collapse Shuangfei Zhai Tatiana Likhomanenko Etai Littwin Dan Busbridge Jason Ramapuram Yizhe Zhang Jiatao Gu J. Susskind AAML 46 65 0 11 Mar 2023
A Simple Yet Effective SVD-GCN for Directed Graphs Chunya Zou Andi Han Lequan Lin Junbin Gao GNN 66 7 0 19 May 2022
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Chenyu You Guosheng Lin 246 1,492 0 02 Sep 2021
Benchmarking Graph Neural Networks Vijay Prakash Dwivedi Chaitanya K. Joshi Anh Tuan Luu T. Laurent Yoshua Bengio Xavier Bresson 189 917 0 02 Mar 2020
Representation Learning on Graphs with Jumping Knowledge Networks Keyulu Xu Chengtao Li Yonglong Tian Tomohiro Sonobe Ken-ichi Kawarabayashi Stefanie Jegelka GNN 279 1,944 0 09 Jun 2018
Densely Connected Convolutional Networks Gao Huang Zhuang Liu L. V. D. van der Maaten Kilian Q. Weinberger PINN 3DV 315 36,381 0 25 Aug 2016