Geometric Hyena Networks for Large-scale Equivariant Learning

28 May 2025

Papers citing "Geometric Hyena Networks for Large-scale Equivariant Learning"

50 / 52 papers shown

Title
HELM: Hierarchical Encoding for mRNA Language Modeling Mehdi Yazdani-Jahromi Mangal Prakash Tommaso Mansi Artem Moskalev Rui Liao 119 3 0 13 Mar 2025
Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems Maksim Zhdanov Max Welling Jan-Willem van de Meent AI4CE 98 2 0 24 Feb 2025
Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks? Jiacheng Cen Anyi Li Ning Lin Yuxiang Ren Zihe Wang Wenbing Huang 76 4 0 15 Oct 2024
Beyond Sequence: Impact of Geometric Context for RNA Property Prediction Junjie Xu Artem Moskalev Tommaso Mansi Mangal Prakash Rui Liao AI4CE 66 3 0 15 Oct 2024
Neural P $^3$ M: A Long-Range Interaction Modeling Enhancer for Geometric GNNs Yusong Wang Chaoran Cheng Shaoning Li Yuxuan Ren Jia Zhang Ge Liu Pheng-Ann Heng Nanning Zheng AI4CE 49 4 0 26 Sep 2024
Understanding Virtual Nodes: Oversquashing and Node Heterogeneity Joshua Southern Francesco Di Giovanni Michael M. Bronstein J. Lutzeyer 114 1 0 22 May 2024
Mechanistic Design and Scaling of Hybrid Architectures Michael Poli Armin W. Thomas Eric N. D. Nguyen Pragaash Ponnusamy Bjorn Deiseroth ... Brian Hie Stefano Ermon Christopher Ré Ce Zhang Stefano Massaroli MoE 90 27 0 26 Mar 2024
Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling Yair Schiff Chia-Hsiang Kao Aaron Gokaslan Tri Dao Albert Gu Volodymyr Kuleshov Mamba 55 93 0 05 Mar 2024
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models Soham De Samuel L. Smith Anushan Fernando Aleksandar Botev George-Christian Muraru ... David Budden Yee Whye Teh Razvan Pascanu Nando de Freitas Çağlar Gülçehre Mamba 89 129 0 29 Feb 2024
TorchMD-Net 2.0: Fast Neural Network Potentials for Molecular Simulations Raúl P. Peláez Guillem Simeon Raimondas Galvelis Antonio Mirarchi Peter K. Eastman Stefan Doerr Philipp Thölke T. Markland Gianni De Fabritiis AI4CE 57 15 0 27 Feb 2024
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model Lianghui Zhu Bencheng Liao Qian Zhang Xinlong Wang Wenyu Liu Xinggang Wang Mamba 88 768 0 17 Jan 2024
Mamba: Linear-Time Sequence Modeling with Selective State Spaces Albert Gu Tri Dao Mamba 124 2,665 0 01 Dec 2023
Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformers P. D. Haan Taco S. Cohen Johann Brehmer 45 10 0 08 Nov 2023
Fast, Expressive SE $(n)$ Equivariant Networks through Weight-Sharing in Position-Orientation Space Erik J. Bekkers Sharvaree P. Vadgama Rob D. Hesselink P. A. V. D. Linden David W. Romero 47 29 0 04 Oct 2023
On genuine invariance learning without weight-tying A. Moskalev A. Sepliarskaia Erik J. Bekkers A. Smeulders CML OOD 40 8 0 07 Aug 2023
Geometric Algebra Transformer Johann Brehmer P. D. Haan S. Behrends Taco S. Cohen 64 31 0 28 May 2023
Clifford Group Equivariant Neural Networks David Ruhe Johannes Brandstetter Patrick Forré 52 41 0 18 May 2023
A new perspective on building efficient and expressive 3D equivariant graph neural networks Weitao Du Yuanqi Du Limei Wang Dieqiao Feng Guifeng Wang Shuiwang Ji Carla P. Gomes Zhixin Ma AI4CE 73 39 0 07 Apr 2023
A Survey on Oversmoothing in Graph Neural Networks T. Konstantin Rusch Michael M. Bronstein Siddhartha Mishra 79 205 0 20 Mar 2023
Hyena Hierarchy: Towards Larger Convolutional Language Models Michael Poli Stefano Massaroli Eric Q. Nguyen Daniel Y. Fu Tri Dao S. Baccus Yoshua Bengio Stefano Ermon Christopher Ré VLM 76 298 0 21 Feb 2023
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 97 396 0 28 Dec 2022
Implicit Convolutional Kernels for Steerable CNNs Maksim Zhdanov Nico Hoffmann Gabriele Cesa 100 7 0 12 Dec 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 305 514 0 24 Sep 2022
Pure Transformers are Powerful Graph Learners Jinwoo Kim Tien Dat Nguyen Seonwoo Min Sungjun Cho Moontae Lee Honglak Lee Seunghoon Hong 79 200 0 06 Jul 2022
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs Yi-Lun Liao Tess E. Smidt 115 240 0 23 Jun 2022
VN-Transformer: Rotation-Equivariant Attention for Vector Neurons Serge Assaad Carlton Downey Rami Al-Rfou Nigamaa Nayakanti Benjamin Sapp 54 19 0 08 Jun 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 193 2,214 0 27 May 2022
Recipe for a General, Powerful, Scalable Graph Transformer Ladislav Rampášek Mikhail Galkin Vijay Prakash Dwivedi Anh Tuan Luu Guy Wolf Dominique Beaini 106 564 0 25 May 2022
Efficiently Modeling Long Sequences with Structured State Spaces Albert Gu Karan Goel Christopher Ré 186 1,773 0 31 Oct 2021
Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers Albert Gu Isys Johnson Karan Goel Khaled Kamal Saab Tri Dao Atri Rudra Christopher Ré 108 593 0 26 Oct 2021
Deep learning models for predicting RNA degradation via dual crowdsourcing H. Wayment-Steele W. Kladwang Andrew Watkins Do Soon Kim Bojan Tunguz ... Emin Öztürk K. Amer Mohamed Fares Eterna Participants Rhiju Das 42 19 0 14 Oct 2021
Geometric and Physical Quantities Improve E(3) Equivariant Message Passing Johannes Brandstetter Rob D. Hesselink Elise van der Pol Erik J. Bekkers Max Welling 80 242 0 06 Oct 2021
Rethinking Graph Transformers with Spectral Attention Devin Kreuzer Dominique Beaini William L. Hamilton Vincent Létourneau Prudencio Tossou 88 536 0 07 Jun 2021
Equivariant Graph Neural Networks for 3D Macromolecular Structure Bowen Jing Stephan Eismann Pratham N. Soni R. Dror 52 101 0 07 Jun 2021
GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph Junhan Yang Zheng Liu Shitao Xiao Chaozhuo Li Defu Lian Sanjay Agrawal Amit Singh Guangzhong Sun Xing Xie AI4CE 44 157 0 06 May 2021
Vector Neurons: A General Framework for SO(3)-Equivariant Networks Congyue Deng Or Litany Yueqi Duan A. Poulenard Andrea Tagliasacchi Leonidas Guibas 3DPC 170 325 0 25 Apr 2021
E(n) Equivariant Graph Neural Networks Victor Garcia Satorras Emiel Hoogeboom Max Welling 92 997 0 19 Feb 2021
CKConv: Continuous Kernel Convolution For Sequential Data David W. Romero Anna Kuzina Erik J. Bekkers Jakub M. Tomczak Mark Hoogendoorn 56 125 0 04 Feb 2021
Learning from Protein Structure with Geometric Vector Perceptrons Bowen Jing Stephan Eismann Patricia Suriana Raphael J. L. Townshend R. Dror GNN 3DV 64 482 0 03 Sep 2020
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks F. Fuchs Daniel E. Worrall Volker Fischer Max Welling 3DPC 147 692 0 18 Jun 2020
Implicit Neural Representations with Periodic Activation Functions Vincent Sitzmann Julien N. P. Martel Alexander W. Bergman David B. Lindell Gordon Wetzstein AI4TS 120 2,548 0 17 Jun 2020
Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities Jonas Köhler Leon Klein Frank Noé DRL 79 271 0 03 Jun 2020
Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs P. D. Haan Maurice Weiler Taco S. Cohen Max Welling 134 131 0 11 Mar 2020
General $E(2)$ -Equivariant Steerable CNNs Maurice Weiler Gabriele Cesa 91 524 0 19 Nov 2019
A Comprehensive Survey on Graph Neural Networks Zonghan Wu Shirui Pan Fengwen Chen Guodong Long Chengqi Zhang Philip S. Yu FaML GNN AI4TS AI4CE 722 8,517 0 03 Jan 2019
PointConv: Deep Convolutional Networks on 3D Point Clouds Wenxuan Wu Zhongang Qi Fuxin Li 3DPC 114 1,560 0 17 Nov 2018
Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds Nathaniel Thomas Tess E. Smidt S. Kearnes Lusann Yang Li Li Kai Kohlhoff Patrick F. Riley 3DPC 82 969 0 22 Feb 2018
SchNet: A continuous-filter convolutional neural network for modeling quantum interactions Kristof T. Schütt Pieter-Jan Kindermans Huziel Enoc Sauceda Felix Stefan Chmiela A. Tkatchenko K. Müller 146 1,074 0 26 Jun 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 658 131,414 0 12 Jun 2017
Neural Message Passing for Quantum Chemistry Justin Gilmer S. Schoenholz Patrick F. Riley Oriol Vinyals George E. Dahl 566 7,441 0 04 Apr 2017