SEA: State-Exchange Attention for High-Fidelity Physics Based
Transformers

v1v2 (latest)

SEA: State-Exchange Attention for High-Fidelity Physics Based Transformers

20 October 2024

Amirhossein Dadashzadeh

Nicolas Larrosa

ArXiv (abs)PDF HTML Github (4★)

Papers citing "SEA: State-Exchange Attention for High-Fidelity Physics Based Transformers"

18 / 18 papers shown

Title
Scalable Diffusion Models with Transformers William S. Peebles Saining Xie GNN 93 2,301 0 19 Dec 2022
Residual-based physics-informed transfer learning: A hybrid method for accelerating long-term CFD simulations via deep learning J. Jeon Juhyeong Lee Ricardo Vinuesa S. J. Kim AI4CE 44 29 0 14 Jun 2022
Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics Xin-Yang Liu Min Zhu Lu Lu Hao Sun Jian-Xun Wang PINN AI4CE 89 51 0 09 May 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 204 1,612 0 07 Apr 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 428 15,515 0 20 Dec 2021
CAT: Cross Attention in Vision Transformer Hezheng Lin Xingyi Cheng Xiangyu Wu Fan Yang Dong Shen Zhongyuan Wang Qing Song Wei Yuan ViT 56 153 0 10 Jun 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 278 2,453 0 20 Apr 2021
ViViT: A Video Vision Transformer Anurag Arnab Mostafa Dehghani G. Heigold Chen Sun Mario Lucic Cordelia Schmid ViT 222 2,150 0 29 Mar 2021
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification Chun-Fu Chen Quanfu Fan Yikang Shen ViT 71 1,478 0 27 Mar 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 197 1,015 0 04 Mar 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 372 2,053 0 09 Feb 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 657 41,103 0 22 Oct 2020
Fourier Neural Operator for Parametric Partial Differential Equations Zong-Yi Li Nikola B. Kovachki Kamyar Azizzadenesheli Burigede Liu K. Bhattacharya Andrew M. Stuart Anima Anandkumar AI4CE 497 2,414 0 18 Oct 2020
Learning Mesh-Based Simulation with Graph Networks Tobias Pfaff Meire Fortunato Alvaro Sanchez-Gonzalez Peter W. Battaglia AI4CE 82 790 0 07 Oct 2020
Learning to Simulate Complex Physics with Graph Networks Alvaro Sanchez-Gonzalez Jonathan Godwin Tobias Pfaff Rex Ying J. Leskovec Peter W. Battaglia PINN AI4CE 139 1,091 0 21 Feb 2020
Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow S. Wiewel M. Becher N. Thürey AI4CE 86 276 0 27 Feb 2018
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 172 5,011 0 27 Jun 2016
How transferable are features in deep neural networks? J. Yosinski Jeff Clune Yoshua Bengio Hod Lipson OOD 231 8,336 0 06 Nov 2014