What Makes Convolutional Models Great on Long Sequence Modeling?

What Makes Convolutional Models Great on Long Sequence Modeling?

17 October 2022

Tianle Cai

Papers citing "What Makes Convolutional Models Great on Long Sequence Modeling?"

18 / 18 papers shown

Title
A Survey on Mamba Architecture for Vision Applications Fady Ibrahim Guangjun Liu Guanghui Wang Mamba 62 2 0 11 Feb 2025
Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences Zicheng Liu Siyuan Li Li Wang Zedong Wang Yunfan Liu Stan Z. Li 35 7 0 12 Jun 2024
Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks Jerome Sieber Carmen Amo Alonso A. Didier M. Zeilinger Antonio Orvieto AAML 50 8 0 24 May 2024
Learning Enriched Features via Selective State Spaces Model for Efficient Image Deblurring Huiyu Gao Depeng Dang 43 13 0 29 Mar 2024
Theoretical Foundations of Deep Selective State-Space Models Nicola Muca Cirone Antonio Orvieto Benjamin Walker C. Salvi Terry Lyons Mamba 59 25 0 29 Feb 2024
Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling Mahdi Karami Ali Ghodsi VLM 48 6 0 28 Feb 2024
Accel-GCN: High-Performance GPU Accelerator Design for Graph Convolution Networks Xiaoru Xie Hongwu Peng Amit Hasan Shaoyi Huang Jiahui Zhao Haowen Fang Wei Zhang Tong Geng O. Khan Caiwen Ding GNN 38 30 0 22 Aug 2023
AutoReP: Automatic ReLU Replacement for Fast Private Network Inference Hongwu Peng Shaoyi Huang Tong Zhou Yukui Luo Chenghong Wang ... Tony Geng Kaleel Mahmood Wujie Wen Xiaolin Xu Caiwen Ding OffRL 47 38 0 20 Aug 2023
Focus Your Attention (with Adaptive IIR Filters) Shahar Lutati Itamar Zimerman Lior Wolf 32 9 0 24 May 2023
Scallop: A Language for Neurosymbolic Programming Ziyang Li Jiani Huang Mayur Naik ReLM LRM NAI 21 30 0 10 Apr 2023
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 16 52 0 13 Feb 2023
Scaling Up Computer Vision Neural Networks Using Fast Fourier Transform Siddharth Agrawal 15 0 0 02 Feb 2023
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 30 109 0 31 Aug 2022
How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections Albert Gu Isys Johnson Aman Timalsina Atri Rudra Christopher Ré Mamba 95 89 0 24 Jun 2022
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes David W. Romero Robert-Jan Bruintjes Jakub M. Tomczak Erik J. Bekkers Mark Hoogendoorn J. C. V. Gemert 80 82 0 15 Oct 2021
RepVGG: Making VGG-style ConvNets Great Again Xiaohan Ding Xinming Zhang Ningning Ma Jungong Han Guiguang Ding Jian Sun 136 1,548 0 11 Jan 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 285 2,015 0 28 Jul 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018