Efficiently Modeling Long Sequences with Structured State Spaces

31 October 2021

Papers citing "Efficiently Modeling Long Sequences with Structured State Spaces"

50 / 1,151 papers shown

Title
SMR: State Memory Replay for Long Sequence Modeling Biqing Qi Junqi Gao Kaiyan Zhang Dong Li Jianxing Liu Ligang Wu Bowen Zhou 59 5 0 27 May 2024
The Expressive Capacity of State Space Models: A Formal Language Perspective Yash Sarrof Yana Veitsman Michael Hahn Mamba 43 8 0 27 May 2024
Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective Zhen Qin Xuyang Shen Weigao Sun Dong Li Stanley T. Birchfield Leonid Sigal Yiran Zhong 58 6 0 27 May 2024
Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention Zhen Qin Weigao Sun Dong Li Xuyang Shen Weixuan Sun Yiran Zhong 46 9 0 27 May 2024
Rethinking Transformers in Solving POMDPs Chenhao Lu Ruizhe Shi Yuyao Liu Kaizhe Hu Simon S. Du Huazhe Xu AI4CE 58 3 0 27 May 2024
LCM: Locally Constrained Compact Point Cloud Model for Masked Point Modeling Yaohua Zha Naiqi Li Yanzi Wang Tao Dai Hang Guo Bin Chen Zhi Wang Zhihao Ouyang Shu-Tao Xia Mamba 58 8 0 27 May 2024
Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports Yi Tian Xu Yun Fu 40 1 0 27 May 2024
Zamba: A Compact 7B SSM Hybrid Model Paolo Glorioso Quentin G. Anthony Yury Tokpanov James Whittington Jonathan Pilault Adam Ibrahim Beren Millidge 35 45 0 26 May 2024
Mamba4KT:An Efficient and Effective Mamba-based Knowledge Tracing Model Yang Cao Wei Zhang Mamba 41 4 0 26 May 2024
Image Deraining with Frequency-Enhanced State Space Model Shugo Yamashita Masaaki Ikehara 48 3 0 26 May 2024
MambaTS: Improved Selective State Space Models for Long-term Time Series Forecasting Xiuding Cai Yaoyao Zhu Xueyao Wang Yu Yao Mamba AI4TS 54 7 0 26 May 2024
Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory Nikola Zubić Federico Soldá Aurelio Sulser Davide Scaramuzza LRM BDL 59 6 0 26 May 2024
Time-SSM: Simplifying and Unifying State Space Models for Time Series Forecasting Jiaxi Hu Disen Lan Ziyu Zhou Qingsong Wen Yuxuan Liang AI4TS 44 8 0 25 May 2024
MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space Jiangwei Weng Zhiqiang Yan Ying Tai J. Qian Jian Yang Jun Li Mamba 34 10 0 25 May 2024
Scaling Diffusion Mamba with Bidirectional SSMs for Efficient Image and Video Generation Shentong Mo Yapeng Tian Mamba 76 16 0 24 May 2024
CausalConceptTS: Causal Attributions for Time Series Classification using High Fidelity Diffusion Models Juan Miguel Lopez Alcaraz Nils Strodthoff DiffM AI4TS CML 34 3 0 24 May 2024
Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks Jerome Sieber Carmen Amo Alonso A. Didier Melanie Zeilinger Antonio Orvieto AAML 79 8 0 24 May 2024
PoinTramba: A Hybrid Transformer-Mamba Framework for Point Cloud Analysis Zicheng Wang Zhen Chen Yiming Wu Zhen Zhao Luping Zhou Dong Xu Mamba 50 14 0 24 May 2024
MambaVC: Learned Visual Compression with Selective State Spaces Shiyu Qin Jinpeng Wang Yimin Zhou Bin Chen Tianci Luo Baoyi An Tao Dai Shu-Tao Xia Yaowei Wang Mamba 46 13 0 24 May 2024
Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Rate Fan Luo Zuolin Tu Zefang Huang Yang Yu OffRL 54 0 0 24 May 2024
On the Identification of Temporally Causal Representation with Instantaneous Dependence Zijian Li Yifan Shen Kaitao Zheng Ruichu Cai Xiangchen Song Biwei Huang Zhengmao Zhu Guan-Hong Chen Kun Zhang CML 64 5 0 24 May 2024
PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learning Qingdong He Jiangning Zhang Jinlong Peng Haoyang He Yabiao Wang Chengjie Wang 3DPC 51 13 0 24 May 2024
Mamba-R: Vision Mamba ALSO Needs Registers Feng Wang Jiahao Wang Sucheng Ren Guoyizhe Wei Jieru Mei Wei Shao Yuyin Zhou Alan Yuille Cihang Xie Mamba 39 22 0 23 May 2024
Lessons from the Trenches on Reproducible Evaluation of Language Models Stella Biderman Hailey Schoelkopf Lintang Sutawika Leo Gao J. Tow ... Xiangru Tang Kevin A. Wang Genta Indra Winata Franccois Yvon Andy Zou ELM ALM 143 54 3 23 May 2024
EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records Adibvafa Fallahpour Mahshid Alinoori Arash Afkanpour Amrit Krishnan Mamba 32 3 0 23 May 2024
Scalable Visual State Space Model with Fractal Scanning Lv Tang Haoke Xiao Peng-Tao Jiang Hao Zhang Jinwei Chen Yue Liu Mamba 48 7 0 23 May 2024
Efficient Visual State Space Model for Image Deblurring Lingshun Kong Jiangxin Dong Ming-Hsuan Yang Jinshan Pan 54 4 0 23 May 2024
Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model Yuheng Shi Minjing Dong Chang Xu Mamba 67 35 0 23 May 2024
HeteGraph-Mamba: Heterogeneous Graph Learning via Selective State Space Model Zhenyu Pan Yoonsung Jeong Xiaoda Liu Han Liu 60 1 0 22 May 2024
Audio Mamba: Pretrained Audio State Space Model For Audio Tagging Jiaju Lin Haoxuan Hu Mamba 39 8 0 22 May 2024
Mamba in Speech: Towards an Alternative to Self-Attention Xiangyu Zhang Qiquan Zhang Hexin Liu Tianyi Xiao Xinyuan Qian Beena Ahmed E. Ambikairajah Haizhou Li Julien Epps Mamba 59 41 0 21 May 2024
3DSS-Mamba: 3D-Spectral-Spatial Mamba for Hyperspectral Image Classification Yan He Bing Tu Bo Liu Jun Li Antonio J. Plaza Mamba 41 25 0 21 May 2024
TENNs-PLEIADES: Building Temporal Kernels with Orthogonal Polynomials Yan Ru Pei Olivier Coenen 42 3 0 20 May 2024
Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning? Yang Dai Oubo Ma Longfei Zhang Xingxing Liang Shengchao Hu Mengzhu Wang Shouling Ji Jincai Huang Li Shen Mamba 45 4 0 20 May 2024
Mamba-in-Mamba: Centralized Mamba-Cross-Scan in Tokenized Mamba Model for Hyperspectral Image Classification Weilian Zhou Sei-ichiro Kamata Haipeng Wang Man-Sing Wong Huiying H. Hou Mamba 37 31 0 20 May 2024
SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model Siavash Shams Sukru Samet Dindar Xilin Jiang N. Mesgarani Mamba 74 20 0 20 May 2024
LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions Victor Agostinelli Sanghyun Hong Lizhong Chen KELM 51 1 0 18 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 75 48 0 17 May 2024
CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation Mushui Liu Jun Dan Ziqian Lu Yunlong Yu Yingming Li Xi Li Mamba 47 20 0 17 May 2024
Improving Transformers with Dynamically Composable Multi-Head Attention Da Xiao Qingye Meng Shengping Li Xingyuan Yuan 40 3 0 14 May 2024
Rethinking Scanning Strategies with Vision Mamba in Semantic Segmentation of Remote Sensing Imagery: An Experimental Study Qinfeng Zhu Yuan-Sheng Fang Yuanzhi Cai Cheng Chen Lei Fan Mamba 50 16 0 14 May 2024
WaterMamba: Visual State Space Model for Underwater Image Enhancement Meisheng Guan Haiyong Xu G. Jiang Mei Yu Yeyao Chen Ting Luo Yang Song Mamba 70 3 0 14 May 2024
MambaOut: Do We Really Need Mamba for Vision? Weihao Yu Xinchao Wang Mamba 55 53 0 13 May 2024
GMSR:Gradient-Guided Mamba for Spectral Reconstruction from RGB Images Xinying Wang Zhixiong Huang Sifan Zhang Jiawen Zhu Lin Feng Mamba 37 5 0 13 May 2024
Predictive Modeling in the Reservoir Kernel Motif Space Peter Tino Robert Simon Fong R. Leonarduzzi AI4TS 32 2 0 11 May 2024
DTMamba : Dual Twin Mamba for Time Series Forecasting Zexue Wu Yifeng Gong Aoqian Zhang 26 1 0 11 May 2024
Linearizing Large Language Models Jean Mercat Igor Vasiljevic Sedrick Scott Keh Kushal Arora Achal Dave Adrien Gaidon Thomas Kollar 51 19 0 10 May 2024
An Investigation of Incorporating Mamba for Speech Enhancement Rong-Yu Chao Wen-Huang Cheng Moreno La Quatra Sabato Marco Siniscalchi Chao-Han Huck Yang Szu-Wei Fu Yu Tsao Mamba 67 27 0 10 May 2024
State-Free Inference of State-Space Models: The Transfer Function Approach Rom N. Parnichkun Stefano Massaroli Alessandro Moro Jimmy T.H. Smith Ramin Hasani ... Hajime Asama Stefano Ermon Taiji Suzuki Atsushi Yamashita Michael Poli 56 6 0 10 May 2024
Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Mask Zineb Senane Lele Cao V. Buchner Yusuke Tashiro Lei You P. Herman Mats Nordahl Ruibo Tu Vilhelm von Ehrenheim DiffM AI4TS 41 10 0 09 May 2024