v1v2 (latest)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models

26 February 2024

ArXiv (abs)PDF HTML Github (87★)

Papers citing "DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models"

20 / 20 papers shown

Title
SSD4Rec: A Structured State Space Duality Model for Efficient Sequential Recommendation Haohao Qu Yifeng Zhang Liangbo Ning Wenqi Fan Qing Li Mamba 172 9 0 17 Jan 2025
FACTS: A Factored State-Space Framework For World Modelling Li Nanbo Firas Laakom Yucheng Xu Wenyi Wang Jürgen Schmidhuber AI4TS 526 1 0 28 Oct 2024
Spatial-Mamba: Effective Visual State Space Models via Structure-aware State Fusion Chaodong Xiao Minghan Li Zhengqiang Zhang Deyu Meng Lei Zhang Mamba 122 5 0 19 Oct 2024
Hierarchical Spatio-Temporal State-Space Modeling for fMRI Analysis Yuxiang Wei A. Abrol Reihaneh Hassanzadeh Mamba 80 3 0 23 Aug 2024
Gated Linear Attention Transformers with Hardware-Efficient Training Aaron Courville Bailin Wang Songlin Yang Yikang Shen Yoon Kim 108 180 0 11 Dec 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 235 607 0 22 May 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng Zhang Yuxiao Dong Jie Tang BDL LRM 355 1,094 0 05 Oct 2022
Long Range Language Modeling via Gated State Spaces Harsh Mehta Ankit Gupta Ashok Cutkosky Behnam Neyshabur Mamba 107 242 0 27 Jun 2022
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute Tao Lei RALM VLM 119 49 0 24 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 475 2,120 0 31 Dec 2020
HiPPO: Recurrent Memory with Optimal Polynomial Projections Albert Gu Tri Dao Stefano Ermon Atri Rudra Christopher Ré 126 533 0 17 Aug 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 203 1,786 0 29 Jun 2020
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning Edoardo Ponti Goran Glavaš Olga Majewska Qianchu Liu Ivan Vulić Anna Korhonen LRM 84 326 0 01 May 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 611 4,905 0 23 Jan 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 485 20,317 0 23 Oct 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 244 1,551 0 24 May 2019
Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning Stefan Elfwing E. Uchibe Kenji Doya 136 1,739 0 10 Feb 2017
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 338 2,898 0 26 Sep 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 793 36,881 0 25 Aug 2016
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 174 5,042 0 27 Jun 2016