MatMamba: A Matryoshka State Space Model

9 October 2024

Papers citing "MatMamba: A Matryoshka State Space Model"

23 / 23 papers shown

Title
Chain-of-Model Learning for Language Model Kaitao Song Xiaohua Wang Xu Tan Huiqiang Jiang Chengruidong Zhang ... Xiaoqing Zheng Tao Qin Yuqing Yang Dongsheng Li Lili Qiu LRM AI4CE 152 1 0 17 May 2025
MambaVision: A Hybrid Mamba-Transformer Vision Backbone Ali Hatamizadeh Jan Kautz Mamba 116 69 0 10 Jul 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 114 103 0 05 Jul 2024
An Empirical Study of Mamba-based Language Models R. Waleffe Wonmin Byeon Duncan Riach Brandon Norick V. Korthikanti ... Vartika Singh Jared Casper Jan Kautz Mohammad Shoeybi Bryan Catanzaro 107 74 0 12 Jun 2024
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality Tri Dao Albert Gu Mamba 97 502 0 31 May 2024
MatFormer: Nested Transformer for Elastic Inference Devvrit Sneha Kudugunta Aditya Kusupati Tim Dettmers Kaifeng Chen ... Yulia Tsvetkov Hannaneh Hajishirzi Sham Kakade Ali Farhadi Prateek Jain 86 30 0 11 Oct 2023
Compressing LLMs: The Truth is Rarely Pure and Never Simple Ajay Jaiswal Zhe Gan Xianzhi Du Bowen Zhang Zhangyang Wang Yinfei Yang MQ 87 50 0 02 Oct 2023
FFCV: Accelerating Training by Removing Data Bottlenecks Guillaume Leclerc Andrew Ilyas Logan Engstrom Sung Min Park Hadi Salman Aleksander Madry 41 69 0 21 Jun 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 207 593 0 22 May 2023
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 113 702 0 30 Nov 2022
Matryoshka Representation Learning Aditya Kusupati Gantavya Bhatt Aniket Rege Matthew Wallingford Aditya Sinha ... William Howard-Snyder Kaifeng Chen Sham Kakade Prateek Jain Ali Farhadi 82 86 0 26 May 2022
DeiT III: Revenge of the ViT Hugo Touvron Matthieu Cord Hervé Jégou ViT 118 412 0 14 Apr 2022
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers Andreas Steiner Alexander Kolesnikov Xiaohua Zhai Ross Wightman Jakob Uszkoreit Lucas Beyer ViT 107 632 0 18 Jun 2021
Deep Learning for Instance Retrieval: A Survey Wei Chen Yu Liu Weiping Wang E. Bakker Theodoros Georgiou Paul Fieguth Li Liu M. Lew VLM 55 149 0 27 Jan 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 632 41,003 0 22 Oct 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 201 1,755 0 29 Jun 2020
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 221 3,485 0 30 Sep 2019
Once-for-All: Train One Network and Specialize it for Efficient Deployment Han Cai Chuang Gan Tianzhe Wang Zhekai Zhang Song Han OOD 102 1,280 0 26 Aug 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 609 4,778 0 13 May 2019
Slimmable Neural Networks Jiahui Yu L. Yang N. Xu Jianchao Yang Thomas Huang 75 552 0 21 Dec 2018
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 276 9,760 0 25 Oct 2017
Random Erasing Data Augmentation Zhun Zhong Liang Zheng Guoliang Kang Shaozi Li Yi Yang 90 3,635 0 16 Aug 2017
Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning Stefan Elfwing E. Uchibe Kenji Doya 133 1,723 0 10 Feb 2017