Deep Equilibrium Models

3 September 2019

J. Zico Kolter

Papers citing "Deep Equilibrium Models"

50 / 57 papers shown

Title
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao Wenyuan Xu Shibiao Xu ViT 366 0 0 06 May 2025
Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning Anh Tong Thanh Nguyen-Tang Dongeun Lee Duc Nguyen Toan M. Tran David Hall Cheongwoong Kang Jaesik Choi 82 1 0 03 Mar 2025
Hyper-SET: Designing Transformers via Hyperspherical Energy Minimization Yunzhe Hu Difan Zou Dong Xu 67 1 0 17 Feb 2025
A Generalization Bound for a Family of Implicit Networks Samy Wu Fung Benjamin Berkels 105 0 0 28 Jan 2025
Towards Scalable and Stable Parallelization of Nonlinear RNNs Xavier Gonzalez Andrew Warrington Jimmy T.H. Smith Scott W. Linderman 145 10 0 17 Jan 2025
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA Sangmin Bae Adam Fisch Hrayr Harutyunyan Ziwei Ji Seungyeon Kim Tal Schuster KELM 93 6 0 28 Oct 2024
Efficient, Accurate and Stable Gradients for Neural ODEs Sam McCallum James Foster 68 5 0 15 Oct 2024
IGNN-Solver: A Graph Neural Solver for Implicit Graph Neural Networks Junchao Lin Zenan Ling Zhanbo Feng Feng Zhou Jingwen Xu Feng Zhou Tianqi Hou Zhenyu Liao Robert C. Qiu GNN AI4CE 84 0 0 11 Oct 2024
On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding Kevin Xu Issei Sato 68 4 0 02 Oct 2024
Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling Jinghan Li Zhicheng Sun Fei Li 130 2 0 02 Oct 2024
Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding Yao Teng Han Shi Xian Liu Xuefei Ning Guohao Dai Yu Wang Zhenguo Li Xihui Liu 74 11 0 02 Oct 2024
Unconditional stability of a recurrent neural circuit implementing divisive normalization Shivang Rawat David J. Heeger Stefano Martiniani 99 1 0 27 Sep 2024
First-Order Methods for Linearly Constrained Bilevel Optimization Guy Kornowski Swati Padmanabhan Kai Wang Zhe Zhang S. Sra 112 5 0 18 Jun 2024
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 63 1 0 01 Feb 2024
Global Convergence Rate of Deep Equilibrium Models with General Activations Lan V. Truong 53 2 0 11 Feb 2023
Implicit vs Unfolded Graph Neural Networks Yongyi Yang Tang Liu Yangkun Wang Zengfeng Huang David Wipf 125 15 0 12 Nov 2021
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 211 42,038 0 03 Dec 2019
Implicit Deep Learning L. Ghaoui Fangda Gu Bertrand Travacca Armin Askari Alicia Y. Tsai AI4CE 50 176 0 17 Aug 2019
SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver Po-Wei Wang P. Donti Bryan Wilder Zico Kolter LRM NAI 45 262 0 29 May 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 62 1,880 0 23 Apr 2019
Equilibrated Recurrent Neural Network: Neuronal Time-Delayed Self-Feedback Improves Accuracy and Stability Ziming Zhang Anil Kag Alan Sullivan Venkatesh Saligrama 34 5 0 02 Mar 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 126 3,707 0 09 Jan 2019
Reversible Recurrent Neural Networks M. Mackay Paul Vicol Jimmy Ba Roger C. Grosse 18 52 0 25 Oct 2018
Trellis Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun 49 146 0 15 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 870 93,936 0 11 Oct 2018
Adaptive Input Representations for Neural Language Modeling Alexei Baevski Michael Auli 81 389 0 28 Sep 2018
Character-Level Language Modeling with Deeper Self-Attention Rami Al-Rfou Dokook Choe Noah Constant Mandy Guo Llion Jones 89 388 0 09 Aug 2018
Recurrent Stacking of Layers for Compact Neural Machine Translation Models Raj Dabre Atsushi Fujita 43 60 0 14 Jul 2018
DARTS: Differentiable Architecture Search Hanxiao Liu Karen Simonyan Yiming Yang 159 4,326 0 24 Jun 2018
Neural Ordinary Differential Equations T. Chen Yulia Rubanova J. Bettencourt David Duvenaud AI4CE 220 5,024 0 19 Jun 2018
Relational recurrent neural networks Adam Santoro Ryan Faulkner David Raposo Jack W. Rae Mike Chrzanowski T. Weber Daan Wierstra Oriol Vinyals Razvan Pascanu Timothy Lillicrap GNN 98 211 0 05 Jun 2018
An Analysis of Neural Language Modeling at Multiple Scales Stephen Merity N. Keskar R. Socher 45 170 0 22 Mar 2018
Reviving and Improving Recurrent Back-Propagation Renjie Liao Yuwen Xiong Ethan Fetaya Lisa Zhang Kijung Yoon Xaq Pitkow R. Urtasun R. Zemel BDL 57 118 0 16 Mar 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun DRL 75 4,759 0 04 Mar 2018
Learning Longer-term Dependencies in RNNs with Auxiliary Losses Trieu H. Trinh Andrew M. Dai Thang Luong Quoc V. Le 60 180 0 01 Mar 2018
SparseMAP: Differentiable Sparse Structured Inference Vlad Niculae André F. T. Martins Mathieu Blondel Claire Cardie 35 120 0 12 Feb 2018
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model Zhilin Yang Zihang Dai Ruslan Salakhutdinov William W. Cohen BDL 45 367 0 10 Nov 2017
Regularizing and Optimizing LSTM Language Models Stephen Merity N. Keskar R. Socher 137 1,093 0 07 Aug 2017
On the State of the Art of Evaluation in Neural Language Models Gábor Melis Chris Dyer Phil Blunsom 42 532 0 18 Jul 2017
The Reversible Residual Network: Backpropagation Without Storing Activations Aidan Gomez Mengye Ren R. Urtasun Roger C. Grosse 54 545 0 14 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 422 129,831 0 12 Jun 2017
Stable Architectures for Deep Neural Networks E. Haber Lars Ruthotto 66 722 0 09 May 2017
OptNet: Differentiable Optimization as a Layer in Neural Networks Brandon Amos J. Zico Kolter 130 952 0 01 Mar 2017
Language Modeling with Gated Convolutional Networks Yann N. Dauphin Angela Fan Michael Auli David Grangier 191 2,377 0 23 Dec 2016
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 375 5,346 0 05 Nov 2016
Quasi-Recurrent Neural Networks James Bradbury Stephen Merity Caiming Xiong R. Socher 68 437 0 05 Nov 2016
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 147 2,783 0 26 Sep 2016
WaveNet: A Generative Model for Raw Audio Aaron van den Oord Sander Dieleman Heiga Zen Karen Simonyan Oriol Vinyals Alex Graves Nal Kalchbrenner A. Senior Koray Kavukcuoglu DiffM 270 7,361 0 12 Sep 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 235 10,412 0 21 Jul 2016
Training Deep Nets with Sublinear Memory Cost Tianqi Chen Bing Xu Chiyuan Zhang Carlos Guestrin 79 1,154 0 21 Apr 2016