Reservoir Transformers

Reservoir Transformers

30 December 2020

Douwe Kiela

Papers citing "Reservoir Transformers"

12 / 12 papers shown

Title
Learning Music Audio Representations With Limited Data Christos Plachouras Emmanouil Benetos Johan Pauwels 26 0 0 09 May 2025
Partially Randomizing Transformer Weights for Dialogue Response Diversity Jing Yang Lee Kong Aik Lee Woon-Seng Gan 23 0 0 18 Nov 2023
How does the pre-training objective affect what large language models learn about linguistic properties? Ahmed Alajrami Nikolaos Aletras 26 20 0 20 Mar 2022
Efficient and Private Federated Learning with Partially Trainable Networks Hakim Sidahmed Zheng Xu Ankush Garg Yuan Cao Mingqing Chen FedML 49 13 0 06 Oct 2021
What's Hidden in a One-layer Randomly Weighted Transformer? Sheng Shen Z. Yao Douwe Kiela Kurt Keutzer Michael W. Mahoney 26 4 0 08 Sep 2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 45 243 0 14 Apr 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 280 2,015 0 28 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 240 4,469 0 23 Jan 2020
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 230 575 0 12 Sep 2019
Classical Structured Prediction Losses for Sequence to Sequence Learning Sergey Edunov Myle Ott Michael Auli David Grangier MarcÁurelio Ranzato AIMat 53 185 0 14 Nov 2017
Neuromorphic Deep Learning Machines Emre Neftci C. Augustine Somnath Paul Georgios Detorakis BDL 127 257 0 16 Dec 2016
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 255 13,364 0 25 Aug 2014