Chefs' Random Tables: Non-Trigonometric Random Features

30 May 2022

Valerii Likhosherstov

Papers citing "Chefs' Random Tables: Non-Trigonometric Random Features"

36 / 36 papers shown

Title
Ethical and social risks of harm from Language Models Laura Weidinger John F. J. Mellor Maribeth Rauh Conor Griffin J. Uesato ... Lisa Anne Hendricks William S. Isaac Sean Legassick G. Irving Iason Gabriel PILM 114 1,041 0 08 Dec 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 467 7,757 0 11 Nov 2021
On Learning the Transformer Kernel Sankalan Pal Chowdhury Adamos Solomou Kumar Avinava Dubey Mrinmaya Sachan ViT 89 14 0 15 Oct 2021
Hybrid Random Features K. Choromanski Haoxian Chen Han Lin Yuanzhe Ma Arijit Sehanobish ... Andy Zeng Valerii Likhosherstov Dmitry Kalashnikov Vikas Sindhwani Adrian Weller 59 21 0 08 Oct 2021
Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality Lukas Gonon 63 37 0 14 Jun 2021
Nonparametric adaptive control and prediction: theory and randomized algorithms Nicholas M. Boffi Stephen Tu Jean-Jacques E. Slotine 49 22 0 07 Jun 2021
Random Features for the Neural Tangent Kernel Insu Han H. Avron N. Shoham Chaewon Kim Jinwoo Shin 53 9 0 03 Apr 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown Basel Alomair Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 492 1,923 0 14 Dec 2020
Long Range Arena: A Benchmark for Efficient Transformers Yi Tay Mostafa Dehghani Samira Abnar Songlin Yang Dara Bahri Philip Pham J. Rao Liu Yang Sebastian Ruder Donald Metzler 147 720 0 08 Nov 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 657 41,103 0 22 Oct 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 184 1,597 0 30 Sep 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 546 2,086 0 28 Jul 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 201 1,771 0 29 Jun 2020
Array Programming with NumPy Charles R. Harris K. Millman S. Walt R. Gommers Pauli Virtanen ... Tyler Reddy Warren Weckesser Hameer Abbasi C. Gohlke T. Oliphant 154 14,959 0 18 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 216 1,706 0 08 Jun 2020
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Peter Hawkins Jared Davis David Belanger Lucy J. Colwell Adrian Weller 78 85 0 05 Jun 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 223 3,139 0 16 May 2020
Synthesizer: Rethinking Self-Attention in Transformer Models Yi Tay Dara Bahri Donald Metzler Da-Cheng Juan Zhe Zhao Che Zheng 61 336 0 02 May 2020
Sparse Sinkhorn Attention Yi Tay Dara Bahri Liu Yang Donald Metzler Da-Cheng Juan 86 340 0 26 Feb 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 192 2,320 0 13 Jan 2020
Deep Kernel Learning via Random Fourier Features Jiaxuan Xie Fanghui Liu Kaijie Wang Xiaolin Huang 34 19 0 07 Oct 2019
Energy and Policy Considerations for Deep Learning in NLP Emma Strubell Ananya Ganesh Andrew McCallum 69 2,657 0 05 Jun 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 129 1,899 0 23 Apr 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018
But How Does It Work in Theory? Linear SVM with Random Features Yitong Sun A. Gilbert Ambuj Tewari VLM 56 64 0 12 Sep 2018
Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees H. Avron Michael Kapralov Cameron Musco Christopher Musco A. Velingker A. Zandieh 68 156 0 26 Apr 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,182 0 20 Apr 2018
Image Transformer Niki Parmar Ashish Vaswani Jakob Uszkoreit Lukasz Kaiser Noam M. Shazeer Alexander Ku Dustin Tran ViT 138 1,680 0 15 Feb 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 713 131,652 0 12 Jun 2017
The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings K. Choromanski Mark Rowland Adrian Weller 84 85 0 02 Mar 2017
Operator-Valued Bochner Theorem, Fourier Feature Maps for Operator-Valued Kernels, and Vector-Valued Learning H. Q. Minh 66 18 0 19 Aug 2016
Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books Yukun Zhu Ryan Kiros R. Zemel Ruslan Salakhutdinov R. Urtasun Antonio Torralba Sanja Fidler 127 2,552 0 22 Jun 2015
On the Error of Random Fourier Features Danica J. Sutherland J. Schneider 78 192 0 09 Jun 2015
Optimal Rates for Random Fourier Features Bharath K. Sriperumbudur Z. Szabó 75 130 0 06 Jun 2015
Fast Function to Function Regression Junier Oliva Willie Neiswanger Barnabás Póczós Eric Xing J. Schneider 30 32 0 27 Oct 2014
Differentially Private Empirical Risk Minimization Kamalika Chaudhuri C. Monteleoni Anand D. Sarwate 137 1,487 0 01 Dec 2009