Gaussian Error Linear Units (GELUs)

27 June 2016

Papers citing "Gaussian Error Linear Units (GELUs)"

50 / 886 papers shown

Title
ViViT: A Video Vision Transformer Anurag Arnab Mostafa Dehghani G. Heigold Chen Sun Mario Lucic Cordelia Schmid ViT 30 2,088 0 29 Mar 2021
Vision Transformers for Dense Prediction René Ranftl Alexey Bochkovskiy V. Koltun ViT MDE 45 1,662 0 24 Mar 2021
Scaling Local Self-Attention for Parameter Efficient Visual Backbones Ashish Vaswani Prajit Ramachandran A. Srinivas Niki Parmar Blake A. Hechtman Jonathon Shlens 27 395 0 23 Mar 2021
Scalable Vision Transformers with Hierarchical Pooling Zizheng Pan Bohan Zhuang Jing Liu Haoyu He Jianfei Cai ViT 27 126 0 19 Mar 2021
TransMed: Transformers Advance Multi-modal Medical Image Classification Yin Dai Yifan Gao ViT MedIm 38 280 0 10 Mar 2021
Pretrained Transformers as Universal Computation Engines Kevin Lu Aditya Grover Pieter Abbeel Igor Mordatch 28 217 0 09 Mar 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 91 976 0 04 Mar 2021
Fixing Data Augmentation to Improve Adversarial Robustness Sylvestre-Alvise Rebuffi Sven Gowal D. A. Calian Florian Stimberg Olivia Wiles Timothy A. Mann AAML 36 269 0 02 Mar 2021
Single-Shot Motion Completion with Transformer Yinglin Duan Tianyang Shi Zhengxia Zou Yenan Lin Zhehui Qian Bohan Zhang U. Michigan ViT 26 75 0 01 Mar 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 295 1,524 0 27 Feb 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 165 27,772 0 26 Feb 2021
TransMask: A Compact and Fast Speech Separation Model Based on Transformer Zining Zhang Bingsheng He Zhenjie Zhang 24 21 0 19 Feb 2021
Low Curvature Activations Reduce Overfitting in Adversarial Training Vasu Singla Sahil Singla David Jacobs S. Feizi AAML 32 45 0 15 Feb 2021
Radflow: A Recurrent, Aggregated, and Decomposable Model for Networks of Time Series Alasdair Tran A. Mathews Cheng Soon Ong Lexing Xie AI4TS AI4CE 21 12 0 15 Feb 2021
Optimizing Inference Performance of Transformers on CPUs D. Dice Alex Kogan 19 15 0 12 Feb 2021
VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention Peng Liu Yuewen Cao Songxiang Liu Na Hu Guangzhi Li Chao Weng Dan Su 42 22 0 12 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 223 512 0 11 Feb 2021
BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data Demetres Kostas Stephane Aroca-Ouellette Frank Rudzicz SSL 41 202 0 28 Jan 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 290 980 0 27 Jan 2021
E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials Simon L. Batzner Albert Musaelian Lixin Sun Mario Geiger J. Mailoa M. Kornbluth N. Molinari Tess E. Smidt Boris Kozinsky 233 1,240 0 08 Jan 2021
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 105 341 0 05 Jan 2021
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 24 126 0 31 Dec 2020
Transformer Interpretability Beyond Attention Visualization Hila Chefer Shir Gur Lior Wolf 45 644 0 17 Dec 2020
Trex: Learning Execution Semantics from Micro-Traces for Binary Similarity Kexin Pei Zhou Xuan Junfeng Yang Suman Jana Baishakhi Ray 24 88 0 16 Dec 2020
Exploring wav2vec 2.0 on speaker verification and language identification Zhiyun Fan Meng Li Shiyu Zhou Bo Xu 117 202 0 11 Dec 2020
Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection Dennis Ulmer Giovanni Cina OODD 35 31 0 09 Dec 2020
PlueckerNet: Learn to Register 3D Line Reconstructions Liu Liu Hongdong Li Haodong Yao Ruyi Zha 3DPC 3DV 25 6 0 02 Dec 2020
Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery Zhengyang Wang Meng Liu Youzhi Luo Zhao Xu Yaochen Xie ... Lei Cai Q. Qi Zhuoning Yuan Tianbao Yang Shuiwang Ji 36 100 0 02 Dec 2020
Generative Layout Modeling using Constraint Graphs W. Para Paul Guerrero Tom Kelly Leonidas J. Guibas Peter Wonka 31 68 0 26 Nov 2020
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images R. Child BDL VLM 56 337 0 20 Nov 2020
Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining Grigorii Guz Patrick Huber Giuseppe Carenini 35 11 0 06 Nov 2020
CharBERT: Character-aware Pre-trained Language Model Wentao Ma Yiming Cui Chenglei Si Ting Liu Shijin Wang Guoping Hu 31 104 0 03 Nov 2020
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning Simon Ging Mohammadreza Zolfaghari Hamed Pirsiavash Thomas Brox ViT CLIP 20 168 0 01 Nov 2020
Deep Learning is Singular, and That's Good Daniel Murfet Susan Wei Biwei Huang Hui Li Jesse Gell-Redman T. Quella UQCV 24 26 0 22 Oct 2020
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples Sven Gowal Chongli Qin J. Uesato Timothy A. Mann Pushmeet Kohli AAML 17 324 0 07 Oct 2020
Deep Learning in Diabetic Foot Ulcers Detection: A Comprehensive Evaluation Moi Hoon Yap Ryo Hachiuma A. Alavi Raphael Brüngel B. Cassidy ... David Gillespie N. Reeves Joseph M Pappachan C. O'Shea E. Frank FedML 21 122 0 07 Oct 2020
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Ikuya Yamada Akari Asai Hiroyuki Shindo Hideaki Takeda Yuji Matsumoto 22 662 0 02 Oct 2020
XDA: Accurate, Robust Disassembly with Transfer Learning Kexin Pei Jonas Guan David Williams-King Junfeng Yang Suman Jana 9 58 0 02 Oct 2020
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning Ye Liu Yao Wan Lifang He Hao Peng Philip S. Yu 29 188 0 26 Sep 2020
No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension Xuguang Wang Linjun Shou Ming Gong Nan Duan Daxin Jiang 24 12 0 25 Sep 2020
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers Jaemin Cho Jiasen Lu Dustin Schwenk Hannaneh Hajishirzi Aniruddha Kembhavi VLM MLLM 30 102 0 23 Sep 2020
Composed Variational Natural Language Generation for Few-shot Intents Congying Xia Caiming Xiong Philip Yu R. Socher VLM DRL 21 31 0 21 Sep 2020
SAPAG: A Self-Adaptive Privacy Attack From Gradients Yijue Wang Jieren Deng Danyi Guo Chenghong Wang Xianrui Meng Hang Liu Caiwen Ding Sanguthevar Rajasekaran 4 22 0 14 Sep 2020
Complexity Measures for Neural Networks with General Activation Functions Using Path-based Norms Zhong Li Chao Ma Lei Wu 28 24 0 14 Sep 2020
A Qualitative Study of the Dynamic Behavior for Adaptive Gradient Algorithms Chao Ma Lei Wu E. Weinan ODL 11 23 0 14 Sep 2020
Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems Zhe Chen Yuyan Wang Dong Lin D. Cheng Lichan Hong Ed H. Chi Claire Cui 28 16 0 17 Aug 2020
Can weight sharing outperform random architecture search? An investigation with TuNAS Gabriel Bender Hanxiao Liu Bo Chen Grace Chu Shuyang Cheng Pieter-Jan Kindermans Quoc V. Le OOD 15 121 0 13 Aug 2020
Aligning AI With Shared Human Values Dan Hendrycks Collin Burns Steven Basart Andrew Critch Jingkai Li D. Song Jacob Steinhardt 57 517 0 05 Aug 2020
Combining Deep Reinforcement Learning and Search for Imperfect-Information Games Noam Brown A. Bakhtin Adam Lerer Qucheng Gong 20 133 0 27 Jul 2020
Counterfactual Data Augmentation using Locally Factored Dynamics Silviu Pitis Elliot Creager Animesh Garg BDL OffRL 21 85 0 06 Jul 2020