Gaussian Error Linear Units (GELUs)

27 June 2016

Papers citing "Gaussian Error Linear Units (GELUs)"

50 / 843 papers shown

Title
Geometric sparsification in recurrent neural networks Wyatt Mackey Ioannis Schizas Jared Deighton David L. Boothe, Jr. Vasileios Maroulas 33 0 0 10 Jun 2024
Aligning Agents like Large Language Models Adam Jelley Yuhan Cao Dave Bignell Sam Devlin Tabish Rashid LM&Ro 41 1 0 06 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 77 3 0 05 Jun 2024
Textless Acoustic Model with Self-Supervised Distillation for Noise-Robust Expressive Speech-to-Speech Translation Min-Jae Hwang Ilia Kulikov Benjamin Peloquin Hongyu Gong Peng-Jen Chen Ann Lee 32 1 0 04 Jun 2024
PixOOD: Pixel-Level Out-of-Distribution Detection Tomávs Vojívr Jan Sochman Jivrí Matas OODD 49 9 0 30 May 2024
Learning Diffeomorphism for Image Registration with Time-Continuous Networks using Semigroup Regularization Mohammadjavad Matinkia Nilanjan Ray MedIm 54 0 0 29 May 2024
Efficient Prior Calibration From Indirect Data O. Deniz Akyildiz Mark Girolami Andrew M. Stuart Arnaud Vadeboncoeur 38 1 0 28 May 2024
SoK: Leveraging Transformers for Malware Analysis Pradip Kunwar Kshitiz Aryal Maanak Gupta Mahmoud Abdelsalam Elisa Bertino 90 0 0 27 May 2024
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models Byung-Kwan Lee Chae Won Kim Beomchan Park Yonghyun Ro MLLM LRM 41 18 0 24 May 2024
Emergence of a High-Dimensional Abstraction Phase in Language Transformers Emily Cheng Diego Doimo Corentin Kervadec Iuri Macocco Jade Yu A. Laio Marco Baroni 112 11 0 24 May 2024
Infinite-Dimensional Feature Interaction Chenhui Xu Fuxun Yu Maoliang Li Zihao Zheng Zirui Xu Jinjun Xiong Xiang Chen 42 1 0 22 May 2024
Quantum Vision Transformers for Quark-Gluon Classification Marçal Comajoan Cara Gopal Ramesh Dahale Zhongtian Dong Roy T. Forestano S. Gleyzer ... Kyoungchul Kong Tom Magorsch Konstantin T. Matchev Katia Matcheva Eyup B. Unlu 48 9 0 16 May 2024
Tilt your Head: Activating the Hidden Spatial-Invariance of Classifiers Johann Schmidt Sebastian Stober 43 1 0 06 May 2024
Discretization Error of Fourier Neural Operators S. Lanthaler Andrew M. Stuart Margaret Trautner 45 4 0 03 May 2024
Multi-modal Learnable Queries for Image Aesthetics Assessment Zhiwei Xiong Yunfan Zhang Zhiqi Shen Peiran Ren Han Yu EGVM 35 1 0 02 May 2024
CUE-Net: Violence Detection Video Analytics with Spatial Cropping, Enhanced UniformerV2 and Modified Efficient Additive Attention Damith Chamalke Senadeera Xiaoyun Yang Dimitrios Kollias Gregory G. Slabaugh 32 0 0 27 Apr 2024
Sparse Reconstruction of Optical Doppler Tomography with Alternative State Space Model and Attention Zhenghong Li Jiaxiang Ren Wensheng Cheng C. Du Yingtian Pan Haibin Ling 50 0 0 26 Apr 2024
TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation Sai Kumar Dwivedi Yu Sun Priyanka Patel Yao Feng Michael J. Black 3DH 44 27 0 25 Apr 2024
Goal Exploration via Adaptive Skill Distribution for Goal-Conditioned Reinforcement Learning Lisheng Wu Ke Chen 29 0 0 19 Apr 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 54 1 0 16 Apr 2024
Contrastive Mean-Shift Learning for Generalized Category Discovery Sua Choi Dahyun Kang Minsu Cho 29 10 0 15 Apr 2024
PillarTrack:Boosting Pillar Representation for Transformer-based 3D Single Object Tracking on Point Clouds Weisheng Xu Sifan Zhou Jiaqi Xiong Ziyu Zhao Zhihang Yuan 45 2 0 11 Apr 2024
GAMA-IR: Global Additive Multidimensional Averaging for Fast Image Restoration Youssef Mansour Reinhard Heckel 40 0 0 31 Mar 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model Shujie Hu Long Zhou Shujie Liu Sanyuan Chen Hongkun Hao ... Xunying Liu Jinyu Li S. Sivasankaran Linquan Liu Furu Wei AuLLM 21 43 0 31 Mar 2024
IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions Zhijun Tu Kunpeng Du Hanting Chen Hai-lin Wang Wei Li Jie Hu Yunhe Wang ViT 44 4 0 31 Mar 2024
Nonlinearity Enhanced Adaptive Activation Functions David Yevick 25 1 0 29 Mar 2024
Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights Moein Heidari Reza Azad Sina Ghorbani Kolahi René Arimond Leon Niggemeier ... Afshin Bozorgpour Ehsan Khodapanah Aghdam A. Kazerouni I. Hacihaliloglu Dorit Merhof 51 7 0 28 Mar 2024
X-MIC: Cross-Modal Instance Conditioning for Egocentric Action Generalization Anna Kukleva Fadime Sener Edoardo Remelli Bugra Tekin Eric Sauser Bernt Schiele Shugao Ma VLM EgoV 45 1 0 28 Mar 2024
Fusion Transformer with Object Mask Guidance for Image Forgery Analysis Dimitrios Karageorgiou Giorgos Kordopatis-Zilos Symeon Papadopoulos ViT 28 5 0 18 Mar 2024
PAON: A New Neuron Model using Padé Approximants Onur Keleş A. Murat Tekalp 40 1 0 18 Mar 2024
MARVIS: Motion & Geometry Aware Real and Virtual Image Segmentation Jiayi Wu Xiao-sheng Lin S. Negahdaripour Cornelia Fermuller Yiannis Aloimonos 48 3 0 14 Mar 2024
A Survey of Vision Transformers in Autonomous Driving: Current Trends and Future Directions Quoc-Vinh Lai-Dang ViT 33 2 0 12 Mar 2024
Joint-Embedding Masked Autoencoder for Self-supervised Learning of Dynamic Functional Connectivity from the Human Brain Jungwon Choi Hyungi Lee Byung-Hoon Kim Juho Lee 80 0 0 11 Mar 2024
AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors Kaishen Yuan Zitong Yu Xin Liu Weicheng Xie Huanjing Yue Jingyu Yang ViT 31 12 0 07 Mar 2024
FriendNet: Detection-Friendly Dehazing Network Yihua Fan Yongzhen Wang Mingqiang Wei F. Wang H. Xie 41 4 0 07 Mar 2024
A Unified Framework for Microscopy Defocus Deblur with Multi-Pyramid Transformer and Contrastive Learning Yuelin Zhang Pengyu Zheng Wanquan Yan Chengyu Fang Shing Shin Cheng MedIm 37 7 0 05 Mar 2024
Explicit Motion Handling and Interactive Prompting for Video Camouflaged Object Detection Xin Zhang Tao Xiao Gepeng Ji Xuan Wu Keren Fu Qijun Zhao 55 2 0 04 Mar 2024
DRSI-Net: Dual-Residual Spatial Interaction Network for Multi-Person Pose Estimation Shang Wu Bin Wang 29 2 0 26 Feb 2024
Discovering Artificial Viscosity Models for Discontinuous Galerkin Approximation of Conservation Laws using Physics-Informed Machine Learning Matteo Caldana P. Antonietti Luca Dede' AI4CE PINN 32 1 0 26 Feb 2024
Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators Benedikt Alkin Andreas Fürst Simon Schmid Lukas Gruber Markus Holzleitner Johannes Brandstetter PINN AI4CE 48 8 0 19 Feb 2024
CoLLaVO: Crayon Large Language and Vision mOdel Byung-Kwan Lee Beomchan Park Chae Won Kim Yonghyun Ro VLM MLLM 32 16 0 17 Feb 2024
RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model Jianhao Yuan Shuyang Sun Daniel Omeiza Bo-Lu Zhao Paul Newman Lars Kunze Matthew Gadd LRM 36 48 0 16 Feb 2024
Only the Curve Shape Matters: Training Foundation Models for Zero-Shot Multivariate Time Series Forecasting through Next Curve Shape Prediction Cheng Feng Long Huang Denis Krompass AI4TS 23 5 0 12 Feb 2024
TEE4EHR: Transformer Event Encoder for Better Representation Learning in Electronic Health Records Hojjat Karami David Atienza Anisoara Ionescu AI4TS 36 1 0 09 Feb 2024
SoftEDA: Rethinking Rule-Based Data Augmentation with Soft Labels Juhwan Choi Kyohoon Jin Junho Lee Sang-hyŏn Song Youngbin Kim 13 5 0 08 Feb 2024
ReLU $^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs Zhengyan Zhang Yixin Song Guanghui Yu Xu Han Yankai Lin Chaojun Xiao Chenyang Song Zhiyuan Liu Zeyu Mi Maosong Sun 22 31 0 06 Feb 2024
DeepLag: Discovering Deep Lagrangian Dynamics for Intuitive Fluid Prediction Qilong Ma Haixu Wu Lanxiang Xing Jianmin Wang Mingsheng Long AI4CE 26 0 0 04 Feb 2024
Leveraging Continuously Differentiable Activation Functions for Learning in Quantized Noisy Environments Vivswan Shah Nathan Youngblood 41 2 0 04 Feb 2024
BAT: Learning to Reason about Spatial Sounds with Large Language Models Zhisheng Zheng Puyuan Peng Ziyang Ma Xie Chen Eunsol Choi David Harwath LRM 35 14 0 02 Feb 2024
OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning Chu Myaet Thwal Minh N. H. Nguyen Ye Lin Tun Seongjin Kim My T. Thai Choong Seon Hong 61 5 0 22 Jan 2024