Efficient Exact Gradient Update for training Deep Networks with Very
Large Sparse Targets

Efficient Exact Gradient Update for training Deep Networks with Very Large Sparse Targets

22 December 2014

Pascal Vincent

A. D. Brébisson

Xavier Bouthillier

Papers citing "Efficient Exact Gradient Update for training Deep Networks with Very Large Sparse Targets"

12 / 12 papers shown

Title
Sparse-softmax: A Simpler and Faster Alternative Softmax Transformation Shaoshi Sun Zhenyuan Zhang B. Huang Pengbin Lei Jianlin Su Shengfeng Pan Jiarun Cao UQCV 13 7 0 23 Dec 2021
Escaping the Gradient Vanishing: Periodic Alternatives of Softmax in Attention Mechanism Shulun Wang Bin Liu Feng Liu 25 16 0 16 Aug 2021
Optimal Approximation -- Smoothness Tradeoffs for Soft-Max Functions Alessandro Epasto Mohammad Mahdian Vahab Mirrokni Manolis Zampetakis 23 15 0 22 Oct 2020
Sampled Softmax with Random Fourier Features A. S. Rawat Jiecao Chen Felix X. Yu A. Suresh Sanjiv Kumar 36 55 0 24 Jul 2019
Improving Negative Sampling for Word Representation using Self-embedded Features Long Chen Fajie Yuan J. Jose Weinan Zhang SSL 26 43 0 26 Oct 2017
Deep Unsupervised Similarity Learning using Partially Ordered Sets Miguel Angel Bautista A. Sanakoyeu Bjorn Ommer DRL SSL 14 28 0 07 Apr 2017
Neural Machine Translation and Sequence-to-sequence Models: A Tutorial Graham Neubig AIMat 37 171 0 05 Mar 2017
Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation Yacine Jernite A. Choromańska David Sontag 25 35 0 14 Oct 2016
Efficient softmax approximation for GPUs Edouard Grave Armand Joulin Moustapha Cissé David Grangier Hervé Jégou 23 270 0 14 Sep 2016
The Z-loss: a shift and scale invariant classification loss belonging to the Spherical Family A. D. Brébisson Pascal Vincent 17 10 0 29 Apr 2016
BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabularies Shihao Ji S.V.N. Vishwanathan N. Satish Michael J. Anderson Pradeep Dubey 13 77 0 21 Nov 2015
An Exploration of Softmax Alternatives Belonging to the Spherical Loss Family A. D. Brébisson Pascal Vincent 11 97 0 16 Nov 2015