Language Modeling with Deep Transformers

10 May 2019

Papers citing "Language Modeling with Deep Transformers"

50 / 106 papers shown

Title
Paradigm Shift in Language Modeling: Revisiting CNN for Modeling Sanskrit Originated Bengali and Hindi Language C. R. Rahman Md. Hasibur Rahman Mohammad Rafsan S. Zakir Mohammed Eunus Ali Rafsanjani Muhammod 15 1 0 25 Oct 2021
The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization Róbert Csordás Kazuki Irie Jürgen Schmidhuber AI4CE 33 55 0 14 Oct 2021
On Language Model Integration for RNN Transducer based Speech Recognition Wei Zhou Zuoyun Zheng Ralf Schluter Hermann Ney 37 22 0 13 Oct 2021
Hierarchical Conditional End-to-End ASR with CTC and Multi-Granular Subword Units Yosuke Higuchi Keita Karube Tetsuji Ogawa Tetsunori Kobayashi 18 22 0 08 Oct 2021
ASR Rescoring and Confidence Estimation with ELECTRA Hayato Futami Hirofumi Inaguma Masato Mimura S. Sakai Tatsuya Kawahara KELM 62 20 0 05 Oct 2021
Private Language Model Adaptation for Speech Recognition Zhe Liu Ke Li Shreyan Bakshi Fuchun Peng 34 6 0 28 Sep 2021
Adapting GPT, GPT-2 and BERT Language Models for Speech Recognition Xianrui Zheng Chao Zhang P. Woodland 6 46 0 29 Jul 2021
The IWSLT 2021 BUT Speech Translation Systems Hari Krishna Vydana Martin Karafiát L. Burget J. Černocký 15 2 0 13 Jul 2021
Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformers Huahuan Zheng Wenjie Peng Zhijian Ou Jinsong Zhang 28 5 0 07 Jul 2021
Cross-Modal Transformer-Based Neural Correction Models for Automatic Speech Recognition Tomohiro Tanaka Ryo Masumura Mana Ihori Akihiko Takashima Takafumi Moriya Takanori Ashihara Shota Orihashi Naoki Makishima 16 7 0 04 Jul 2021
The DEformer: An Order-Agnostic Distribution Estimating Transformer Michael A. Alcorn Anh Totti Nguyen 11 4 0 13 Jun 2021
Cross-utterance Reranking Models with BERT and Graph Convolutional Networks for Conversational Speech Recognition Shih-Hsuan Chiu Tien-Hong Lo Fu-An Chao Berlin Chen BDL 33 10 0 13 Jun 2021
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers Kazuki Irie Imanol Schlag Róbert Csordás Jürgen Schmidhuber 33 57 0 11 Jun 2021
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 53 1,088 0 08 Jun 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat Fahad Shahbaz Khan Ming-Hsuan Yang ViT 265 621 0 21 May 2021
Relative Positional Encoding for Transformers with Linear Complexity Antoine Liutkus Ondřej Cífka Shih-Lun Wu Umut Simsekli Yi-Hsuan Yang Gaël Richard 33 44 0 18 May 2021
On Sampling-Based Training Criteria for Neural Language Modeling Yingbo Gao David Thulke Alexander Gerstenberger Viet Anh Khoa Tran Ralf Schluter Hermann Ney 19 2 0 21 Apr 2021
Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Models Mohammad Zeineldeen Aleksandr Glushko Wilfried Michel Albert Zeyer Ralf Schluter Hermann Ney AuLLM 11 39 0 12 Apr 2021
Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures Nick Rossenbach Mohammad Zeineldeen Benedikt Hilmes Ralf Schluter Hermann Ney 28 12 0 12 Apr 2021
On Architectures and Training for Raw Waveform Feature Extraction in ASR Peter Vieting Christoph Luscher Wilfried Michel Ralf Schluter Hermann Ney 30 9 0 09 Apr 2021
Capturing Multi-Resolution Context by Dilated Self-Attention Niko Moritz Takaaki Hori Jonathan Le Roux 19 7 0 07 Apr 2021
Attention, please! A survey of Neural Attention Models in Deep Learning Alana de Santana Correia Esther Luna Colombini HAI 23 175 0 31 Mar 2021
A Parallelizable Lattice Rescoring Strategy with Neural Language Models Ke Li Daniel Povey Sanjeev Khudanpur 13 16 0 08 Mar 2021
Linear Transformers Are Secretly Fast Weight Programmers Imanol Schlag Kazuki Irie Jürgen Schmidhuber 46 225 0 22 Feb 2021
Centroid Transformers: Learning to Abstract with Attention Lemeng Wu Xingchao Liu Qiang Liu 3DPC 61 28 0 17 Feb 2021
Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition Priyabrata Karmakar S. Teng Guojun Lu 27 25 0 14 Feb 2021
End-to-end Audio-visual Speech Recognition with Conformers Pingchuan Ma Stavros Petridis M. Pantic 84 225 0 12 Feb 2021
Transformer Language Models with LSTM-based Cross-utterance Information Representation G. Sun C. Zhang P. Woodland 76 32 0 12 Feb 2021
Bayesian Transformer Language Models for Speech Recognition Boyang Xue Jianwei Yu Junhao Xu Shansong Liu Shoukang Hu Zi Ye Mengzhe Geng Xunying Liu Helen Meng BDL 76 26 0 09 Feb 2021
baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotemporal Modeling Michael A. Alcorn Anh Totti Nguyen 17 19 0 05 Feb 2021
Text Augmentation for Language Models in High Error Recognition Scenario Karel Beneš L. Burget 21 3 0 11 Nov 2020
Warped Language Models for Noise Robust Language Understanding Mahdi Namazifar Gokhan Tur Dilek Z. Hakkani-Tür 17 7 0 03 Nov 2020
Multitask Learning and Joint Optimization for Transformer-RNN-Transducer Speech Recognition J. Jeon Eesung Kim 4 13 0 02 Nov 2020
Adaptable Multi-Domain Language Model for Transformer ASR Taewoo Lee Min-Joong Lee Tae Gyoon Kang Seokyeong Jung Minseok Kwon ... Ho-Gyeong Kim Jiseung Jeong Jihyun Lee Hosik Lee Y. S. Choi 19 17 0 14 Aug 2020
TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling Shuai Zhang Peng Zhang Xindian Ma Junqiu Wei Ning Wang Qun Liu 14 5 0 28 Jul 2020
Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR Balázs Tarján György Szaszák Tibor Fegyó P. Mihajlik 11 3 0 14 Jul 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Z. Chen MoE 43 1,108 0 30 Jun 2020
On the Effectiveness of Neural Text Generation based Data Augmentation for Recognition of Morphologically Rich Speech Balázs Tarján György Szaszák Tibor Fegyó P. Mihajlik 6 2 0 09 Jun 2020
Early Stage LM Integration Using Local and Global Log-Linear Combination Wilfried Michel Ralf Schluter Hermann Ney 11 11 0 20 May 2020
Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces Frank Zhang Yongqiang Wang Xiaohui Zhang Chunxi Liu Yatharth Saraf Geoffrey Zweig 10 20 0 19 May 2020
Contextualizing ASR Lattice Rescoring with Hybrid Pointer Network Language Model Da-Rong Liu Chunxi Liu Frank Zhang Gabriel Synnaeve Yatharth Saraf Geoffrey Zweig 28 19 0 15 May 2020
Research on Modeling Units of Transformer Transducer for Mandarin Speech Recognition Li Fu Xiaoxiao Li Libo Zi 22 5 0 26 Apr 2020
The RWTH ASR System for TED-LIUM Release 2: Improving Hybrid HMM with SpecAugment Wei Zhou Wilfried Michel Kazuki Irie M. Kitza Ralf Schluter Hermann Ney 11 42 0 02 Apr 2020
Code Prediction by Feeding Trees to Transformers Seohyun Kim Jinman Zhao Yuchi Tian S. Chandra 38 216 0 30 Mar 2020
Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation Necati Cihan Camgöz Oscar Koller Simon Hadfield Richard Bowden SLR 28 489 0 30 Mar 2020
Serialized Output Training for End-to-End Overlapped Speech Recognition Naoyuki Kanda Yashesh Gaur Xiaofei Wang Zhong Meng Takuya Yoshioka 6 113 0 28 Mar 2020
Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss Qian Zhang Han Lu Hasim Sak Anshuman Tripathi Erik McDermott Stephen Koo Shankar Kumar 8 474 0 07 Feb 2020
Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems Nick Rossenbach Albert Zeyer Ralf Schluter Hermann Ney 18 83 0 19 Dec 2019
Long-span language modeling for speech recognition S. Parthasarathy W. Gale Xie Chen George Polovets Shuangyu Chang RALM 6 10 0 11 Nov 2019
A Simplified Fully Quantized Transformer for End-to-end Speech Recognition Alex Bie Bharat Venkitesh João Monteiro Md. Akmal Haidar Mehdi Rezagholizadeh MQ 32 27 0 09 Nov 2019