SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

19 August 2018

Papers citing "SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing"

50 / 1,950 papers shown

Title
Unified Knowledge Prompt Pre-training for Customer Service Dialogues Keqing He Jingang Wang Chaobo Sun Wei Wu 64 4 0 31 Aug 2022
CJaFr-v3 : A Freely Available Filtered Japanese-French Aligned Corpus Raoul Blin Fabien Cromierès 30 1 0 28 Aug 2022
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation Nataniel Ruiz Yuanzhen Li Varun Jampani Yael Pritch Michael Rubinstein Kfir Aberman 304 2,904 0 25 Aug 2022
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Wenhui Wang Hangbo Bao Li Dong Johan Bjorck Zhiliang Peng ... Kriti Aggarwal O. Mohammed Saksham Singhal Subhojit Som Furu Wei MLLM VLM ViT 157 645 0 22 Aug 2022
A Novel Multi-Task Learning Approach for Context-Sensitive Compound Type Identification in Sanskrit Jivnesh Sandhan Ashish Gupta Hrishikesh Terdalkar Tushar Sandhan S. Samanta Laxmidhar Behera Pawan Goyal 73 4 0 22 Aug 2022
A Syntax Aware BERT for Identifying Well-Formed Queries in a Curriculum Framework Avinash Madasu Anvesh Rao Vijjini 32 0 0 21 Aug 2022
Uconv-Conformer: High Reduction of Input Sequence Length for End-to-End Speech Recognition A. Andrusenko R. Nasretdinov A. Romanenko 85 18 0 16 Aug 2022
Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages Paul Soulos Sudha Rao Caitlin Smith Eric Rosen Asli Celikyilmaz ... Coleman Haley Roland Fernandez Hamid Palangi Jianfeng Gao P. Smolensky 72 7 0 11 Aug 2022
Domain-Specific Text Generation for Machine Translation Yasmin Moslem Rejwanul Haque John D. Kelleher Andy Way 64 18 0 11 Aug 2022
Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation Muhammad N. ElNokrashy Amr Hendy Mohamed Maher Mohamed Afify Hany Awadalla 62 2 0 11 Aug 2022
Reducing Retraining by Recycling Parameter-Efficient Prompts Brian Lester Joshua Yurtsever Siamak Shakeri Noah Constant 51 12 0 10 Aug 2022
The Analysis about Building Cross-lingual Sememe Knowledge Base Based on Deep Clustering Network Xiaoran Li Toshiaki Takano 22 0 0 10 Aug 2022
Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation Nuno M. Guerreiro Elena Voita André F. T. Martins HILM 78 57 0 10 Aug 2022
How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation? Ali Araabi Christof Monz Vlad Niculae 56 10 0 10 Aug 2022
A High-Quality and Large-Scale Dataset for English-Vietnamese Speech Translation L. T. Nguyen Nguyen Luong Tran Long Doan Manh Luong Dat Quoc Nguyen 62 4 0 08 Aug 2022
Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning Ting-Li Chen Ruixiang Zhang Geoffrey E. Hinton DiffM 132 313 0 08 Aug 2022
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model Saleh Soltan Shankar Ananthakrishnan Jack G. M. FitzGerald Rahul Gupta Wael Hamza ... Mukund Sridhar Fabian Triefenbach Apurv Verma Gokhan Tur Premkumar Natarajan 129 83 0 02 Aug 2022
Pronunciation-aware unique character encoding for RNN Transducer-based Mandarin speech recognition Peng Shen Xugang Lu Hisashi Kawai 35 2 0 29 Jul 2022
Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attention Mengsay Loem Sho Takase Masahiro Kaneko Naoaki Okazaki 39 1 0 27 Jul 2022
SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation Artem Ploujnikov Mirco Ravanelli 26 18 0 27 Jul 2022
Learning a Dual-Mode Speech Recognition Model via Self-Pruning Chunxi Liu Yuan Shangguan Haichuan Yang Yangyang Shi Raghuraman Krishnamoorthi Ozlem Kalinli SSL 87 7 0 25 Jul 2022
PanGu-Coder: Program Synthesis with Function-Level Language Modeling Fenia Christopoulou Gerasimos Lampouras Milan Gritta Guchun Zhang Yinpeng Guo ... Guangtai Liang Jia Wei Xin Jiang Qianxiang Wang Qun Liu ELM SyDa ALM 109 76 0 22 Jul 2022
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling? Yi Tay Mostafa Dehghani Samira Abnar Hyung Won Chung W. Fedus J. Rao Sharan Narang Vinh Q. Tran Dani Yogatama Donald Metzler AI4CE 122 107 0 21 Jul 2022
Deep equilibrium networks are sensitive to initialization statistics Atish Agarwala S. Schoenholz 93 7 0 19 Jul 2022
MAD for Robust Reinforcement Learning in Machine Translation Domenic Donato Lei Yu Wang Ling Chris Dyer MoE 52 7 0 18 Jul 2022
Knowledge Transfer and Distillation from Autoregressive to Non-Autoregressive Speech Recognition Xun Gong Zhikai Zhou Y. Qian 138 4 0 15 Jul 2022
Language Modelling with Pixels Phillip Rust Jonas F. Lotz Emanuele Bugliarello Elizabeth Salesky Miryam de Lhoneux Desmond Elliott VLM 107 46 0 14 Jul 2022
Visual Context-driven Audio Feature Enhancement for Robust End-to-End Audio-Visual Speech Recognition Joanna Hong Minsu Kim Daehun Yoo Y. Ro 62 21 0 13 Jul 2022
Sockeye 3: Fast Neural Machine Translation with PyTorch Felix Hieber Michael J. Denkowski Tobias Domhan Barbara Darques Barros Celina Dong Ye ... Maria Nadejde Surafel Melaku Lakew Prashant Mathur Anna Currey Marcello Federico OSLM 88 10 0 12 Jul 2022
End-to-end speech recognition modeling from de-identified data M. Flechl Shou-Chun Yin Junho Park Peter Skala 44 5 0 12 Jul 2022
Speaker Anonymization with Phonetic Intermediate Representations Sarina Meyer Florian Lux Pavel Denisov Julia Koch Pascal Tilli Ngoc Thang Vu 83 28 0 11 Jul 2022
End-to-end Speech-to-Punctuated-Text Recognition Jumon Nozaki Tatsuya Kawahara K. Ishizuka Taiichi Hashimoto 65 12 0 07 Jul 2022
Cross-Lingual QA as a Stepping Stone for Monolingual Open QA in Icelandic Vésteinn Snæbjarnarson H. Einarsson 57 6 0 05 Jul 2022
Generating Repetitions with Appropriate Repeated Words Toshiki Kawamoto Hidetaka Kamigaito Kotaro Funakoshi Manabu Okumura 36 3 0 03 Jul 2022
Improving Transformer-based Conversational ASR by Inter-Sentential Attention Mechanism Kun Wei Pengcheng Guo Ning Jiang 82 11 0 02 Jul 2022
Reduce Indonesian Vocabularies with an Indonesian Sub-word Separator Mukhlis Amien Chong Feng Heyan Huang 34 0 0 01 Jul 2022
Learning Job Titles Similarity from Noisy Skill Labels Rabih Zbib Lucas Lacasa Alvarez Federico Retyk Rus Poves Juan Aizpuru H. Fabregat Vaidotas Šimkus Emília Garcia Casademont 63 3 0 01 Jul 2022
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations Akiko Eriguchi Shufang Xie Tao Qin Hany Awadalla LRM 91 8 0 30 Jun 2022
Finstreder: Simple and fast Spoken Language Understanding with Finite State Transducers using modern Speech-to-Text models Daniel Bermuth Alexander Poeppel Wolfgang Reif 66 7 0 29 Jun 2022
On Comparison of Encoders for Attention based End to End Speech Recognition in Standalone and Rescoring Mode Raviraj Joshi Subodh Kumar 75 2 0 26 Jun 2022
Pruned RNN-T for fast, memory-efficient ASR training Fangjun Kuang Liyong Guo Wei Kang Long Lin Mingshuang Luo Zengwei Yao Daniel Povey 101 69 0 23 Jun 2022
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation Jiahui Yu Yuanzhong Xu Jing Yu Koh Thang Luong Gunjan Baid ... Zarana Parekh Xin Li Han Zhang Jason Baldridge Yonghui Wu EGVM 223 1,134 0 22 Jun 2022
Learning Multiscale Transformer Models for Sequence Generation Bei Li Tong Zheng Yi Jing Chengbo Jiao Tong Xiao Jingbo Zhu 70 9 0 19 Jun 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 171 412 0 17 Jun 2022
Know your audience: specializing grounded language models with listener subtraction Aaditya K. Singh David Ding Andrew M. Saxe Felix Hill Andrew Kyle Lampinen 65 2 0 16 Jun 2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid 149 239 0 16 Jun 2022
Text normalization for low-resource languages: the case of Ligurian S. Lusito Edoardo Ferrante Jean Maillard 52 6 0 16 Jun 2022
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems Jack G. M. FitzGerald Shankar Ananthakrishnan Konstantine Arkoudas Davide Bernardi Abhishek Bhagia ... Pan Wei Haiyang Yu Shuai Zheng Gokhan Tur Premkumar Natarajan ELM 46 30 0 15 Jun 2022
Language Models are General-Purpose Interfaces Y. Hao Haoyu Song Li Dong Shaohan Huang Zewen Chi Wenhui Wang Shuming Ma Furu Wei MLLM 78 102 0 13 Jun 2022
Revisiting End-to-End Speech-to-Text Translation From Scratch Biao Zhang Barry Haddow Rico Sennrich 81 39 0 09 Jun 2022