ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1804.10959
  4. Cited By
Subword Regularization: Improving Neural Network Translation Models with
  Multiple Subword Candidates

Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

29 April 2018
Taku Kudo
ArXiv (abs)PDFHTML

Papers citing "Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates"

50 / 628 papers shown
Title
Zero-Shot Tokenizer Transfer
Zero-Shot Tokenizer Transfer
Benjamin Minixhofer
Edoardo Ponti
Ivan Vulić
VLM
83
13
0
13 May 2024
Fishing for Magikarp: Automatically Detecting Under-trained Tokens in
  Large Language Models
Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models
Sander Land
Max Bartolo
116
25
0
08 May 2024
Revisiting N-Gram Models: Their Impact in Modern Neural Networks for
  Handwritten Text Recognition
Revisiting N-Gram Models: Their Impact in Modern Neural Networks for Handwritten Text Recognition
Solène Tarride
Christopher Kermorvant
72
1
0
30 Apr 2024
A cost minimization approach to fix the vocabulary size in a tokenizer
  for an End-to-End ASR system
A cost minimization approach to fix the vocabulary size in a tokenizer for an End-to-End ASR system
Sunil Kumar Kopparapu
Ashish Panda
53
0
0
29 Apr 2024
Can Perplexity Predict Fine-tuning Performance? An Investigation of Tokenization Effects on Sequential Language Models for Nepali
Can Perplexity Predict Fine-tuning Performance? An Investigation of Tokenization Effects on Sequential Language Models for Nepali
Nishant Luitel
Nirajan Bekoju
Anand Kumar Sah
Subarna Shakya
112
1
0
28 Apr 2024
Act as a Honeytoken Generator! An Investigation into Honeytoken
  Generation with Large Language Models
Act as a Honeytoken Generator! An Investigation into Honeytoken Generation with Large Language Models
Daniel Reti
Norman Becker
Tillmann Angeli
Anasuya Chattopadhyay
Daniel Schneider
Sebastian Vollmer
Hans D. Schotten
77
5
0
24 Apr 2024
Evaluating Subword Tokenization: Alien Subword Composition and OOV
  Generalization Challenge
Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge
Khuyagbaatar Batsuren
Ekaterina Vylomova
Verna Dankers
Tsetsuukhei Delgerbaatar
Omri Uzan
Yuval Pinter
Gábor Bella
96
10
0
20 Apr 2024
Simultaneous Interpretation Corpus Construction by Large Language Models
  in Distant Language Pair
Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair
Yusuke Sakai
Mana Makinae
Hidetaka Kamigaito
Taro Watanabe
103
5
0
18 Apr 2024
On the Effect of (Near) Duplicate Subwords in Language Modelling
On the Effect of (Near) Duplicate Subwords in Language Modelling
Anton Schäfer
Thomas Hofmann
Imanol Schlag
Tiago Pimentel
70
2
0
09 Apr 2024
Training LLMs over Neurally Compressed Text
Training LLMs over Neurally Compressed Text
Brian Lester
Jaehoon Lee
A. Alemi
Jeffrey Pennington
Adam Roberts
Jascha Narain Sohl-Dickstein
Noah Constant
92
7
0
04 Apr 2024
Dynamic Neural Control Flow Execution: An Agent-Based Deep Equilibrium
  Approach for Binary Vulnerability Detection
Dynamic Neural Control Flow Execution: An Agent-Based Deep Equilibrium Approach for Binary Vulnerability Detection
Litao Li
Steven H. H. Ding
Andrew Walenstein
P. Charland
Benjamin C. M. Fung
58
0
0
03 Apr 2024
Revisiting subword tokenization: A case study on affixal negation in
  large language models
Revisiting subword tokenization: A case study on affixal negation in large language models
Thinh Hung Truong
Yulia Otmakhova
Karin Verspoor
Trevor Cohn
Timothy Baldwin
77
2
0
03 Apr 2024
Forklift: An Extensible Neural Lifter
Forklift: An Extensible Neural Lifter
Jordi Armengol-Estapé
Rodrigo C. O. Rocha
Jackson Woodruff
Pasquale Minervini
Michael F. P. O'Boyle
63
1
0
01 Apr 2024
An Analysis of BPE Vocabulary Trimming in Neural Machine Translation
An Analysis of BPE Vocabulary Trimming in Neural Machine Translation
Marco Cognetta
Tatsuya Hiraoka
Naoaki Okazaki
Rico Sennrich
Yuval Pinter
109
2
0
30 Mar 2024
A Systematic Analysis of Subwords and Cross-Lingual Transfer in
  Multilingual Translation
A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation
Francois Meyer
Jan Buys
90
1
0
29 Mar 2024
AlloyBERT: Alloy Property Prediction with Large Language Models
AlloyBERT: Alloy Property Prediction with Large Language Models
Akshat Chaudhari
Chakradhar Guntuboina
Hongshuo Huang
A. Farimani
119
4
0
28 Mar 2024
Homogeneous Tokenizer Matters: Homogeneous Visual Tokenizer for Remote
  Sensing Image Understanding
Homogeneous Tokenizer Matters: Homogeneous Visual Tokenizer for Remote Sensing Image Understanding
Run Shao
Zhaoyang Zhang
Chao Tao
Yunsheng Zhang
Chengli Peng
Haifeng Li
VLM
81
6
0
27 Mar 2024
Can Language Beat Numerical Regression? Language-Based Multimodal
  Trajectory Prediction
Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction
Inhwan Bae
Junoh Lee
Hae-Gon Jeon
105
22
0
27 Mar 2024
Provably Secure Disambiguating Neural Linguistic Steganography
Provably Secure Disambiguating Neural Linguistic Steganography
Yuang Qi
Kejiang Chen
Kai Zeng
Weiming Zhang
Neng H. Yu
43
5
0
26 Mar 2024
Cross-lingual Contextualized Phrase Retrieval
Cross-lingual Contextualized Phrase Retrieval
Huayang Li
Deng Cai
Zhi Qu
Qu Cui
Hidetaka Kamigaito
Lemao Liu
Taro Watanabe
56
0
0
25 Mar 2024
Synthetic Data Generation and Joint Learning for Robust Code-Mixed
  Translation
Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation
Kamal Kumar
Yinhan Liu
Parth Patwa
Tanmoy
Mihir Adam Roberts
95
2
0
25 Mar 2024
More than Just Statistical Recurrence: Human and Machine Unsupervised
  Learning of Māori Word Segmentation across Morphological Processes
More than Just Statistical Recurrence: Human and Machine Unsupervised Learning of Māori Word Segmentation across Morphological Processes
A. Varatharaj
Simon Todd
32
0
0
21 Mar 2024
Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced
  Arabic Language Models
Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Models
M. Alrefaie
Nour Eldin Morsy
Nada Samir
59
6
0
17 Mar 2024
Using Contextual Information for Sentence-level Morpheme Segmentation
Using Contextual Information for Sentence-level Morpheme Segmentation
Prabin Bhandari
Abhishek Paudel
28
1
0
15 Mar 2024
Token Alignment via Character Matching for Subword Completion
Token Alignment via Character Matching for Subword Completion
Ben Athiwaratkun
Shiqi Wang
Mingyue Shang
Yuchen Tian
Zijian Wang
Sujan Kumar Gonugondla
Sanjay Krishna Gouda
Rob Kwiatowski
Ramesh Nallapati
Bing Xiang
99
6
0
13 Mar 2024
Triples-to-isiXhosa (T2X): Addressing the Challenges of Low-Resource
  Agglutinative Data-to-Text Generation
Triples-to-isiXhosa (T2X): Addressing the Challenges of Low-Resource Agglutinative Data-to-Text Generation
Francois Meyer
Jan Buys
57
2
0
12 Mar 2024
MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki
MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki
Timothee Mickus
Stig-Arne Gronroos
Joseph Attieh
M. Boggia
Ona de Gibert
Shaoxiong Ji
Niki Andreas Lopi
Alessandro Raganato
Raúl Vázquez
Jörg Tiedemann
57
4
0
12 Mar 2024
Unpacking Tokenization: Evaluating Text Compression and its Correlation
  with Model Performance
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance
Omer Goldman
Avi Caciularu
Matan Eyal
Kris Cao
Idan Szpektor
Reut Tsarfaty
115
31
0
10 Mar 2024
Authorship Attribution in Bangla Literature (AABL) via Transfer Learning
  using ULMFiT
Authorship Attribution in Bangla Literature (AABL) via Transfer Learning using ULMFiT
Aisha Khatun
Anisur Rahman
Md. Saiful Islam
Hemayet Ahmed Chowdhury
A. Tasnim
69
3
0
08 Mar 2024
Did Translation Models Get More Robust Without Anyone Even Noticing?
Did Translation Models Get More Robust Without Anyone Even Noticing?
Ben Peters
André F. T. Martins
66
3
0
06 Mar 2024
A Generative Approach for Wikipedia-Scale Visual Entity Recognition
A Generative Approach for Wikipedia-Scale Visual Entity Recognition
Mathilde Caron
Ahmet Iscen
Alireza Fathi
Cordelia Schmid
94
5
0
04 Mar 2024
Transformers for Low-Resource Languages:Is Féidir Linn!
Transformers for Low-Resource Languages:Is Féidir Linn!
Séamus Lankford
H. Alfi
Tamás Sarlós
93
17
0
04 Mar 2024
Language and Speech Technology for Central Kurdish Varieties
Language and Speech Technology for Central Kurdish Varieties
Sina Ahmadi
Daban Q. Jaff
Md Mahfuz Ibn Alam
Antonios Anastasopoulos
99
2
0
04 Mar 2024
adaptNMT: an open-source, language-agnostic development environment for
  Neural Machine Translation
adaptNMT: an open-source, language-agnostic development environment for Neural Machine Translation
Séamus Lankford
Haithem Afli
Andy Way
73
3
0
04 Mar 2024
Human Evaluation of English--Irish Transformer-Based NMT
Human Evaluation of English--Irish Transformer-Based NMT
Séamus Lankford
Haithem Afli
Andy Way
124
11
0
04 Mar 2024
VBART: The Turkish LLM
VBART: The Turkish LLM
Meliksah Turker
Mehmet Erdi Ari
Aydin Han
VLM
44
4
0
02 Mar 2024
Greed is All You Need: An Evaluation of Tokenizer Inference Methods
Greed is All You Need: An Evaluation of Tokenizer Inference Methods
Omri Uzan
Craig W. Schmidt
Chris Tanner
Yuval Pinter
79
17
0
02 Mar 2024
Rethinking Tokenization: Crafting Better Tokenizers for Large Language
  Models
Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models
Jinbiao Yang
LLMAG
169
11
0
01 Mar 2024
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent
  on Language Models
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
Frederik Kunstner
Robin Yadav
Alan Milligan
Mark Schmidt
Alberto Bietti
95
34
0
29 Feb 2024
Beyond Language Models: Byte Models are Digital World Simulators
Beyond Language Models: Byte Models are Digital World Simulators
Shangda Wu
Xu Tan
Zili Wang
Rui Wang
Xiaobing Li
Maosong Sun
65
13
0
29 Feb 2024
CEBin: A Cost-Effective Framework for Large-Scale Binary Code Similarity
  Detection
CEBin: A Cost-Effective Framework for Large-Scale Binary Code Similarity Detection
Hao Wang
Zeyu Gao
Chao Zhang
Mingyang Sun
Yuchen Zhou
Han Qiu
Xiangwei Xiao
65
13
0
29 Feb 2024
Tokenization Is More Than Compression
Tokenization Is More Than Compression
Craig W. Schmidt
Varshini Reddy
Haoran Zhang
Alec Alameddine
Omri Uzan
Yuval Pinter
Chris Tanner
124
38
0
28 Feb 2024
Natural Language Processing Methods for Symbolic Music Generation and
  Information Retrieval: a Survey
Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey
Dinh-Viet-Toan Le
Louis Bigo
Mikaela Keller
Dorien Herremans
MedIm
91
14
0
27 Feb 2024
CLAP: Learning Transferable Binary Code Representations with Natural
  Language Supervision
CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision
Hao Wang
Zeyu Gao
Chao Zhang
Zihan Sha
Mingyang Sun
Yuchen Zhou
Wenyu Zhu
Wenju Sun
Han Qiu
Xiangwei Xiao
92
22
0
26 Feb 2024
How Important Is Tokenization in French Medical Masked Language Models?
How Important Is Tokenization in French Medical Masked Language Models?
Yanis Labrak
Adrien Bazoge
B. Daille
Mickael Rouvier
Richard Dufour
66
1
0
22 Feb 2024
Tokenization counts: the impact of tokenization on arithmetic in
  frontier LLMs
Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs
Aaditya K. Singh
DJ Strouse
116
61
0
22 Feb 2024
The Impact of Word Splitting on the Semantic Content of Contextualized
  Word Representations
The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations
Aina Garí Soler
Matthieu Labeau
Chloé Clavel
VLM
75
2
0
22 Feb 2024
Two Counterexamples to Tokenization and the Noiseless Channel
Two Counterexamples to Tokenization and the Noiseless Channel
Marco Cognetta
Vilém Zouhar
Sangwhan Moon
Naoaki Okazaki
51
0
0
22 Feb 2024
Subobject-level Image Tokenization
Subobject-level Image Tokenization
Delong Chen
Samuel Cahyawijaya
Jianfeng Liu
Baoyuan Wang
Pascale Fung
VLMOCL
278
9
0
22 Feb 2024
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand
  for Multilingual Instructions?
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?
Alexander Arno Weber
Klaudia Thellmann
Jan Ebert
Nicolas Flores-Herr
Jens Lehmann
Michael Fromm
Mehdi Ali
72
5
0
21 Feb 2024
Previous
123456...111213
Next