ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1804.10959
  4. Cited By
Subword Regularization: Improving Neural Network Translation Models with
  Multiple Subword Candidates

Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

29 April 2018
Taku Kudo
ArXiv (abs)PDFHTML

Papers citing "Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates"

50 / 628 papers shown
Title
Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
Gyeongje Cho
Yeonkyoun So
Chanwoo Park
Sangmin Lee
Sungmok Jung
Jaejin Lee
VLM
30
0
0
18 Jun 2025
Detecting Hard-Coded Credentials in Software Repositories via LLMs
Detecting Hard-Coded Credentials in Software Repositories via LLMs
Chidera Biringa
Gökhan Kul
31
0
0
16 Jun 2025
One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers
One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers
Diana Abagyan
Alejandro Salamanca
Andres Felipe Cruz-Salinas
Kris Cao
Hangyu Lin
Acyr Locatelli
Marzieh Fadaee
Ahmet Üstün
Sara Hooker
CLL
135
0
0
12 Jun 2025
Canonical Autoregressive Generation
Canonical Autoregressive Generation
Ivi Chatzi
N. C. Benz
Stratis Tsirtsis
Manuel Gomez Rodriguez
17
0
0
06 Jun 2025
ViCocktail: Automated Multi-Modal Data Collection for Vietnamese Audio-Visual Speech Recognition
Thai-Binh Nguyen
T. Nguyen
Quoc Truong Do
Chi Mai Luong
88
0
0
05 Jun 2025
TokAlign: Efficient Vocabulary Adaptation via Token Alignment
TokAlign: Efficient Vocabulary Adaptation via Token Alignment
Chong Li
Jiajun Zhang
Chengqing Zong
VLM
59
0
0
04 Jun 2025
Beyond Text Compression: Evaluating Tokenizers Across Scales
Beyond Text Compression: Evaluating Tokenizers Across Scales
Jonas F. Lotz
António V. Lopes
Stephan Peitz
Hendra Setiawan
Leonardo Emili
61
0
0
03 Jun 2025
Leveraging Natural Language Processing to Unravel the Mystery of Life: A Review of NLP Approaches in Genomics, Transcriptomics, and Proteomics
Leveraging Natural Language Processing to Unravel the Mystery of Life: A Review of NLP Approaches in Genomics, Transcriptomics, and Proteomics
Ella Rannon
David Burstein
AI4TS
29
0
0
02 Jun 2025
StochasTok: Improving Fine-Grained Subword Understanding in LLMs
StochasTok: Improving Fine-Grained Subword Understanding in LLMs
Anya Sims
Thom Foster
Klara Kaleb
Tuan-Duy H. Nguyen
Joseph Lee
Jakob N. Foerster
Yee Whye Teh
Cong Lu
94
1
0
02 Jun 2025
The Warmup Dilemma: How Learning Rate Strategies Impact Speech-to-Text Model Convergence
The Warmup Dilemma: How Learning Rate Strategies Impact Speech-to-Text Model Convergence
Marco Gaido
Sara Papi
L. Bentivogli
Alessio Brutti
Mauro Cettolo
R. Gretter
M. Matassoni
Mohamed Nabih
Matteo Negri
47
0
0
29 May 2025
Spontaneous Speech Variables for Evaluating LLMs Cognitive Plausibility
Spontaneous Speech Variables for Evaluating LLMs Cognitive Plausibility
Sheng-Fu Wang
Laurent Prevot
Jou-an Chi
Ri-Sheng Huang
Shu-Kai Hsieh
LRM
67
0
0
22 May 2025
BanglaByT5: Byte-Level Modelling for Bangla
BanglaByT5: Byte-Level Modelling for Bangla
Pramit Bhattacharyya
Arnab Bhattacharya
47
0
0
21 May 2025
GIF: Generative Inspiration for Face Recognition at Scale
GIF: Generative Inspiration for Face Recognition at Scale
Saeed Ebrahimi
Sahar Rahimi
Ali Dabouei
Srinjoy Das
Jeremy M. Dawson
Nasser M. Nasrabadi
CVBM
554
0
0
05 May 2025
An Empirical Study on the Effectiveness of Large Language Models for Binary Code Understanding
An Empirical Study on the Effectiveness of Large Language Models for Binary Code Understanding
Xiuwei Shang
Zhenkan Fu
Shaoyin Cheng
Guoqiang Chen
Gangyang Li
Li Hu
Weinan Zhang
N. Yu
98
0
0
30 Apr 2025
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
Enes Özeren
Yihong Liu
Hinrich Schütze
69
0
0
21 Apr 2025
TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
Liang-Hsuan Tseng
Yi-Chang Chen
Kuan-Yi Lee
Da-shan Shiu
Hung-yi Lee
AuLLM
161
0
0
09 Apr 2025
Limitations of Religious Data and the Importance of the Target Domain: Towards Machine Translation for Guinea-Bissau Creole
Limitations of Religious Data and the Importance of the Target Domain: Towards Machine Translation for Guinea-Bissau Creole
Jacqueline Rowe
Edward Gow-Smith
Mark Hepple
112
0
0
03 Apr 2025
Overcoming Vocabulary Constraints with Pixel-level Fallback
Overcoming Vocabulary Constraints with Pixel-level Fallback
Jonas F. Lotz
Hendra Setiawan
Stephan Peitz
Yova Kementchedjhieva
100
1
0
02 Apr 2025
From Smør-re-brød to Subwords: Training LLMs on Danish, One Morpheme at a Time
From Smør-re-brød to Subwords: Training LLMs on Danish, One Morpheme at a Time
Mikkel Wildner Kildeberg
Emil Allerslev Schledermann
Nicolaj Larsen
Rob van der Goot
78
0
0
02 Apr 2025
Tokenization of Gaze Data
Tokenization of Gaze Data
Tim Rolff
Jurik Karimian
Niklas Hypki
S. Schmidt
Markus Lappe
Frank Steinicke
118
0
0
28 Mar 2025
ParsiPy: NLP Toolkit for Historical Persian Texts in Python
ParsiPy: NLP Toolkit for Historical Persian Texts in Python
Farhan Farsi
Parnian Fazel
Sepand Haghighi
Sadra Sabouri
Farzaneh Goshtasb
Nadia Hajipour
Ehsaneddin Asgari
Hossein Sameti
68
0
0
22 Mar 2025
Splintering Nonconcatenative Languages for Better Tokenization
Splintering Nonconcatenative Languages for Better Tokenization
Bar Gazit
Shaltiel Shmidman
Avi Shmidman
Yuval Pinter
106
0
0
18 Mar 2025
Annotating and Inferring Compositional Structures in Numeral Systems Across Languages
Arne Rubehn
Christoph Rzymski
Luca Ciucci
Kellen Parker van Dam
Alžběta Kučerová
Katja Bocklage
David Snee
Abishek Stephen
Johann-Mattis List
73
0
0
03 Mar 2025
Scaling Laws for Downstream Task Performance in Machine Translation
Scaling Laws for Downstream Task Performance in Machine Translation
Berivan Isik
Natalia Ponomareva
Hussein Hazimeh
Dimitris Paparas
Sergei Vassilvitskii
Sanmi Koyejo
165
23
0
24 Feb 2025
Deterministic Reversible Data Augmentation for Neural Machine Translation
Deterministic Reversible Data Augmentation for Neural Machine Translation
Jiashu Yao
Heyan Huang
Zeming Liu
Yuhang Guo
172
0
0
21 Feb 2025
PLDR-LLMs Learn A Generalizable Tensor Operator That Can Replace Its Own Deep Neural Net At Inference
PLDR-LLMs Learn A Generalizable Tensor Operator That Can Replace Its Own Deep Neural Net At Inference
Burc Gokden
118
0
0
19 Feb 2025
TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking
TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking
Shahriar Kabir Nahin
R. N. Nandi
Sagor Sarker
Quazi Sarwar Muhtaseem
Md. Kowsher
Apu Chandraw Shill
Md Ibrahim
Mehadi Hasan Menon
Tareq Al Muntasir
Firoj Alam
189
0
0
16 Feb 2025
Enhancing LLM Character-Level Manipulation via Divide and Conquer
Enhancing LLM Character-Level Manipulation via Divide and Conquer
Zhen Xiong
Yujun Cai
Bryan Hooi
Nanyun Peng
Kai-Wei Chang
Zhecheng Li
162
0
0
12 Feb 2025
Beyond Literal Token Overlap: Token Alignability for Multilinguality
Beyond Literal Token Overlap: Token Alignability for Multilinguality
Katharina Hämmerl
Tomasz Limisiewicz
Jindrich Libovický
Alexander Fraser
73
0
0
10 Feb 2025
Scaling Embedding Layers in Language Models
Scaling Embedding Layers in Language Models
Da Yu
Edith Cohen
Badih Ghazi
Yangsibo Huang
Pritish Kamath
Ravi Kumar
Daogao Liu
Chiyuan Zhang
172
1
0
03 Feb 2025
Efficient Language Modeling for Low-Resource Settings with Hybrid RNN-Transformer Architectures
Efficient Language Modeling for Low-Resource Settings with Hybrid RNN-Transformer Architectures
Gabriel Lindenmaier
Sean Papay
Sebastian Padó
154
0
0
02 Feb 2025
Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
Nadav Timor
Jonathan Mamou
Daniel Korat
Moshe Berchansky
Oren Pereg
Gaurav Jain
Roy Schwartz
Moshe Wasserblat
189
3
0
31 Jan 2025
Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
Go Kamoda
Benjamin Heinzerling
Tatsuro Inaba
Keito Kudo
Keisuke Sakaguchi
Kentaro Inui
MILM
133
3
0
27 Jan 2025
Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability Graphs
Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability Graphs
Amirmohammad Farzaneh
Osvaldo Simeone
520
0
0
22 Jan 2025
Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions
Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions
Doaa Mahmud
Hadeel Hajmohamed
Shamma Almentheri
Shamma Alqaydi
Lameya Aldhaheri
R. A. Khalil
Nasir Saeed
AI4TS
99
12
0
08 Jan 2025
LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
LLM-jp
Akiko Aizawa
Eiji Aramaki
Bowen Chen
Fei Cheng
...
Yuya Yamamoto
Yusuke Yamauchi
Hitomi Yanaka
Rio Yokota
Koichiro Yoshino
111
17
0
31 Dec 2024
Mask Factory: Towards High-quality Synthetic Data Generation for
  Dichotomous Image Segmentation
Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation
Haotian Qian
YD Chen
Shengtao Lou
Fahad Shahbaz Khan
Xiaogang Jin
Deng-Ping Fan
DiffM
83
6
0
26 Dec 2024
The Language of Motion: Unifying Verbal and Non-verbal Language of 3D
  Human Motion
The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion
Changan Chen
Juze Zhang
S. K. Lakshmikanth
Yusu Fang
Ruizhi Shao
Gordon Wetzstein
L. Fei-Fei
Ehsan Adeli
VGen
135
5
0
13 Dec 2024
Efficient Continual Pre-training of LLMs for Low-resource Languages
Efficient Continual Pre-training of LLMs for Low-resource Languages
Arijit Nag
Soumen Chakrabarti
Animesh Mukherjee
Niloy Ganguly
117
1
0
13 Dec 2024
Multi-Head Encoding for Extreme Label Classification
Multi-Head Encoding for Extreme Label Classification
Daojun Liang
Haixia Zhang
Dongfeng Yuan
Minggao Zhang
118
0
0
13 Dec 2024
From Language Models over Tokens to Language Models over Characters
From Language Models over Tokens to Language Models over Characters
Tim Vieira
Ben LeBrun
Mario Giulianelli
Juan Luis Gastaldi
Brian DuSell
John Terilla
Timothy J. O'Donnell
Ryan Cotterell
156
13
0
04 Dec 2024
Linguistic Laws Meet Protein Sequences: A Comparative Analysis of
  Subword Tokenization Methods
Linguistic Laws Meet Protein Sequences: A Comparative Analysis of Subword Tokenization Methods
Burak Suyunu
Enes Taylan
Arzucan Özgür
99
3
0
26 Nov 2024
Xmodel-1.5: An 1B-scale Multilingual LLM
Xmodel-1.5: An 1B-scale Multilingual LLM
Wang Qun
Liu Yang
Lin Qingquan
Jiang Ling
LRM
71
0
0
15 Nov 2024
Number Cookbook: Number Understanding of Language Models and How to Improve It
Number Cookbook: Number Understanding of Language Models and How to Improve It
Haotong Yang
Yi Hu
Shijia Kang
Zhouchen Lin
Muhan Zhang
LRM
112
8
0
06 Nov 2024
MoCE: Adaptive Mixture of Contextualization Experts for Byte-based Neural Machine Translation
MoCE: Adaptive Mixture of Contextualization Experts for Byte-based Neural Machine Translation
Langlin Huang
Mengyu Bu
Yang Feng
101
0
0
03 Nov 2024
Morphological Typology in BPE Subword Productivity and Language Modeling
Morphological Typology in BPE Subword Productivity and Language Modeling
Iñigo Parra
61
0
0
31 Oct 2024
PLDR-LLM: Large Language Model from Power Law Decoder Representations
PLDR-LLM: Large Language Model from Power Law Decoder Representations
Burc Gokden
59
1
0
22 Oct 2024
Neural Search Space in Gboard Decoder
Neural Search Space in Gboard Decoder
Yanxiang Zhang
Yuanbo Zhang
Haicheng Sun
Yun Wang
Billy Dou
Gary Sivek
Shumin Zhai
102
0
0
21 Oct 2024
LLM The Genius Paradox: A Linguistic and Math Expert's Struggle with Simple Word-based Counting Problems
LLM The Genius Paradox: A Linguistic and Math Expert's Struggle with Simple Word-based Counting Problems
Nan Xu
Xuezhe Ma
LRM
160
5
0
18 Oct 2024
Qtok: A Comprehensive Framework for Evaluating Multilingual Tokenizer
  Quality in Large Language Models
Qtok: A Comprehensive Framework for Evaluating Multilingual Tokenizer Quality in Large Language Models
Iaroslav Chelombitko
Egor Safronov
Aleksey Komissarov
76
1
0
16 Oct 2024
1234...111213
Next