Exploring Internal Numeracy in Language Models: A Case Study on ALBERT

25 April 2024

Papers citing "Exploring Internal Numeracy in Language Models: A Case Study on ALBERT"

13 / 13 papers shown

Title
On the Transformation of Latent Space in Fine-Tuned NLP Models Nadir Durrani Hassan Sajjad Fahim Dalvi Firoj Alam 87 19 0 23 Oct 2022
Improving Downstream Task Performance by Treating Numbers as Entities Dhanasekar Sundararaman Vivek Subramanian Guoyin Wang Liyan Xu Lawrence Carin 40 5 0 07 May 2022
The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models Ulme Wennberg G. Henter MILM 56 22 0 03 Jun 2021
Pre-Training Transformers as Energy-Based Cloze Models Kevin Clark Minh-Thang Luong Quoc V. Le Christopher D. Manning 48 80 0 15 Dec 2020
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Ikuya Yamada Akari Asai Hiroyuki Shindo Hideaki Takeda Yuji Matsumoto 80 669 0 02 Oct 2020
Learning Numeral Embeddings Chengyue Jiang Zhonglin Nian Kaihao Guo Shanbo Chu Yinggong Zhao Libin Shen Kewei Tu 39 21 0 28 Dec 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 322 6,441 0 26 Sep 2019
Do NLP Models Know Numbers? Probing Numeracy in Embeddings Eric Wallace Yizhong Wang Sujian Li Sameer Singh Matt Gardner 71 265 0 17 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 518 24,351 0 26 Jul 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 232 2,307 0 02 May 2019
Analysing Mathematical Reasoning Abilities of Neural Models D. Saxton Edward Grefenstette Felix Hill Pushmeet Kohli LRM 160 428 0 02 Apr 2019
Meta-Learning for Low-Resource Neural Machine Translation Jiatao Gu Yong Wang Yun Chen Kyunghyun Cho Victor O.K. Li 74 342 0 25 Aug 2018
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov Ilya Sutskever Kai Chen G. Corrado J. Dean NAI OCL 355 33,500 0 16 Oct 2013