Probing Across Time: What Does RoBERTa Know and When?

16 April 2021

Papers citing "Probing Across Time: What Does RoBERTa Know and When?"

20 / 20 papers shown

Title
A distributional simplicity bias in the learning dynamics of transformers Riccardo Rende Federica Gerace A. Laio Sebastian Goldt 79 8 0 17 Feb 2025
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition Jiyeon Kim Hyunji Lee Hyowon Cho Joel Jang Hyeonbin Hwang Seungpil Won Youbin Ahn Dohaeng Lee Minjoon Seo KELM 125 3 0 02 Oct 2024
How Do Large Language Models Acquire Factual Knowledge During Pretraining? Hoyeon Chang Jinho Park Seonghyeon Ye Sohee Yang Youngkyung Seo Du-Seong Chang Minjoon Seo KELM 37 32 0 17 Jun 2024
Language Models Represent Space and Time Wes Gurnee Max Tegmark 47 142 0 03 Oct 2023
The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations Vinitra Swamy Jibril Frej Tanja Kaser 34 14 0 01 Jul 2023
Does ChatGPT have Theory of Mind? B. Holterman Kees van Deemter LRM AI4CE 36 22 0 23 May 2023
A Natural Bias for Language Generation Models Clara Meister Wojciech Stokowiec Tiago Pimentel Lei Yu Laura Rimell A. Kuncoro MILM 33 6 0 19 Dec 2022
Gender Biases Unexpectedly Fluctuate in the Pre-training Stage of Masked Language Models Kenan Tang Hanchun Jiang AI4CE 18 1 0 26 Nov 2022
SocioProbe: What, When, and Where Language Models Learn about Sociodemographics Anne Lauscher Federico Bianchi Samuel R. Bowman Dirk Hovy 29 7 0 08 Nov 2022
Probing with Noise: Unpicking the Warp and Weft of Embeddings Filip Klubicka John D. Kelleher 30 4 0 21 Oct 2022
CEFER: A Four Facets Framework based on Context and Emotion embedded features for Implicit and Explicit Emotion Recognition Fereshte Khoshnam Ahmad Baraani-Dastjerdi M. J. Liaghatdar 13 0 0 28 Sep 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 29 187 0 22 May 2022
Life after BERT: What do Other Muppets Understand about Language? Vladislav Lialin Kevin Zhao Namrata Shivagunde Anna Rumshisky 47 6 0 21 May 2022
Neural reality of argument structure constructions Bai Li Zining Zhu Guillaume Thomas Frank Rudzicz Yang Xu 46 26 0 24 Feb 2022
Interpreting Language Models Through Knowledge Graph Extraction Vinitra Swamy Angelika Romanou Martin Jaggi 28 20 0 16 Nov 2021
The Grammar-Learning Trajectories of Neural Language Models Leshem Choshen Guy Hacohen D. Weinshall Omri Abend 29 28 0 13 Sep 2021
A Bayesian Framework for Information-Theoretic Probing Tiago Pimentel Ryan Cotterell 28 24 0 08 Sep 2021
The MultiBERTs: BERT Reproductions for Robustness Analysis Thibault Sellam Steve Yadlowsky Jason W. Wei Naomi Saphra Alexander DÁmour ... Iulia Turc Jacob Eisenstein Dipanjan Das Ian Tenney Ellie Pavlick 24 93 0 30 Jun 2021
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 29 136 0 10 Nov 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 419 2,588 0 03 Sep 2019