RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,783 papers shown

Title
Emulating the Human Mind: A Neural-symbolic Link Prediction Model with Fast and Slow Reasoning and Filtered Rules Mohammad Hossein Khojasteh Najmeh Torabian Ali Farjami Saeid Hosseini B. Minaei-Bidgoli LRM 63 0 0 21 Oct 2023
Implications of Annotation Artifacts in Edge Probing Test Datasets Sagnik Ray Choudhury Jushaan Kalra 50 0 0 20 Oct 2023
Foundation Model's Embedded Representations May Detect Distribution Shift Max Vargas Adam Tsou A. Engel Tony Chiang 70 1 0 20 Oct 2023
Plausibility Processing in Transformer Language Models: Focusing on the Role of Attention Heads in GPT Soo Hyun Ryu 48 0 0 20 Oct 2023
LanPose: Language-Instructed 6D Object Pose Estimation for Robotic Assembly Bowen Fu Sek Kun Leong Yan Di Jiwen Tang Xiangyang Ji 98 5 0 20 Oct 2023
How Much Consistency Is Your Accuracy Worth? Jacob K. Johnson Ana Marasović 58 1 0 20 Oct 2023
Copyright Violations and Large Language Models Antonia Karamolegkou Jiaang Li Li Zhou Anders Sogaard 76 67 0 20 Oct 2023
Retrieval-Augmented Neural Response Generation Using Logical Reasoning and Relevance Scoring Nicholas Walker Stefan Ultes Pierre Lison RALM LRM 79 2 0 20 Oct 2023
The Perils & Promises of Fact-checking with Large Language Models Dorian Quelle Alexandre Bovet 79 26 0 20 Oct 2023
Explaining Interactions Between Text Spans Sagnik Ray Choudhury Pepa Atanasova Isabelle Augenstein 60 2 0 20 Oct 2023
DistillCSE: Distilled Contrastive Learning for Sentence Embeddings Jiahao Xu Wei Shao Lihui Chen Lemao Liu FedML 72 6 0 20 Oct 2023
Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning Lucas Weber Elia Bruni Dieuwke Hupkes 97 28 0 20 Oct 2023
An LLM can Fool Itself: A Prompt-Based Adversarial Attack Xilie Xu Keyi Kong Ning Liu Li-zhen Cui Di Wang Jingfeng Zhang Mohan Kankanhalli AAML SILM 129 88 0 20 Oct 2023
Large-Scale and Multi-Perspective Opinion Summarization with Diverse Review Subsets Han Jiang Rui Wang Zhihua Wei Yu Li Xinpeng Wang 75 5 0 20 Oct 2023
Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting Chenkai Sun Jinning Li Yi R. Fung Hou Pong Chan Tarek Abdelzaher Chengxiang Zhai Heng Ji 83 16 0 20 Oct 2023
On the Language Encoder of Contrastive Cross-modal Models Mengjie Zhao Junya Ono Zhi-Wei Zhong Chieh-Hsin Lai Yuhta Takida Naoki Murata Wei-Hsiang Liao Takashi Shibuya Hiromi Wakaki Yuki Mitsufuji VLM 63 0 0 20 Oct 2023
A Quality-based Syntactic Template Retriever for Syntactically-controlled Paraphrase Generation Xue Zhang Songming Zhang Yunlong Liang Jinan Xu Jian Liu Wenjuan Han Jinan Xu 95 1 0 20 Oct 2023
Visual Grounding Helps Learn Word Meanings in Low-Data Regimes Chengxu Zhuang Evelina Fedorenko Jacob Andreas 74 12 0 20 Oct 2023
Multi-level Contrastive Learning for Script-based Character Understanding Dawei Li Hengyuan Zhang Yanran Li Shiping Yang 117 17 0 20 Oct 2023
The Less the Merrier? Investigating Language Representation in Multilingual Models H. Nigatu A. Tonja Jugal Kalita 81 1 0 20 Oct 2023
Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models Jaeyoung Choe Keonwoong Noh Nayeon Kim Seyun Ahn Woohwan Jung 129 4 0 20 Oct 2023
NameGuess: Column Name Expansion for Tabular Data Jiani Zhang Zhengyuan Shen Balasubramaniam Srinivasan Shen Wang Huzefa Rangwala George Karypis 49 6 0 19 Oct 2023
CLIFT: Analysing Natural Distribution Shift on Question Answering Models in Clinical Domain Ankit Pal 72 2 0 19 Oct 2023
Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model Zhuoer Wang Yicheng Wang Ziwei Zhu James Caverlee 90 0 0 19 Oct 2023
Do Language Models Learn about Legal Entity Types during Pretraining? Claire Barale Michael Rovatsos Nehal Bhuta ELM 56 2 0 19 Oct 2023
From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues Shivani Kumar S. Ramaneswaran Md. Shad Akhtar Tanmoy Chakraborty 74 23 0 19 Oct 2023
Frozen Transformers in Language Models Are Effective Visual Encoder Layers Ziqi Pang Ziyang Xie Yunze Man Yu-Xiong Wang 146 27 0 19 Oct 2023
Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems David T. Hoffmann Simon Schrodi Jelena Bratulić Nadine Behrmann Volker Fischer Thomas Brox 116 8 0 19 Oct 2023
A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models Yi Zhou Jose Camacho-Collados Danushka Bollegala 159 6 0 19 Oct 2023
StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding Cheng Jiayang Lin Qiu Tszho Chan Tianqing Fang Weiqi Wang ... Qipeng Guo Hongming Zhang Yangqiu Song Yue Zhang Zheng Zhang 100 32 0 19 Oct 2023
Knowledge-Augmented Language Model Verification Jinheon Baek Soyeong Jeong Minki Kang Jong C. Park Sung Ju Hwang RALM 85 14 0 19 Oct 2023
Model Merging by Uncertainty-Based Gradient Matching Nico Daheim Thomas Möllenhoff Edoardo Ponti Iryna Gurevych Mohammad Emtiyaz Khan MoMe FedML 108 53 0 19 Oct 2023
Label-Aware Automatic Verbalizer for Few-Shot Text Classification Thanakorn Thaminkaew Piyawat Lertvittayakumjorn P. Vateekul VLM 44 1 0 19 Oct 2023
Character-level Chinese Backpack Language Models Hao Sun John Hewitt 61 0 0 19 Oct 2023
On the Optimization and Generalization of Multi-head Attention Puneesh Deora Rouzbeh Ghaderi Hossein Taheri Christos Thrampoulidis MLT 89 34 0 19 Oct 2023
Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing Yue Guo Zian Xu Yi Yang ELM 40 10 0 19 Oct 2023
Predict the Future from the Past? On the Temporal Data Distribution Shift in Financial Sentiment Classifications Yue Guo Chenxi Hu Yi Yang 66 8 0 19 Oct 2023
Time-Aware Representation Learning for Time-Sensitive Question Answering Jungbin Son Alice Oh 73 6 0 19 Oct 2023
Pretraining Language Models with Text-Attributed Heterogeneous Graphs Tao Zou Le Yu Yifei Huang Leilei Sun Bo Du AI4CE 62 17 0 19 Oct 2023
DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text Shuaiyi Li Yang Deng Wai Lam 93 2 0 19 Oct 2023
Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Prompt Gangwei Jiang Caigao Jiang Siqiao Xue James Y. Zhang Junqing Zhou Defu Lian Ying Wei VLM 73 7 0 19 Oct 2023
Contrastive Learning for Inference in Dialogue Etsuko Ishii Yan Xu Bryan Wilie Ziwei Ji Holy Lovenia Willy Chung Pascale Fung 70 0 0 19 Oct 2023
MTS-LOF: Medical Time-Series Representation Learning via Occlusion-Invariant Features Huayu Li Ana S. Carreon-Rascon Xiwen Chen Geng Yuan Ao Li AI4TS 38 5 0 19 Oct 2023
A Read-and-Select Framework for Zero-shot Entity Linking Zhenran Xu Yulin Chen Baotian Hu Min Zhang 76 6 0 19 Oct 2023
Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer Qingru Zhang Dhananjay Ram Cole Hawkins Sheng Zha Tuo Zhao 105 16 0 19 Oct 2023
Automated Repair of Declarative Software Specifications in the Era of Large Language Models Md Rashedul Hasan Jiawei Li Iftekhar Ahmed Hamid Bagheri 84 3 0 19 Oct 2023
Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding Jianing Wang Qiushi Sun Nuo Chen Chengyu Wang Jun Huang Ming Gao Xiang Li UQLM 66 4 0 19 Oct 2023
Solving Hard Analogy Questions with Relation Embedding Chains Nitesh Kumar Steven Schockaert 78 1 0 18 Oct 2023
SHARCS: Efficient Transformers through Routing with Dynamic Width Sub-networks Mohammadreza Salehi Sachin Mehta Aditya Kusupati Ali Farhadi Hannaneh Hajishirzi 118 6 0 18 Oct 2023
CORE: A Few-Shot Company Relation Classification Dataset for Robust Domain Adaptation Philipp Borchert Jochen De Weerdt Kristof Coussement Arno De Caigny Marie-Francine Moens 74 3 0 18 Oct 2023