SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

2 May 2019

Amanpreet Singh

Papers citing "SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems"

50 / 489 papers shown

Title
On the Evaluation of Engineering Artificial General Intelligence Sandeep Neema Susmit Jha Adam Nagel Ethan Lew Chandrasekar Sureshkumar Aleksa Gordic Chase Shimmin Hieu Nguygen Paul Eremenko ELM 22 0 0 15 May 2025
Towards Contamination Resistant Benchmarks Rahmatullah Musawi Sheng Lu 42 0 0 13 May 2025
Measuring Hong Kong Massive Multi-Task Language Understanding Chuxue Cao Zhenghao Zhu Junqi Zhu Guoying Lu Siyu Peng Juntao Dai Weijie Shi Sirui Han Yike Guo ELM 148 0 0 04 May 2025
Token-free Models for Sarcasm Detection Sumit Mamtani Maitreya Sonawane Kanika Agarwal Nishanth Sanjeev 48 0 0 02 May 2025
Generative AI in Education: Student Skills and Lecturer Roles Stefanie Krause Ashish Dalvi Syed Khubaib Zaidi 159 0 0 28 Apr 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong Xuzhao Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Tianwei Zhang ALM ELM 86 2 0 26 Apr 2025
FLUKE: A Linguistically-Driven and Task-Agnostic Framework for Robustness Evaluation Yulia Otmakhova Hung Thinh Truong Rahmad Mahendra Zenan Zhai Rongxin Zhu Daniel Beck Jey Han Lau ELM 70 0 0 24 Apr 2025
UrbanPlanBench: A Comprehensive Urban Planning Benchmark for Evaluating Large Language Models Yu Zheng Longyi Liu Yuming Lin Jie Feng Guozhen Zhang Depeng Jin Yong Li ELM 73 0 0 23 Apr 2025
ViQA-COVID: COVID-19 Machine Reading Comprehension Dataset for Vietnamese H. Phung Ngoc C. Lê Van-Chien Nguyen Hang Thi Nguyen Thuy Phuong Thi Nguyen 75 1 0 21 Apr 2025
DMind Benchmark: Toward a Holistic Assessment of LLM Capabilities across the Web3 Domain Miracle Master Rainy Sun Anya Reese Joey Ouyang Alex Chen ... James Yi Garry Zhao Tony Ling Hobert Wong Lowes Yang ALM ELM 74 0 0 18 Apr 2025
CPG-EVAL: A Multi-Tiered Benchmark for Evaluating the Chinese Pedagogical Grammar Competence of Large Language Models Dong Wang ELM 33 0 0 17 Apr 2025
Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora Alex Warstadt Aaron Mueller Leshem Choshen E. Wilcox Chengxu Zhuang ... Rafael Mosquera Bhargavi Paranjape Adina Williams Tal Linzen Ryan Cotterell 38 108 0 10 Apr 2025
ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion Rana Muhammad Shahroz Khan Dongwen Tang Pingzhi Li Kai Wang Tianlong Chen AI4CE 142 0 0 31 Mar 2025
HRET: A Self-Evolving LLM Evaluation Toolkit for Korean Hanwool Albert Lee Soo Yong Kim Dasol Choi Sangwon Baek Seunghyeok Hong Ilgyun Jeong Inseon Hwang Naeun Lee Guijin Son VLM 46 0 0 29 Mar 2025
MoQa: Rethinking MoE Quantization with Multi-stage Data-model Distribution Awareness Zihao Zheng Xiuping Cui Size Zheng Maoliang Li Jiayu Chen Yun Liang Xiang Chen MQ MoE 64 0 0 27 Mar 2025
CASE -- Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement Gaifan Zhang Yi Zhou Danushka Bollegala 153 0 0 21 Mar 2025
Measuring AI Ability to Complete Long Tasks Thomas Kwa Ben West Joel Becker Amy Deng Katharyn Garcia ... Lucas Jun Koba Sato H. Wijk Daniel M. Ziegler Elizabeth Barnes Lawrence Chan ELM 82 6 0 18 Mar 2025
TLUE: A Tibetan Language Understanding Evaluation Benchmark Fan Gao Cheng Huang Nyima Tashi Xiangxiang Wang Thupten Tsering ... Gadeng Luosang Rinchen Dongrub Dorje Tashi Xiao Feng Yongbin Yu ELM 76 2 0 15 Mar 2025
LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama Naome A. Etori Kevin Lu Randu Karisa Arturs Kanepajs LRM ELM 160 0 0 14 Mar 2025
Tgea: An error-annotated dataset and benchmark tasks for text generation from pretrained language models Jie He Bo Peng Yi-Lun Liao Qun Liu Deyi Xiong 60 8 0 06 Mar 2025
BIG-Bench Extra Hard Mehran Kazemi Bahare Fatemi Hritik Bansal John Palowitch Chrysovalantis Anastasiou ... Kate Olszewska Yi Tay Vinh Q. Tran Quoc V. Le Orhan Firat ELM LRM 122 5 0 26 Feb 2025
Unveiling Reasoning Thresholds in Language Models: Scaling, Fine-Tuning, and Interpretability through Attention Maps Yen-Che Hsiao Abhishek Dutta LRM ReLM ELM 66 0 0 24 Feb 2025
PiCO: Peer Review in LLMs based on the Consistency Optimization Kun-Peng Ning Shuo Yang Yu-Yang Liu Jia-Yu Yao Zhen-Hui Liu Yu Wang Ming Pang Li Yuan ALM 71 8 0 24 Feb 2025
Understand User Opinions of Large Language Models via LLM-Powered In-the-Moment User Experience Interviews Mengqiao Liu Tevin Wang Cassandra A. Cohen Sarah Li Chenyan Xiong LRM 69 0 0 24 Feb 2025
BERTtime Stories: Investigating the Role of Synthetic Story Data in Language Pre-training Nikitas Theodoropoulos Giorgos Filandrianos Vassilis Lyberatos Maria Lymperaiou Giorgos Stamou SyDa 54 1 0 24 Feb 2025
Reasoning and the Trusting Behavior of DeepSeek and GPT: An Experiment Revealing Hidden Fault Lines in Large Language Models Rubing Li João Sedoc Arun Sundararajan LRM 68 1 0 20 Feb 2025
MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models Zhen Zhang Yuqing Yang Kai Zhen Nathan Susanj Athanasios Mouchtaris Siegfried Kunzmann Zheng Zhang 54 0 0 17 Feb 2025
Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language Models Haoyang Li Xuejia Chen Zhanchao Xu Darian Li Nicole Hu ... Heng Chang Luyu Qiu C. Zhang Qing Li Lei Chen LRM ELM 42 1 0 16 Feb 2025
RideKE: Leveraging Low-Resource, User-Generated Twitter Content for Sentiment and Emotion Detection in Kenyan Code-Switched Dataset Naome A. Etori Maria Gini 81 2 0 10 Feb 2025
Unbiased Evaluation of Large Language Models from a Causal Perspective Meilin Chen Jian Tian Liang Ma Di Xie Weijie Chen Jiang Zhu ALM ELM 54 0 0 10 Feb 2025
Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Models S. Poddar Paramita Koley Janardan Misra Niloy Ganguly Saptarshi Ghosh Saptarshi Ghosh 64 0 0 08 Feb 2025
Brain-inspired sparse training enables Transformers and LLMs to perform as fully connected Yingtao Zhang Jialin Zhao Wenjing Wu Ziheng Liao Umberto Michieli C. Cannistraci 51 0 0 31 Jan 2025
Understanding Why Adam Outperforms SGD: Gradient Heterogeneity in Transformers Akiyoshi Tomihari Issei Sato ODL 61 1 0 31 Jan 2025
Survey and Improvement Strategies for Gene Prioritization with Large Language Models Matthew Neeley Guantong Qi Guanchu Wang Ruixiang Tang Dongxue Mao ... Bo Yuan Fan Xia Pengfei Liu Zhandong Liu Xia Hu LM&MA 104 0 0 30 Jan 2025
A linguistically-motivated evaluation methodology for unraveling model's abilities in reading comprehension tasks Elie Antoine Frédéric Béchet Géraldine Damnati Philippe Langlais 56 1 0 29 Jan 2025
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models Yibin Wang Haizhou Shi Ligong Han Dimitris N. Metaxas Hao Wang BDL UQLM 116 6 0 28 Jan 2025
Merino: Entropy-driven Design for Generative Language Models on IoT Devices Youpeng Zhao Ming Lin Huadong Tang Qiang Wu Jun Wang 83 0 0 28 Jan 2025
Decentralized Low-Rank Fine-Tuning of Large Language Models Sajjad Ghiasvand Mahnoosh Alizadeh Ramtin Pedarsani ALM 66 0 0 26 Jan 2025
CS-Eval: A Comprehensive Large Language Model Benchmark for CyberSecurity Zhengmin Yu Jiutian Zeng Siyi Chen Wenhan Xu Dandan Xu Xiangyu Liu Zonghao Ying Nan Wang Yuan Zhang Min Yang ELM 108 1 0 20 Jan 2025
Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words Gouki Minegishi Hiroki Furuta Yusuke Iwasawa Y. Matsuo 49 1 0 09 Jan 2025
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning Yehonathan Refael Jonathan Svirsky Boris Shustin Wasim Huleihel Ofir Lindenbaum 47 3 0 31 Dec 2024
GPT or BERT: why not both? Lucas Georges Gabriel Charpentier David Samuel 55 5 0 31 Dec 2024
ChemTEB: Chemical Text Embedding Benchmark, an Overview of Embedding Models Performance & Efficiency on a Specific Domain Ali Shiraee Kasmaee Mohammad Khodadad Mohammad Arshi Saloot Nick Sherck Stephen Dokas H. Mahyar Soheila Samiee ELM 180 0 0 30 Nov 2024
Streamlining Prediction in Bayesian Deep Learning Rui Li Marcus Klasson Arno Solin Martin Trapp UQCV BDL 97 2 0 27 Nov 2024
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games Davide Paglieri Bartłomiej Cupiał Samuel Coward Ulyana Piterbarg Maciej Wolczyk ... Lerrel Pinto Rob Fergus Jakob Foerster Jack Parker-Holder Tim Rocktaschel LLMAG LRM 113 10 0 20 Nov 2024
Task Calibration: Calibrating Large Language Models on Inference Tasks Yingjie Li Yun Luo Xiaotian Xie Yue Zhang LRM 21 0 0 24 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Active Learning of Robot Vision Using Adaptive Path Planning Julius Ruckin Federico Magistri Cyrill Stachniss Marija Popović SSL 26 0 0 14 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 44 0 0 12 Oct 2024
StablePrompt: Automatic Prompt Tuning using Reinforcement Learning for Large Language Models Minchan Kwon Gaeun Kim Jongsuk Kim Haeil Lee Junmo Kim OffRL LRM LLMAG 26 2 0 10 Oct 2024