Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on
Zero-shot LLM Assessment

Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment

21 February 2024

Mark J. F. Gales

Papers citing "Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment"

15 / 15 papers shown

Title
SAS-Bench: A Fine-Grained Benchmark for Evaluating Short Answer Scoring with Large Language Models Peichao Lai Kaipeng Zhang Yi Lin L. Zhang Feiyang Ye ... Yanwei Xu Conghui He Yixuan Wang Wentao Zhang Bin Cui ELM LRM 47 0 0 12 May 2025
LLM-as-a-Judge: Reassessing the Performance of LLMs in Extractive QA Xanh Ho Jiahao Huang Florian Boudin Akiko Aizawa ELM 36 0 0 16 Apr 2025
Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework Kaishuai Xu Tiezheng YU Wenjun Hou Yi Cheng Liangyou Li Xin Jiang Lifeng Shang Qiang Liu Wenjie Li ELM 66 0 0 26 Feb 2025
Investigating Non-Transitivity in LLM-as-a-Judge Yi Xu Laura Ruis Tim Rocktaschel Robert Kirk 43 0 0 19 Feb 2025
PRISMe: A Novel LLM-Powered Tool for Interactive Privacy Policy Assessment Vincent Freiberger Arthur Fleig Erik Buchmann 48 2 0 28 Jan 2025
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge Dawei Li Bohan Jiang Liangjie Huang Alimohammad Beigi Chengshuai Zhao ... Canyu Chen Tianhao Wu Kai Shu Lu Cheng Huan Liu ELM AILaw 120 67 0 25 Nov 2024
Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation Dongryeol Lee Yerin Hwang Yongil Kim Joonsuk Park Kyomin Jung ELM 72 5 0 28 Oct 2024
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Jing Jiang Min-Bin Lin 41 8 0 09 Oct 2024
From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks Andreas Stephan D. Zhu Matthias Aßenmacher Xiaoyu Shen Benjamin Roth ELM 47 4 0 06 Sep 2024
CERT-ED: Certifiably Robust Text Classification for Edit Distance Zhuoqun Huang Yipeng Wang Seunghee Shin Benjamin I. P. Rubinstein AAML 48 1 0 01 Aug 2024
MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankers Fang Guo Wenyu Li Honglei Zhuang Yun Luo Yafu Li Qi Zhu Le Yan Yue Zhang ALM 73 6 0 18 Apr 2024
Self-playing Adversarial Language Game Enhances LLM Reasoning Pengyu Cheng Tianhao Hu Han Xu Zhisong Zhang Yong Dai Lei Han Nan Du Nan Du Xiaolong Li SyDa LRM ReLM 98 29 0 16 Apr 2024
LLM Evaluators Recognize and Favor Their Own Generations Arjun Panickssery Samuel R. Bowman Shi Feng 44 156 0 15 Apr 2024
JudgeLM: Fine-tuned Large Language Models are Scalable Judges Lianghui Zhu Xinggang Wang Xinlong Wang ELM ALM 56 108 0 26 Oct 2023
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,815 0 14 Dec 2020