Title
Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT Xiaoshuai Song Keqing He Pei Wang Guanting Dong Yutao Mou Jingang Wang Yunsen Xian Xunliang Cai Weiran Xu LRM 34 14 0 16 Oct 2023
GLoRE: Evaluating Logical Reasoning of Large Language Models Hanmeng Liu Zhiyang Teng Ruoxi Ning Jian Liu Qiji Zhou Yuexin Zhang Yue Zhang ReLM ELM LRM 70 7 0 13 Oct 2023
Robustness May be More Brittle than We Think under Different Degrees of Distribution Shifts Kaican Li Yifan Zhang Lanqing Hong Zhenguo Li Nevin L. Zhang OOD 38 0 0 10 Oct 2023
A Formalism and Approach for Improving Robustness of Large Language Models Using Risk-Adjusted Confidence Scores Ke Shen Mayank Kejriwal 14 2 0 05 Oct 2023
Functional trustworthiness of AI systems by statistically valid testing Bernhard Nessler Thomas Doms Sepp Hochreiter 26 0 0 04 Oct 2023
Meta Semantic Template for Evaluation of Large Language Models Yachuan Liu Liang Chen Jindong Wang Qiaozhu Mei Xing Xie 22 0 0 01 Oct 2023
Intuitive or Dependent? Investigating LLMs' Behavior Style to Conflicting Prompts Jiahao Ying Yixin Cao Kai Xiong Yidong He Long Cui Yongbin Liu 31 7 0 29 Sep 2023
Language Models as a Service: Overview of a New Paradigm and its Challenges Emanuele La Malfa Aleksandar Petrov Simon Frieder Christoph Weinhuber Ryan Burnell Raza Nazar Anthony Cohn Nigel Shadbolt Michael Wooldridge ALM ELM 32 3 0 28 Sep 2023
Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data Yifan Peng Jinchuan Tian Brian Yan Dan Berrebbi Xuankai Chang ... Yui Sudo Muhammad Shakeel Jee-weon Jung Soumi Maiti Shinji Watanabe VLM 36 35 0 25 Sep 2023
HANS, are you clever? Clever Hans Effect Analysis of Neural Systems Leonardo Ranaldi Fabio Massimo Zanzotto 31 2 0 21 Sep 2023
Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI Mahyar Abbasian Elahe Khatibi Iman Azimi David Oniani Zahra Shakeri Hossein Abad ... Bryant Lin Olivier Gevaert Li-Jia Li Ramesh C. Jain Amir M. Rahmani LM&MA ELM AI4MH 37 66 0 21 Sep 2023
Are Large Language Models Really Robust to Word-Level Perturbations? Haoyu Wang Guozheng Ma Cong Yu Ning Gui Linrui Zhang ... Sen Zhang Li Shen Xueqian Wang Peilin Zhao Dacheng Tao KELM 21 22 0 20 Sep 2023
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time Yeqi Gao Zhao-quan Song Weixin Wang Junze Yin 20 25 0 14 Sep 2023
MathAttack: Attacking Large Language Models Towards Math Solving Ability Zihao Zhou Qiufeng Wang Mingyu Jin Jie Yao Jianan Ye Wei Liu Wei Wang Xiaowei Huang Kaizhu Huang AAML 30 23 0 04 Sep 2023
Identifying and Mitigating the Security Risks of Generative AI Clark W. Barrett Bradley L Boyd Ellie Burzstein Nicholas Carlini Brad Chen ... Zulfikar Ramzan Khawaja Shams D. Song Ankur Taly Diyi Yang SILM 34 92 0 28 Aug 2023
ZhuJiu: A Multi-dimensional, Multi-faceted Chinese Benchmark for Large Language Models Baolin Zhang Hai-Yong Xie Pengfan Du Junhao Chen Pengfei Cao Yubo Chen Shengping Liu Kang Liu Jun Zhao ELM ALM 24 1 0 28 Aug 2023
Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities Maximilian Mozes Xuanli He Bennett Kleinberg Lewis D. Griffin 39 77 0 24 Aug 2023
GPTEval: A Survey on Assessments of ChatGPT and GPT-4 Rui Mao Guanyi Chen Xulang Zhang Frank Guerin Erik Cambria ELM LM&MA 33 101 0 24 Aug 2023
Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis Chang Liu Bo Wu 24 21 0 22 Aug 2023
Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection Zekun Li Baolin Peng Pengcheng He Xifeng Yan ELM SILM AAML 41 23 0 17 Aug 2023
Robustness Over Time: Understanding Adversarial Examples' Effectiveness on Longitudinal Versions of Large Language Models Yugeng Liu Tianshuo Cong Zhengyu Zhao Michael Backes Yun Shen Yang Zhang AAML 41 6 0 15 Aug 2023
Generative Interpretation Yonathan A. Arbel David Hoffman AILaw AI4CE 9 8 0 14 Aug 2023
Mondrian: Prompt Abstraction Attack Against Large Language Models for Cheaper API Pricing Waiman Si Michael Backes Yang Zhang 22 5 0 07 Aug 2023
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models Paul Röttger Hannah Rose Kirk Bertie Vidgen Giuseppe Attanasio Federico Bianchi Dirk Hovy ALM ELM AILaw 25 125 0 02 Aug 2023
LimeAttack: Local Explainable Method for Textual Hard-Label Adversarial Attack HaiXiang Zhu Zhaoqing Yang Weiwei Shang Yuren Wu AAML FAtt 10 3 0 01 Aug 2023
When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities Jin Chen Zheng Liu Xunpeng Huang Chenwang Wu Qi Liu ... Yuxuan Lei Xiaolong Chen Xingmei Wang Defu Lian Enhong Chen ALM 29 110 0 31 Jul 2023
GPT-4 Can't Reason Konstantine Arkoudas ELM LRM AI4MH 9 32 0 21 Jul 2023
Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms Organizers of QueerInAI Nathaniel Dennler Anaelia Ovalle Ashwin Singh Luca Soldaini ... Kyra Yee Irene Font Peradejordi Zeerak Talat Mayra Russo Jessica de Jesus de Pinho Pinhal 18 15 0 15 Jul 2023
Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs Zhikai Chen Haitao Mao Hang Li Wei Jin Haifang Wen ... Shuaiqiang Wang Dawei Yin Wenqi Fan Hui Liu Jiliang Tang AI4CE 57 265 0 07 Jul 2023
A Survey on Evaluation of Large Language Models Yu-Chu Chang Xu Wang Jindong Wang Yuanyi Wu Linyi Yang ... Yue Zhang Yi-Ju Chang Philip S. Yu Qian Yang Xingxu Xie ELM LM&MA ALM 63 1,513 0 06 Jul 2023
When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions Weiming Zhuang Chen Chen Lingjuan Lyu Cheng Chen Yaochu Jin Lingjuan Lyu AIFin AI4CE 99 85 0 27 Jun 2023
Unifying Large Language Models and Knowledge Graphs: A Roadmap Shirui Pan Linhao Luo Yufei Wang Chen Chen Jiapu Wang Xindong Wu KELM 35 715 0 14 Jun 2023
PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts Kaijie Zhu Jindong Wang Jiaheng Zhou Zichen Wang Hao Chen ... Linyi Yang Weirong Ye Yue Zhang Neil Zhenqiang Gong Xingxu Xie SILM 36 145 0 07 Jun 2023
Concurrent Classifier Error Detection (CCED) in Large Scale Machine Learning Systems Pedro Reviriego Ziheng Wang Alvaro Alonso Zhen Gao Farzad Niknia Shanshan Liu Fabrizio Lombardi 21 1 0 02 Jun 2023
Semantic Role Labeling Guided Out-of-distribution Detection Jinan Zou Maihao Guo Yu Tian Yuhao Lin Hai Cao Lingqiao Liu Ehsan Abbasnejad Javen Qinfeng Shi OODD 23 1 0 29 May 2023
On Evaluating Adversarial Robustness of Large Vision-Language Models Yunqing Zhao Tianyu Pang Chao Du Xiao Yang Chongxuan Li Ngai-man Cheung Min-Bin Lin VLM AAML MLLM 19 166 0 26 May 2023
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP Md. Tawkat Islam Khondaker Abdul Waheed El Moatez Billah Nagoudi Muhammad Abdul-Mageed ELM LM&MA 29 62 0 24 May 2023
OverPrompt: Enhancing ChatGPT through Efficient In-Context Learning Jiazheng Li Runcong Zhao Yongxin Yang Yulan He Lin Gui 35 8 0 24 May 2023
Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds Victoria Basmov Yoav Goldberg Reut Tsarfaty ReLM LRM 24 5 0 24 May 2023
Robust Prompt Optimization for Large Language Models Against Distribution Shifts Moxin Li Wenjie Wang Fuli Feng Yixin Cao Jizhi Zhang Tat-Seng Chua OffRL 42 15 0 23 May 2023
Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate Boshi Wang Xiang Yue Huan Sun ELM LRM 40 59 0 22 May 2023
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models Oana Ignat Zhijing Jin Artem Abzaliev Laura Biester Santiago Castro ... Verónica Pérez-Rosas Siqi Shen Zekun Wang Winston Wu Rada Mihalcea LRM 39 6 0 21 May 2023
Evaluating Open-QA Evaluation Cunxiang Wang Sirui Cheng Qipeng Guo Yuanhao Yue Bowen Ding Zhikun Xu Yidong Wang Xiangkun Hu Zheng Zhang Yue Zhang ELM 26 29 0 21 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 39 82 0 19 May 2023
SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting Xiaoying Zhang Baolin Peng Kun Li Jingyan Zhou Helen M. Meng 66 39 0 15 May 2023
Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility Wen-song Ye Mingfeng Ou Tianyi Li Yipeng Chen Xuetao Ma ... Sai Wu Jie Fu Gang Chen Haobo Wang J. Zhao 44 36 0 15 May 2023
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond Jingfeng Yang Hongye Jin Ruixiang Tang Xiaotian Han Qizhang Feng Haoming Jiang Bing Yin Xia Hu LM&MA 131 622 0 26 Apr 2023
Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness Bo Li Gexiang Fang Yang Yang Quansen Wang Wei Ye Wen Zhao Shikun Zhang ELM AI4MH 19 156 0 23 Apr 2023
Why Does ChatGPT Fall Short in Providing Truthful Answers? Shen Zheng Jie Huang Kevin Chen-Chuan Chang HILM AI4MH 19 51 0 20 Apr 2023
Is ChatGPT Equipped with Emotional Dialogue Capabilities? Weixiang Zhao Yanyan Zhao Xin Lu Shilong Wang Yanpeng Tong Bing Qin LLMAG AI4MH 21 57 0 19 Apr 2023