Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark

24 May 2023

Papers citing "Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark"

50 / 54 papers shown

Title
Assessing how hyperparameters impact Large Language Models' sarcasm detection performance Montgomery Gole Andriy Miranskyy AI4MH 21 0 0 08 Apr 2025
RedditESS: A Mental Health Social Support Interaction Dataset -- Understanding Effective Social Support to Refine AI-Driven Support Tools Zeyad Alghamdi Tharindu Kumarage Garima Agrawal Mansooreh Karami Ibrahim Almuteb Huan Liu AI4MH 53 0 0 27 Mar 2025
Leveraging LLMs with Iterative Loop Structure for Enhanced Social Intelligence in Video Question Answering Erika Mori Yue Qiu Hirokatsu Kataoka Y. Aoki 53 0 0 27 Mar 2025
Socially Constructed Treatment Plans: Analyzing Online Peer Interactions to Understand How Patients Navigate Complex Medical Conditions Madhusudan Basak Omar Sharif Jessica Hulsey Elizabeth C. Saunders Daisy J. Goodman Luke J. ArchiBald S. Preum 29 0 0 27 Mar 2025
The Call for Socially Aware Language Technologies Diyi Yang Dirk Hovy David Jurgens Barbara Plank VLM 61 11 0 24 Feb 2025
Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection Maximilian Spliethover Tim Knebler Fabian Fumagalli Maximilian Muschalik Barbara Hammer Eyke Hüllermeier Henning Wachsmuth 105 1 0 10 Feb 2025
SPRIG: Improving Large Language Model Performance by System Prompt Optimization Lechen Zhang Tolga Ergen Lajanugen Logeswaran Moontae Lee David Jurgens LRM 59 8 0 18 Oct 2024
GameTraversalBenchmark: Evaluating Planning Abilities Of Large Language Models Through Traversing 2D Game Maps Muhammad Umair Nasir Steven D. James Julian Togelius ELM LRM 29 2 0 10 Oct 2024
Knowledge Planning in Large Language Models for Domain-Aligned Counseling Summarization Aseem Srivastava Smriti Joshi Tanmoy Chakraborty Md. Shad Akhtar 37 3 0 23 Sep 2024
Prompt Refinement or Fine-tuning? Best Practices for using LLMs in Computational Social Science Tasks Anders Giovanni Moller L. Aiello LLMAG 29 3 0 02 Aug 2024
Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack Xiaoyue Xu Qinyuan Ye Xiang Ren 50 6 0 23 Jul 2024
CLEAR: Can Language Models Really Understand Causal Graphs? Sirui Chen Mengying Xu Kun Wang Xingyu Zeng Rui Zhao Shengjie Zhao Chaochao Lu LRM ELM 32 7 0 24 Jun 2024
Is the Pope Catholic? Yes, the Pope is Catholic. Generative Evaluation of Non-Literal Intent Resolution in LLMs Akhila Yerukola Saujas Vaduguru Daniel Fried Maarten Sap 29 1 0 14 May 2024
Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology Rishav Hada Safiya Husain Varun Gumma Harshita Diddee Aditya Yadavalli ... Nidhi Kulkarni U. Gadiraju Aditya Vashistha Vivek Seshadri Kalika Bali 51 5 0 10 May 2024
Binary Hypothesis Testing for Softmax Models and Leverage Score Models Yeqi Gao Yuzhou Gu Zhao-quan Song 33 0 0 09 May 2024
Can large language models understand uncommon meanings of common words? Jinyang Wu Feihu Che Xinxin Zheng Shuai Zhang Ruihan Jin Shuai Nie Pengpeng Shao Jianhua Tao 36 1 0 09 May 2024
Language Evolution for Evading Social Media Regulation via LLM-based Multi-agent Simulation Jinyu Cai Jialong Li Mingyue Zhang Munan Li Chen-Shu Wang Kenji Tei LLMAG 38 6 0 05 May 2024
Modeling Empathetic Alignment in Conversation Jiamin Yang David Jurgens 24 0 0 02 May 2024
"A good pun is its own reword": Can Large Language Models Understand Puns? Zhijun Xu Siyu Yuan Lingjie Chen Deqing Yang LRM 42 8 0 21 Apr 2024
Comprehensive Reassessment of Large-Scale Evaluation Outcomes in LLMs: A Multifaceted Statistical Approach Kun Sun Rong Wang Anders Sogaard 29 3 0 22 Mar 2024
Academically intelligent LLMs are not necessarily socially intelligent Ruoxi Xu Hongyu Lin Xianpei Han Le Sun Yingfei Sun ELM 29 6 0 11 Mar 2024
In-Memory Learning: A Declarative Learning Framework for Large Language Models Bo Wang Tianxiang Sun Hang Yan Siyin Wang Qingyuan Cheng Xipeng Qiu LLMAG 35 1 0 05 Mar 2024
MIKO: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discovery Feihong Lu Weiqi Wang Yangyifei Luo Ziqin Zhu Qingyun Sun ... Haochen Shi Shiqi Gao Qian Li Yangqiu Song Jianxin Li VLM 32 2 0 28 Feb 2024
Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future Minzhi Li Weiyan Shi Caleb Ziems Diyi Yang 38 9 0 28 Feb 2024
MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms Yiqiao Jin Minje Choi Gaurav Verma Jindong Wang Srijan Kumar 49 19 0 21 Feb 2024
TreeEval: Benchmark-Free Evaluation of Large Language Models through Tree Planning Xiang Li Yunshi Lan Chao Yang ELM 46 8 0 20 Feb 2024
SoMeLVLM: A Large Vision Language Model for Social Media Processing Xinnong Zhang Haoyu Kuang Xinyi Mou Hanjia Lyu Kun Wu Siming Chen Jiebo Luo Xuanjing Huang Zhongyu Wei MLLM 36 5 0 20 Feb 2024
Polarization of Autonomous Generative AI Agents Under Echo Chambers Masaya Ohagi LLMAG 25 7 0 19 Feb 2024
Decoding News Narratives: A Critical Analysis of Large Language Models in Framing Detection Valeria Pastorino Jasivan Sivakumar N. Moosavi 20 2 0 18 Feb 2024
SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Tasks Gourab Dey Adithya V Ganesan Yash Kumar Lal Manal Shah Shreyashee Sinha Matthew Matero Salvatore Giorgi Vivek Kulkarni H. A. Schwartz ALM 61 9 0 03 Feb 2024
Comparing Pre-trained Human Language Models: Is it Better with Human Context as Groups, Individual Traits, or Both? Nikita Soni Niranjan Balasubramanian H. A. Schwartz Dirk Hovy 31 3 0 23 Jan 2024
How Far Are LLMs from Believable AI? A Benchmark for Evaluating the Believability of Human Behavior Simulation Yang Xiao Yi Cheng Jinlan Fu Jiashuo Wang Wenjie Li Pengfei Liu LLMAG 49 4 0 28 Dec 2023
On Sarcasm Detection with OpenAI GPT-based Models Montgomery Gole Williams-Paul Nwadiugwu Andriy Miranskyy 11 8 0 07 Dec 2023
FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicity Shiyao Cui Zhenyu Zhang Yilong Chen Wenyuan Zhang Tianyun Liu Siqi Wang Tingwen Liu 33 13 0 30 Nov 2023
You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments Bangzhao Shu Lechen Zhang Minje Choi Lavinia Dunagan Lajanugen Logeswaran Moontae Lee Dallas Card David Jurgens 19 33 0 16 Nov 2023
Large Human Language Models: A Need and the Challenges Nikita Soni H. A. Schwartz João Sedoc Niranjan Balasubramanian ALM AI4CE 24 11 0 09 Nov 2023
DialogBench: Evaluating LLMs as Human-like Dialogue Systems Jiao Ou Junda Lu Che Liu Yihong Tang Fuzheng Zhang Di Zhang Kun Gai ALM LM&MA 30 14 0 03 Nov 2023
HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning Yongjin Yang Joonkee Kim Yujin Kim Namgyu Ho James Thorne Se-Young Yun 19 21 0 01 Nov 2023
EvalCrafter: Benchmarking and Evaluating Large Video Generation Models Yaofang Liu Xiaodong Cun Xuebo Liu Xintao Wang Yong Zhang Haoxin Chen Yang Liu Tieyong Zeng Raymond H. F. Chan Ying Shan VGen EGVM 18 127 0 17 Oct 2023
Welfare Diplomacy: Benchmarking Language Model Cooperation Gabriel Mukobi Hannah Erlebach Niklas Lauffer Lewis Hammond Alan Chan Jesse Clifton LM&Ro 28 21 0 13 Oct 2023
DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks A. Maritan Jiaao Chen S. Dey Luca Schenato Diyi Yang Xing Xie ELM LRM 24 42 0 29 Sep 2023
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time Yeqi Gao Zhao-quan Song Weixin Wang Junze Yin 20 25 0 14 Sep 2023
A Survey on Large Language Model based Autonomous Agents Lei Wang Chengbang Ma Xueyang Feng Zeyu Zhang Hao-ran Yang ... Xu Chen Yankai Lin Wayne Xin Zhao Zhewei Wei Ji-Rong Wen LLMAG AI4CE LM&Ro 41 1,118 0 22 Aug 2023
A Survey on Evaluation of Large Language Models Yu-Chu Chang Xu Wang Jindong Wang Yuanyi Wu Linyi Yang ... Yue Zhang Yi-Ju Chang Philip S. Yu Qian Yang Xingxu Xie ELM LM&MA ALM 63 1,513 0 06 Jul 2023
The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks Anders Giovanni Møller Jacob Aarup Dalsgaard Arianna Pera L. Aiello 81 35 0 26 Apr 2023
Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary Debates Aida Kostikova Benjamin Paassen Dominik Beese Ole Putz Gregor Wiedemann Steffen Eger 35 3 0 09 Oct 2022
Measure and Improve Robustness in NLP Models: A Survey Xuezhi Wang Haohan Wang Diyi Yang 139 130 0 15 Dec 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
Can Machines Learn Morality? The Delphi Experiment Liwei Jiang Jena D. Hwang Chandra Bhagavatula Ronan Le Bras Jenny T Liang ... Yulia Tsvetkov Oren Etzioni Maarten Sap Regina A. Rini Yejin Choi FaML 127 111 0 14 Oct 2021
Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications Jiaxin Pei David Jurgens 31 29 0 30 Sep 2021