Generative AI for Programming Education: Benchmarking ChatGPT, GPT-4, and Human Tutors

29 June 2023

Tung Phung

Victor-Alexandru Pădurean

Papers citing "Generative AI for Programming Education: Benchmarking ChatGPT, GPT-4, and Human Tutors"

35 / 35 papers shown

Title
Synthesizing High-Quality Programming Tasks with LLM-based Expert and Student Agents Manh Hung Nguyen Victor-Alexandru Pădurean Alkis Gotovos Sebastian Tschiatschek Adish Singla 24 0 0 10 Apr 2025
Design of AI-Powered Tool for Self-Regulation Support in Programming Education Huiyong Li Boxuan Ma AI4Ed 46 0 0 03 Apr 2025
Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubrics Aditya Pathak Rachit Gandhi Vaibhav Uttam Devansh Yashwanth Nakka ... Aditya Mittal Aashna Ased Chirag Khatri Jagat Sesh Challa Dhruv Kumar 45 0 0 31 Mar 2025
AI Literacy in K-12 and Higher Education in the Wake of Generative AI: An Integrative Review Xingjian Gu B. Ericson 44 0 0 27 Feb 2025
Emotionally Enriched Feedback via Generative AI Omar Alsaiari Nilufar Baghaei Hatim Lahza Jason M. Lodge Marie Boden Hassan Khosravi 24 0 0 19 Oct 2024
Crafting Generative Art through Genetic Improvement: Managing Creative Outputs in Diverse Fitness Landscapes Erik M. Fredericks Denton Bobeldyk Jared M. Moore 41 0 0 29 Jul 2024
No Size Fits All: The Perils and Pitfalls of Leveraging LLMs Vary with Company Size Ashok Urlana Charaka Vinayak Kumar B. Garlapati Ajeet Kumar Singh Rahul Mishra 40 1 0 21 Jul 2024
Evaluating Language Models for Generating and Judging Programming Feedback Charles Koutcheme Nicola Dainese Arto Hellas Sami Sarsa Juho Leinonen Syed Ashraf Paul Denny ELM 34 2 0 05 Jul 2024
Program Synthesis Benchmark for Visual Programming in XLogoOnline Environment Chao Wen Jacqueline Staub Adish Singla ELM 44 3 0 17 Jun 2024
Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming Victor-Alexandru Pădurean Adish Singla ELM 54 3 0 14 Jun 2024
Evaluating Contextually Personalized Programming Exercises Created with Generative AI E. Logacheva Arto Hellas James Prather Sami Sarsa Juho Leinonen 37 10 0 11 Jun 2024
Hints-In-Browser: Benchmarking Language Models for Programming Feedback Generation Nachiket Kotalwar Alkis Gotovos Adish Singla ALM 67 4 0 07 Jun 2024
Benchmarking Educational Program Repair Charles Koutcheme Nicola Dainese Sami Sarsa Juho Leinonen Arto Hellas Paul Denny AI4Ed 45 5 0 08 May 2024
Open Source Language Models Can Provide Feedback: Evaluating LLMs' Ability to Help Students Using GPT-4-As-A-Judge Charles Koutcheme Nicola Dainese Sami Sarsa Arto Hellas Juho Leinonen Paul Denny ELM ALM 47 22 0 08 May 2024
Task Synthesis for Elementary Visual Programming in XLogoOnline Environment Chao Wen Ahana Ghosh Jacqueline Staub Adish Singla 36 3 0 03 May 2024
Generating Feedback-Ladders for Logical Errors in Programming using Large Language Models Hasnain Heickal Andrew Lan LRM 27 2 0 01 May 2024
Towards Generalizable Agents in Text-Based Educational Environments: A Study of Integrating RL with LLMs Bahar Radmehr Adish Singla Tanja Kaser LLMAG AI4CE 43 6 0 29 Apr 2024
Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study Wenhan Lyu Yimeng Wang Tingting Rachel Chung Chung Yifan Sun Yixuan Zhang 35 30 0 20 Apr 2024
Enhancing Programming Education with ChatGPT: A Case Study on Student Perceptions and Interactions in a Python Course Boxaun Ma Li Chen Shin’ichi Konomi 29 8 0 20 Mar 2024
Evaluating the Application of Large Language Models to Generate Feedback in Programming Education Sven Jacobs Steffen Jaschke 57 3 0 13 Mar 2024
A systematic evaluation of large language models for generating programming code Wenpin Hou Zhicheng Ji ELM 39 2 0 01 Mar 2024
LLMs with Industrial Lens: Deciphering the Challenges and Prospects -- A Survey Ashok Urlana Charaka Vinayak Kumar Ajeet Kumar Singh B. Garlapati S. Chalamala Rahul Mishra 35 5 0 22 Feb 2024
AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrails Sankalan Pal Chowdhury Vilém Zouhar Mrinmaya Sachan AI4Ed LRM 16 14 0 14 Feb 2024
Using Large Language Models for Student-Code Guided Test Case Generation in Computer Science Education Nischal Ashok Kumar Andrew S. Lan AI4Ed ELM 24 5 0 11 Feb 2024
Generative AI for Education (GAIED): Advances, Opportunities, and Challenges Paul Denny Sumit Gulwani Neil T. Heffernan Tanja Kaser Steven Moore Anna N. Rafferty Adish Singla 34 16 0 02 Feb 2024
Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges Qingyao Li Lingyue Fu Weiming Zhang Xianyu Chen Jingwei Yu Wei Xia Weinan Zhang Ruiming Tang Yong Yu AI4Ed ELM 43 18 0 27 Dec 2023
User Modeling in the Era of Large Language Models: Current Research and Future Directions Zhaoxuan Tan Meng Jiang 30 8 0 11 Dec 2023
Anticipating User Needs: Insights from Design Fiction on Conversational Agents for Computational Thinking Jacob Penney João Felipe Pimentel Igor Steinmacher M. Gerosa AI4CE 23 5 0 12 Nov 2023
Large Language Models for In-Context Student Modeling: Synthesizing Student's Behavior in Visual Programming Manh Hung Nguyen Sebastian Tschiatschek Adish Singla AI4Ed 21 7 0 15 Oct 2023
Automating Human Tutor-Style Programming Feedback: Leveraging GPT-4 Tutor Model for Hint Generation and GPT-3.5 Student Model for Hint Validation Tung Phung Victor-Alexandru Pădurean Anjali Singh Christopher A. Brooks J. Cambronero Sumit Gulwani Adish Singla Gustavo Soares 28 39 0 05 Oct 2023
The Robots are Here: Navigating the Generative AI Revolution in Computing Education James Prather Paul Denny Juho Leinonen Brett A. Becker Ibrahim Albluwi ... Stephen MacNeil Andrew Petersen Raymond Pettit Brent N. Reeves Jaromír Šavelka 36 193 0 01 Oct 2023
Evaluating ChatGPT and GPT-4 for Visual Programming Adish Singla 19 20 0 30 Jul 2023
Neural Task Synthesis for Visual Programming Victor-Alexandru Pădurean Georgios Tzannetos Adish Singla 33 17 0 26 May 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 339 2,232 0 22 Mar 2023
Automatic Generation of Programming Exercises and Code Explanations using Large Language Models Sami Sarsa Paul Denny Arto Hellas Juho Leinonen ELM 99 342 0 03 Jun 2022