Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

7 March 2024

Mostafa Elhoushi

Papers citing "Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks"

15 / 15 papers shown

Title
AutoP2C: An LLM-Based Agent Framework for Code Repository Generation from Multimodal Content in Academic Papers Zijie Lin Yiqing Shen Qilin Cai He Sun Jinrui Zhou Mingjun Xiao 57 0 0 28 Apr 2025
Evaluating the Generalization Capabilities of Large Language Models on Code Reasoning Rem Yang Julian Dai N. Vasilakis Martin Rinard ELM LRM 32 0 0 07 Apr 2025
Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Enhancement Protocol Roham Koohestani Philippe de Bekker M. Izadi VLM 45 0 0 07 Mar 2025
Comparative Analysis of Large Language Models for Context-Aware Code Completion using SAFIM Framework Hang Zhang Yanxin Shen Lun Wang Chuanqi Shi Shaoshuai Du Yiyi Tao Yixian Shen 57 0 0 24 Feb 2025
Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents Benjamin Rombaut Sogol Masoumzadeh Kirill Vasilevski Dayi Lin Ahmed E. Hassan LRM 36 0 0 05 Nov 2024
Decoding Secret Memorization in Code LLMs Through Token-Level Characterization Yuqing Nie Chong Wang Kaixin Wang Guoai Xu Guosheng Xu Haoyu Wang OffRL 136 1 0 11 Oct 2024
In-Context Code-Text Learning for Bimodal Software Engineering Xunzhu Tang Liran Wang Yonghui Liu Linzheng Chai Jian Yang Zhoujun Li Haoye Tian Jacques Klein Tegawende F. Bissyande 35 0 0 08 Oct 2024
Horizon-Length Prediction: Advancing Fill-in-the-Middle Capabilities for Code Generation with Lookahead Planning Yifeng Ding Hantian Ding Shiqi Wang Qing Sun Varun Kumar Zijian Wang 28 2 0 04 Oct 2024
Qwen2.5-Coder Technical Report Binyuan Hui Jian Yang Zeyu Cui Jiaxi Yang Dayiheng Liu ... Fei Huang Xingzhang Ren Xuancheng Ren Jingren Zhou Junyang Lin OSLM 72 205 0 18 Sep 2024
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models Jia Zheng Boxi Cao Zhengzhao Ma Ruotong Pan Hongyu Lin Yaojie Lu Xianpei Han Le Sun ALM 36 4 0 16 Jul 2024
Multi-lingual Evaluation of Code Generation Models Ben Athiwaratkun Sanjay Krishna Gouda Zijian Wang Xiaopeng Li Yuchen Tian ... Baishakhi Ray Parminder Bhatia Sudipta Sengupta Dan Roth Bing Xiang ELM 120 160 0 26 Oct 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 328 11,953 0 04 Mar 2022
A Systematic Evaluation of Large Language Models of Code Frank F. Xu Uri Alon Graham Neubig Vincent J. Hellendoorn ELM ALM 204 631 0 26 Feb 2022
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq R. Joty S. Hoi 241 1,489 0 02 Sep 2021
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 208 624 0 20 May 2021