A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How
Abstract
API suggestion is a critical task in modern software development, assisting programmers by predicting and recommending third-party APIs based on the current context. Recent advancements in large code models (LCMs) have shown promise in the API suggestion task. However, they mainly focus on suggesting which APIs to use, ignoring that programmers may demand more assistance while using APIs in practice including when to use the suggested APIs and how to use the APIs. To mitigate the gap, we conduct a systematic evaluation of LCMs for the API suggestion task in the paper.
BibTeX
@inproceedings{Wang-al:ASE24,
author = {Chaozheng Wang and
Shuzheng Gao and
Cuiyun Gao and
Wenxuan Wang and
Chun Yong Chong and
Shan Gao and
Michael R. Lyu},
title = {A Systematic Evaluation of Large Code Models in {API} Suggestion: When, Which, and How},
booktitle = {ASE},
pages = {281--293},
publisher = {{ACM}},
year = {2024},
}