The JavaScript Package Selection Task: A Comparative Experiment using ChatGPT

HERNÁN CEFERINO VAZQUEZ; JORGE ANDRÉS DÍAZ PACE; ANTONELA TOMMASEL

La Paz

Conferencia; XLIX CONFERENCIA LATINOAMERICANA DE INFORMÁTICA; 2023

When developing JavaScript (JS) applications, the assessment and selection of JS packages have become challenging for developers due to the growing number of technology options available. Given a technology need, a common developers’ strategy is to query Web repositories via search engines (e.g., NPM, Google) and shortlist candidate JS packages. However, these engines might return a long list of results. Furthermore, these results should be ranked according to the developer’s criteria. To address these problems, we developed a recommender system called AIDT that assists developers in the package selection task. AIDT relies on meta-search and machine learning techniques to infer the relevant packages for a query. An initial evaluation of AIDT showed good search effectiveness. Recently, the emergence of ChatGPT has opened new opportunities for this kind of assistants, as reported by some experiments. Anyway, human developers should judge whether the recommendations (e.g., JS packages) of these tools are fit to purpose. In this paper, we report on a user study in which we used both AIDT and ChatGPT on a sample of JS-related queries, compared their results, and also validated them against developers’ criteria and expectations for the task. Our initial findings show that ChatGPT is not yet on par with AIDT or even human efforts for the task at hand, but the model is flexible to be improved and furthermore, it can provide good arguments for its package choices.