Skip to search boxSkip to navigationSkip to main content
Logo Gonzaga University

Generative Pre-Trained Transformer (GPT) Models as a Code Review Feedback Tool in Computer Science Programs

Research Output:
Contribution to journal
Article
 Peer-review

Open access

Abstract

Undergraduate computer science and software engineering students benefit significantly from in-depth reviews of their code early and often in their courses. Performing these reviews is time-consuming for teaching assistants and professors to complete, consequently impacting the timeliness and consistency of the provided feedback. When code feedback is not delivered close to the time of authorship, the utility of the review for students is diminished. Prior work with Automatic Static Analysis Tools has shown promise at using artificial intelligence to automate code reviews, with some success integrating them into classroom environments. To leverage new advances in Generative Pre-Trained Transformer (GPT) models, this work reports on an Automatic Review Tool (ART) to provide timely, automatically generated code reviews. ART was evaluated in a second-semester computer science course by integrating ART into the course's Github-based assignment submission system. A cohort of student volunteers (N = 74) read the ART reviews and provided feedback using a survey spanning two of their course assignments. The results of this pilot study show that students perceived ART was successful at detecting defects and offering style-based suggestions, and students were receptive to receiving future automated reviews of their work.

Access to documents

Bibliographic Information

Output type

Research Output:
Contribution to journal
Article
 Peer-review

Original language

American English

Pages from-to (Number of pages)

Pages 38-47

Journal (Volume, Issue Number)

Journal of Computing Sciences in Colleges (Volume 39, Issue 1)

Publication milestones

  • Published - 2023

Publication status

Published - 2023

ISSN

1937-4771