Automated Rubric-Based Classification of Student Peer Code Review Feedback

This publication was part of the Technology and Innovation for Culture, Education, Language, and Society Research Group at Informatics Department Universitas Atma Jaya Yogyakarta. This article was published in Scopus-indexed journal: International Journal of Information and Education Technology.

Authors: Theresia Devi Indriasari & Yohanes Sigit Purnomo W.P.

Language: English

Abstract:
This study develops automated methods for classifying student peer code review feedback written in Bahasa Indonesia. The classification task uses seven labels: six labels represent code quality rubric criteria in introductory programming courses, namely Variable, Expression, Control Flow, Comments, Layout and Formatting, and Decomposition, while one additional label represents general feedback. The dataset consists of 2,281 student feedback items collected from peer code review activities in a higher education programming course. The annotation process showed strong agreement between raters, with Cohen’s Kappa of 0.9463. The study compares three approaches: classical machine learning, deep learning, and few-shot prompting with large language models. Random Forest with count vectorization achieved the highest performance, with an F1-score of 0.9430. This result was higher than recurrent convolutional neural networks with FastText embeddings and few-shot prompting. The findings suggest that classical machine learning can provide an effective and practical foundation for integrating rubric-based feedback classification into peer code review tools.

Keywords: student peer code review, rubric-based classification, machine learning, deep learning, few-shot prompting, computing education, Bahasa Indonesia

DOI: 10.18178/ijiet.2026.16.5.2598

How to Cite

If you extend or use this work, please cite the paper where it was introduced:

@article{INDRIASARI2026AUTOMATED,
	title = {Automated Rubric-Based Classification of Student Peer Code Review Feedback},
	journal = {International Journal of Information and Education Technology},
	volume = {16},
	number = {5},
	pages = {1298--1314},
	year = {2026},
	issn = {2010-3689},
	doi = {https://doi.org/10.18178/ijiet.2026.16.5.2598},
	url = {https://www.ijiet.org/show-240-3268-1.html},
	author = {Theresia Devi Indriasari and Yohanes Sigit Purnomo W.P.},
	keywords = {student peer code review, rubric-based classification, machine learning, deep learning, few-shot prompting, computing education, Bahasa Indonesia},
	abstract = {This study develops automated methods for classifying student peer code review feedback written in Bahasa Indonesia. The classification task uses seven labels based on code quality rubric criteria and general feedback. Using 2,281 student feedback items collected from peer code review activities, the study compares classical machine learning, deep learning, and few-shot prompting with large language models. The annotated dataset showed strong inter-rater agreement with Cohen's Kappa of 0.9463. Random Forest with count vectorization achieved the best result, with an F1-score of 0.9430. The findings suggest that classical machine learning can provide an effective foundation for supporting rubric-based feedback classification in peer code review tools.}
}