Exploring the use of GPT-3 as a tool for evaluating text-based collaborative discourse
Published in 12th International Conference on Learning Analytics & Knowledge (LAK22), 2022
Recommended citation: Phillips, T., Saleh, A., Hmelo-Silver, C., Glazewski, K., Mott, B., Lester., J. (2022). "Exploring the use of GPT-3 as a tool for evaluating text-based collaborative discourse." 12th International Conference on Learning Analytics & Knowledge (LAK22).
Abstract
Natural language processing (NLP) models have previously been used to classify and summarize student collaborative actions in various online and computerized learning environments. However, due to limitations related to insufficient or inappropriate training data, these models are limited in their applications and impact. In this study, we explore how a new model, GPT-3, summarizes student chat in a computer-supported collaborative learning environment. With only a sentence explaining the context of the learning environment and two training examples, GPT-3 was able to effectively extract and summarize student conversations (properly attributing states such as frustration and confusion), reliably synthesize statements not present in the source text, and effectively ignore extraneous noise in the student chat. We discuss how this summarization could be used to support teachers understanding of student collaboration in computer supported collaborative learning environments.
Keywords
**Natural Language Processing, Collaborative Learning, Discourse **