Student Perceptions of Gamification: A Comparison of Research Studies

Below is the result of a Task 2 in ETEC 500: Research Methodology in Education, a Master’s level course in UBC’s MET program. The task was as follows:

Examine the differences between your selected studies carefully, and briefly try to explain how it is possible that they yielded contradictory results. Be sure to include a comparison of the research design details of each study, and consider how this might have contributed to the contradictory findings. Post a contribution to this forum of maximum 800 words giving complete references for your two studies, and your analysis of and thoughts about their differences.

The studies chosen for this task focused on gathering student perceptions of gamified learning environments. In both studies, participants were asked to share perceptions of their satisfaction with the gamified learning method, their perception of what they had learned, and the perceived impact of the techniques used (Kwon & Özpolat, 2020, p. 71). They were also asked to share thoughts on their perceived motivation or engagement level in gamified learning (Aldemir et al., 2018, p. 243). Both studies were qualitative in nature, asking questions and assessing participants' feelings towards gamification. 

The method of data collection varied by study. In Aldemir et al., the authors used in-class observation, as well as document gathering and interviews to form the basis of their analysis. Documents gathered included emails and comments online as well as artifacts of student work. Interviews were divided between gathering feedback on the gamification methods employed, with a second set of interviews conducted to gather student impressions of the gamified learning experience. The questions asked were vetted by experts with changes made or questions eliminated based on feedback (Aldemir et al., 2018, p. 242). Data gathered by Kwon and Özpolat was in the form of a number of surveys using either a five or seven-point Likert scale asking them if they agreed or disagreed with statements such as “I feel satisfied with the learning method used in this class” (Kwon & Özpolat, 2020, p. 72). The surveys were conducted at eight different times during the school year with the questions being drawn from known teaching method survey models such as Stainsbury and Earnest (2016) and Reinig et al. (2011). A summative survey was conducted at the end of the semester after the final exam to assess students' overall perception of the gamified learning experience (Kwon & Özpolat, 2020, p. 73).

There were significant differences between the two studies in the elements of gamification used, and the delivery of the gamified elements. In Aldemir et al., the gamified elements were built into an entire course, with the main aspect of the study being student perceptions of gamification in course design. The nine key gamification elements: challenge, narrative, leaderboard, reward, badge, teams, points, win-state, and constraints were additionally studied for their individual effectiveness as a subsidiary study (Aldemir et al., 2018, p. 242). Kwon and Özpolat’s studies constrained the gamified elements to the assessment aspects of learning and focused on the naming of assessments using gamified terms such as “farming”, “raids” and “missions” (Kwon & Özpolat, 2020, p. 71). Additionally, a reward structure was implemented called “power-ups” which granted students items such as calculators and notes to assist them in their assessments. Power-up points were granted based on team performance in prior assessments (Kwon & Özpolat, 2020, p. 70).

Students participating in the full course gamification study of Aldemir et al. shared an overall positive experience in a gamified learning environment (Aldemir et al., 2018, p. 251). The results of specific game elements varied, with some receiving more positive perceptions than others (Aldemir et al., 2018, p. 243-247). Students participating in Kwon and Özpolat’s gamified assessment not only reported perceived dissatisfaction with the experience but also performed worse in those assessments. In both quizzes, referred to as farming in the gamified experience, and exams, referred to as boss raids in the gamified experience, the average scores were higher in the control, or non-gamified assessment (Kwon & Özpolat, 2020, p. 73-74).

Embedding game experiences in learning environments is complicated. Even in a controlled study performed by experienced instructional designers delivering what is perceived to be a quality gamified learning experience, there remain a number of challenges. Gamification for gamification’s sake is as Ian Bogost has so eloquently said, bullshit (Bogost, 2015). Attaching alternative names to standard school experiences, and then referring to those experiences as gamified, does not create a high-quality gamified learning experience. It seems unsurprising that students would dislike a gamified assessment experience since, anecdotally, students tend to not enjoy traditional assessment experiences. That the scores were higher in Kwon and Özpolat’s control assessments seems to suggest that a well-established non-gamified environment is preferred to a poorly implemented gamified one. With that in mind, I believe fault for the negative perceptions in Kwon and Özpolat’s gamification study can be partially attributed to a poor design of the gamification experience. Overall, these contradictory studies affirmed that not all gamified experiences are equal, and the method of delivery of a gamified learning experience for the purpose of research is a critical aspect in perceived success.

 References

Aldemir, T., Celik, B., & Kaplan, G. (2018). A qualitative investigation of student perceptions of game elements in a gamified course. Computers in Human Behavior, 78, 235-254. 10.1016/j.chb.2017.10.001

Bogost, I. (2015). Why Gamification is Bullshit. In Walz, S. P., Deterding, S., Zimmerman, E., Bogost, I., Linehan, C., Kirman, B., Roche, B., Pesce, M., Rigby, S., & Hamari, J. (2015). The Gameful World. (pp. 65-79). MIT Press. 10.7551/mitpress/9788.001.0001

Kwon, H. Y., & Özpolat, K. (2021). The Dark Side of Narrow Gamification: Negative Impact of Assessment Gamification on Student Perceptions and Content Knowledge. Transactions on Education, 21(2), 67-81. 10.1287/ited.2019.0227

Reinig, B. A., Horowitz, I., & Whittenburg, G. E. (2011). The Effect of Team-Based Learning on Student Attitudes and Satisfaction. Decision Sciences Journal of Innovative Education, 9(1), 27-47. 10.1111/j.1540-4609.2010.00289.x

Stansbury, J. A., & Earnest, D. R. (2017). Meaningful Gamification in an Industrial/Organizational Psychology Course. Teaching of Psychology, 44(1), 38-45. 10.1177/0098628316677645