Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Gamifying Instructional Videos Did Not Lead to Better Student Comprehension

California State University Long Beach, Long Beach, CA 90840, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1006955

Abstract

Online learning has become a widely adopted method of education. Although it offers many benefits, it also comes with certain drawbacks. For instance, asynchronous courses provide students with flexibility, but distractions in the study environment can affect students' ability to concentrate when viewing online lectures. Moreover, students may not be motivated to continue to watch if the instructional videos are not engaging. To potentially counter this latter issue, the present study examined whether gamification could benefit online learning by improving students’ comprehension of video lecture content. A 2 (Gamification Level: Gamification or No Gamification) x 2 (Content Difficulty: Easy or Hard) mixed design was used. Participants were assigned to either a gamification condition or a non-gamification condition. For both conditions they watched instructional videos containing easy- and hard-level content and were tested on their comprehension of the video content. It was hypothesized that gamification would keep students engaged with the instructional videos, leading to higher scores on comprehension quizzes. However, results of the present study showed no effect lecture video gamification on the quiz scores. Implications of these findings for online learning are discussed.

INTRODUCTION

Although online courses have been available for decades, its mass adoption occurred as a result of the COVID-19 pandemic closures. Even after repopulation, online learning has remained a popular method of course delivery. To better understand aspects of online learning, Singh and Thurman (2019) distinguished between whether the learning occurs in a synchronous or asynchronous environment. A synchronous environment provides students with the ability to engage and interact with their instructors and peers using technology connected to the internet without having to be in the same physical space. These aspects synchronous learning can provide students with flexibility while providing a sense of belonging to help encourage them to keep up with their learning (Christenson, & Anderson, 2002). An asynchronous environment provides even more flexibility for students because it gives them the opportunity to expand their learning environment through self-pacing (Keengwe & Kidd, 2010).

© 2025. Published by AHFE Open Access. All rights reserved. 1432

Gamifying Instructional Videos Did Not Lead to Better Student Comprehension 1433

While there are many benefits to both types of online learning, each comes with its own disadvantages. For instance, synchronous courses may be less convenient for working students because they meet at designated times. However, with asynchronous learning, students have less interactions with instructors and peers. For both types of learning, students’ ability to focus during online lectures can be affected by distractions in their environment. Thus, for online learning to be used as a successful tool for both instructors and their students, information about factors that impact student engagement and comprehension of the learning materials is needed (Dewan, Murshed, & Lin, 2019).

Student engagement can come in many different forms (see e.g., Dewan et al., 2019). Affective engagement is when students show an interest in the topic being taught and are also enjoying what they are learning. Academic engagement is when students show an academic identification and participation towards learning, such as getting along with their teachers and spending time working on tasks assigned. Cognitive engagement is when students willingly and thoughtfully put in the effort to understand and master complex ideas and skills during their learning. Psychological engagement is when students feel a sense of belonging and a relationship with their instructors and peers in the process. In addition, these tradition forms of engagement, adding a fun element into the learning process can also help motivate students to become more interested in the course materials. Therefore, gamification of online learning materials has been suggested as a method to improve student engagement (Lamprinou & Paraskeva, 2015).

Gamification is the idea of using game elements to improve the experience and engagement of users in a non-gaming setting (Deterding et al., 2011), such as online learning. Gaming elements can include interactive components such as rewards for points and badges, and leaderboards to keep score of one’s own progress relative to others. Gaming components that are challenging and/or have an exploration element can also motivate people to stay engaged for long periods of time (Mishra & Kotecha, 2017). Additionally, people stay engaged longer with online games when they have a social component.

When implementing gamification in online courses, designers need to consider two types of gamification methods: structural gamification and content gamification (Lamprinou & Paraskeva, 2015). Structural gamification make use of items that may increase motivation, such as rewards (points), badges, achievement acknowledgements, and leaderboards, but does not make any changes to the content of the lesson itself. Content gamification, on the other hand, is the concept of using game elements, such as challenges and story designs, to make the course content have more of a game-like feel to it. For the most part, gamification has been a popular technique in both work and educational settings (Rajšp et al., 2017), as it promotes interactive content for users to engage independently or with their peers.

Current Study Based on the idea that gamification can keep people engaged for long periods of time even when the content is challenging (Mishra & Kotecha, 2017), the purpose of this study is to investigate the effects of gamification in

1434 Pham and Vu

online learning, specifically in instructional videos. Adding gamification to already existing instructional contents can help further support the idea that educational gamification can improve students’ comprehension of the course materials (Landers, 2014; Sanchez, Langer, & Kaur, 2020). This study will also determine whether gamification can affect student comprehension of easy versus difficult content. Gamification elements consisted of mini activities, such as drawing and matching games, and comprehensive quiz questions that are added at the end of the videos so that students could have an interactive experience while learning.

It is hypothesized that adding gamification elements onto instructional videos can lead to better comprehension of the lecture content, even when students are faced with challenging content materials. This hypothesis is based on the notion that the gamification elements can provide a more interactive and engaging environment for students. According to Çakıro˘glu et al. (2017), gamification elements, specifically the use of leaderboards, can increase engagement in students regardless of their position on the leaderboard. They suggested that it created a controllable environment among students to promote friendly competition. In addition to content difficulty, gamified instructional video is expected to help students tackle hard-to-understand course concepts, leading to better comprehension of the materials. Thus, it is hypothesized that for the easy-level content, there will not be much of a difference in students’ comprehension scores between the gamified instructional videos and the non-gamified instructional videos since the materials are already simple. However, it is predicted that for the hard-level content, gamified instructional videos can help lead to higher comprehension scores than compared to the non-gamified instructional videos since the gamification encourages more engagement with the course materials. Findings from this study can help provide guidelines for instructional designers about the effects of gamification.

METHODS

Participants Thirty-three participants were recruited from California State University, Long Beach (CSULB). Participants were recruited either through CSULB’s Psychology Introductory Psychology Research Participant pool or through IRB-approved recruitment flyers. Participants were compensated with course credits or a $20 gift card. Data from one participant was excluded and not replaced due to their low quiz scores (under 40%). Therefore, the final data set consisted of thirty-two participants (male = 13, female = 19) from the ages of 18 to 45 (M = 23.44, SD = 5.47). Of those thirty-two participants, fifteen identified as Asian (46.9%), eleven identified as Latino or Hispanic (34.4%), four identified as Caucasian (12.5%), one identified as African- American (3.1%), and one identified as Native Hawaiian or Pacific Islander (3.1%). When asked if they liked to play video games, 21 participants responded with “yes” (65%).

Gamifying Instructional Videos Did Not Lead to Better Student Comprehension 1435

Materials

The study was conducted on both Qualtrics, an online survey platform, and Nearpod, an interactive educational online platform. Materials on Qualtrics consisted of a consent form, a pre-test, and various questionnaires. A demographic questionnaire was used to obtain background information on participants, including their experiences and preferences when playing video games. A 1-item Paas Scale for Mental Effort was used to measure how much mental effort participants invested to study the information presented in the instructional video (Pass, 1992). A 7-item Video-Based Learning Activity Engagement Measure was used to measure how participants engage with the videos and their purpose in doing so (Seo et al., 2021). A 10-item System Usability Scale (SUS) was used to measure how user-friendly and accessible the instructional videos were for participants (Lewis et al., 2018).

Materials on Nearpod consisted of instructional videos, comprehension quizzes, and mini activities. The instructional videos were based on an upper- division, undergraduate-level Psychology Cognition course on the topic of visual perception. The hard content consisted of concepts that are more complex and have subtopics that build up to explain the overall topic of the lesson. The easy content consisted of concepts that are simple (e.g., definition-based and identifying differences between the concepts). The 40- minute video was cut into shorter clips using Camtasia, a video-editing software, so that they could be uploaded onto Youtube, an online video sharing and social media platform, to later be embedded onto Nearpod. For the gamification condition, the video was split into ten clips, ranging from one to five minutes, with five clips representing the easy content and the other five representing the hard content. For the non-gamification condition, the video was cut into four clips, ranging from eight to nine minutes, with two clips representing the easy content and the other two representing the hard content. Comprehension quiz questions were added to the end of each video. The gamification condition also had additional mini activities, such as drawing and matching games (see e.g., Figure 1), to encourage more engagement with the learning materials.

Figure 1: Example of a matching activity in the gamification condition.
Original PDF Figure 1:

1436 Pham and Vu

Design The study utilized a 2 (Gamification Level: Gamification or No Gamification) x 2 (Content Difficulty: Easy or Hard) mixed design, with Gamification Level being a between-subjects factor and Content Difficulty being a within- subjects factor. For the Gamification Level, participants were randomly assigned to either the gamification condition or the non-gamification condition. The gamification condition had activities embedded (see Figure 2), including comprehension quizzes designed in the form of a game and mini drawings or matching activities to encourage interactions and engagement with the learning materials. The non-gamification condition did not include any mini activity but the comprehension quiz questions were the same, with the questions being in a plain multiple-choice format. For Content Difficulty, the level of difficulty for the videos was determined through the complexity of the topics shown. The dependent measures of this study were the participants’ comprehension of the lecture materials, which was measured through their percentage of correct responses on the quizzes, and their engagement during the videos, which was measured through their interactions with the content and questions on the video.

Figure 2: Flowchart illustrating the tasks and activities for the gamification condition.
Original PDF Figure 2:

Procedure The study was conducted in-person using a protocol approved by the university Institutional Review Board (IRB). Participants completed an informed consent form on Qualtrics. They were then given a pre-test on visual perception to assess their prior knowledge of the topic. After the pre-test, they were redirected to Nearpod so that they could start watching instructional videos and complete the comprehension quizzes at their own pace. In the non-gamification condition, the instructional video was split into four clips that ranged from eight to nine minutes, with comprehension quizzes at the

Gamifying Instructional Videos Did Not Lead to Better Student Comprehension 1437

end of each clip to assess their learning. In the gamification condition, the instructional video was split into ten clips that ranged from one to five minutes, with either mini activities for engagement or comprehension quizzes embedded (see Figure 2). After participants completed the video condition that was assigned to them, they were then redirected back to Qualtrics to complete several questionnaires: Demographic survey, the Paas Scale for Mental Effort, the Video-Based Learning Activity Engagement Measure, and the System Usability Scale (SUS). Once participants completed all the questionnaires, they were given a verbal debrief on the study and given the opportunity to ask questions before the session ended.

RESULTS

Duration On average, participants took 67 minutes (range: 53–82 minutes) to complete the study as they were allowed to self-pace and were not given a time limit.

Student Comprehension The comprehension quiz scores were calculated as the percentage of correct responses. The mean quiz scores were submitted to a 2 (Gamification Level: Gamification or No Gamification) x 2 (Content Difficulty: Easy or Hard) mixed ANOVA. Gamification Level was a between-subjects factor and Content Difficulty was a within-subjects factor. The analysis revealed no significant main effects or interactions, with overall accuracy scores ranging from 70-78% across conditions (see Figure 3).

Figure 3: Mean comprehension scores as a function of Gamification Level and Content Difficulty. The error bars represent one standard error of the mean.
Original PDF Figure 3:

System Usability Scale (SUS) An independent-samples t-test was conducted on the SUS scores. The analysis did not reveal a significant difference in the average SUS scores between participants in the non-gamification condition (M = 76.406, SD = 13.904)

1438 Pham and Vu

and participants in the gamification condition (M = 82.969, SD = 13.299), t(30) = −1.364, p = .183. A one-sample t-test showed that the average SUS score for all participants (M = 79.69, SD = 13.792) was significantly higher than the score of 68, the cutoff for acceptable usability (Lewis, 2018), t(31) = 32.684, p <.001.

Paas Scale for Mental Effort An independent-samples t-test was conducted on the Paas Scale for Mental Effort scores. The analysis revealed that the effect of gamification was not significant, t(30) = 0.427, p = .673. Mental effort scores for participants who did not receive gamified instructional videos (M = 5.94, SD = 1.806) did not differ significantly from those who received the gamified instructional videos (M = 5.69, SD = 1.493).

Video-Based Learning Activity Engagement When watching instructional videos, participants reported that they typically paused the video because they needed time to think and reflect on what they just watched (62.5%, n = 20), they wanted to write a note (59.4%, n = 19), or something else was grabbing their attention/they needed a break (56.3%, n = 18). Participants would “fast” forward when watching the video because they found the content to be irrelevant/boring/already known to them (46.9%, n = 15) but they would navigate backward when they zoned out/got distracted and wanted to make sure that they did not miss anything (90.6%, n = 29) or they did not understand the explanation the first time (81.3%, n = 26). While many participants indicated that they typically highlighted sections of the video because it would be easier for them to review their annotations than to rewatch the entire video (46.9%, n = 15), others reported that this action did not apply to them (40.6%, n = 13). When asked if participants annotated on the video itself, half responded that this action did not apply to them (50%, n = 16), but some responded that they annotated if they wanted to remember something important for an assignment or test (40.6%, n = 13). Participants also indicated that they would search the filmstrip and/or transcript of the video because they wanted to locate a specific information to navigate to (59.4%, n = 19), they like to read the transcript while watching the video (43.8%, n = 14), or they want to better understand what was being said in the video (37.5%, n = 12). When asked why they would change the playback speed of the video to go either faster and/or slower, participants either claimed that speeding up the playback saves them time (75%, n = 24) or that the speaker was talking too fast/slow (65.6%, n = 21).

Discussion

The goal of the present study was to examine whether gamification influenced students’ engagement and comprehension of college-level course materials. It was initially hypothesized that adding gamification elements onto instructional videos would lead to better comprehension of the lectures. This hypothesis was based on the assumption that if students are provided

Gamifying Instructional Videos Did Not Lead to Better Student Comprehension 1439

with a more interactive and engaging environment to learn, they would be more motivated to learn. The results of the present study, however, did not support this hypothesis. We found that gamifying instructional videos did not have much of an impact on students’ comprehension of the lecture content. Our hypothesis that gamification of instructional videos would be more beneficial for learning hard concepts compared to easy ones was also not supported. A possible reason for these outcomes is that the activities within this study were not engaging enough to result in changes in comprehension. That is, the gamification elements used in this study may been too simplistic or not challenging enough for college-level students since Nearpod is an educational online platform initially developed for K-12 students. In addition, participants’ interest in the content shown could have declined over time in the study, leading to lower motivation to perform well on the quizzes.

It was also hypothesized that gamification would result in higher perceptions of usability, better mental effort devoted to the task, and higher levels of engagement. However, little differences were found between the gamification and non-gamification conditions for SUS scores, mental effort scores, and the activity engagement survey.

Limitations The present study had several limitations. The first is related to the types of gamification elements used in the present study, which as aforementioned, may have been too simple to motivate students at a college-level. Second, the type of quiz questions in the gamification condition may not have been in a form that was perceived as engaging to participants. In addition, while participants were taking the quizzes in the gamification condition, the game had preset background music that could have interfered with the students’ ability to focus. Third, the point system within the game also generated some confusion as large arbitrary numbers were awarded for answering a quiz question correctly. A leaderboard system was incorporated into each comprehension quiz, but it only revealed the top 3 players, so participants who did not make it to the top 3 did not know their overall ranking, and that could have also influenced their motivation. Finally, the sample was small and students studied the materials and were tested in a single session, which does not reflect actual learning/testing timeframes for online courses.

Future Research and Implications for Design Future research should include a variety of gamification elements and activities to determine which are more effective at the college-level. For example, incorporating multiplayer functions into educational online platforms may encourage students to continue learning because it can provide interactions between their peers. Use of teamwork games can also allow students to work with each other and learn how to strengthen their relationships in the process or through competitions so that the students can have fun trying to challenge each other. In addition, a weekly leaderboard can be used to identify which students might be struggling with the weekly lesson.

1440 Pham and Vu

Instructors can then use that information to provide additional materials to help these students improve in specific areas.

CONCLUSION

Although prior studies have shown that gamifying lectures can make learning more fun and enjoyable for students, the present study did not show a benefit in terms of higher quiz scores for gamified instructional videos. Future research should explore specific aspects of educational gamification for content at the college-level.

REFERENCES

Çakıro˘glu, Ü., Ba¸sıbüyük, B., Güler, M., Atabay, M., & Memi¸s, B. Y. (2017).

Gamifying an ICT course: Influences on engagement and academic performance. Computers in Human Behavior, 69, 98–107. https://doi.org/10.1016/j.chb.2016. 12.018. Christenson, S. L., & Anderson, A. R. (2002). Commentary: The centrality of the learning context for students’ academic enabler skills. School Psychology Review, 31(3), 378–393. https://doi.org/10.1080/02796015.2002.12086162. Deterding, S., Sicart, M., Nacke, L., O’Hara, K., & Dixon, D. (2011). Gamification.

Using game-design elements in non-gaming contexts. In CHI’11 Extended Abstracts on Human Factors in Computing Systems (pp. 2425–2428). https:// doi.org/10.1145/1979742.1979575. Dewan, M., Murshed, M., & Lin, F. (2019). Engagement detection in online learning: A review. Smart Learning Environments, 6(1), 1–20. https://doi.org/10.1186/s40561–018-0080-z. Keengwe, J., & Kidd, T. T. (2010). Towards best practices in online learning and teaching in higher education. MERLOT Journal of Online Learning and Teaching, 6(2), 533–541. https://jolt.merlot.org/vol6no2/keengwe_0610.pdf. Lamprinou, D., & Paraskeva, F. (2015). Gamification design framework based on

SDT for student motivation. In 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL) (pp. 406–410). IEEE. https://doi.org/10.1016/j.compedu.2017.11.006. Landers, R. N. (2014). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752–768. https: //doi.org/10.1177/1046878114563660. Lewis, J. R. (2018). The system usability scale: Past, present, and future. International

Journal of Human–Computer Interaction, 34(7), 577–590. https://doi.org/10. 1080/10447318.2018.1455307. Mishra, R., & Kotecha, K. (2017). Students engagement through gamification in education gamifying formative assessment. Journal of Engineering Education Transformations, 30(Special Issue). https://journaleet.in/articles/students-engage ment-through-gamification-in-education-gamifying-formative-assessment. Pass, F. G. W. C. (1992). Training strategies for attaining transfer of problem-solving

skill in statistics: A cognitive load approach. Journal of Educational Psychology, 84(4), 429–434. https://doi.org/10.1037/0022–0663.84.4.429. Rajšp, A., Beraniˇc, T., Heriˇcko, M., & Horng-Jyh, P. W. (2017). Students’ perception of gamification in higher education courses. In Central European Conference on Information and Intelligent Systems (pp. 69–75). Faculty of Organization and Informatics Varazdin.

Gamifying Instructional Videos Did Not Lead to Better Student Comprehension 1441

Sanchez, D. R., Langer, M., & Kaur, R. (2020). Gamification in the classroom: Examining the impact of gamified quizzes on student learning. Computers & Education, 144, 103666. https://doi.org/10.1016/j.compedu.2019.103666. Seo, K., Dodson, S., Harandi, N. M., Roberson, N., Fels, S., & Roll, I. (2021).

Video-based learning activity engagement measure [Database record]. Retrieved from PsycTESTS. https://dx.doi.org/10.1037/t81381–000. Singh, V., & Thurman, A. (2019). How many ways can we define online learning?

A systematic literature review of definitions of online learning (1988–2018). American Journal of Distance Education, 33(4), 289–306. https://doi.org/10. 1080/08923647.2019.1663082.

Cover of the proceedings volume Human Factors in Design, Engineering, and Computing

Explore the complete digital volume

Human Factors in Design, Engineering, and Computing

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-75-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →