Incorporating Kahoot! in core engineering courses: Student engagement and performance

INCORPORATING KAHOOT! IN CORE ENGINEERING COURSES: STUDENT ENGAGEMENT AND PERFORMANCE

Victor Chernov1 , Sivan Klas2 , Yael Furman Shaharabani3

1Department of Mechanical Engineering, ORT Braude College, Karmiel (Israel)
2Department of Biotechnology Engineering, ORT Braude College, Karmiel (Israel)
3Department of Teaching and General Studies, ORT Braude College, Karmiel (Israel)

Received March 2021

Accepted April 2021

Abstract

Technology-enriched lessons can contribute to improving student engagement and learning in engineering courses. In this study we systematically incorporated Kahoot!, a game-based response system, in two mandatory content-intensive undergraduate engineering courses. In both courses, short quizzes were incorporated regularly at the beginning of the lesson. The questions were mostly about concepts, and students earned bonus points. At the end of the course, the students filled in an anonymous survey regarding their perceptions of the Kahoot! part of the lessons. In addition, the students’ grades in the final test were compared to their gains in the Kahoot! quizzes. We found that student satisfaction with the systematic incorporation of the Kahoot! quizzes was high in both courses. Most students were motivated to review the course content before class and about half were motivated to attend class. Most students perceived Kahoot! quizzes as contributing to their understanding of the course content. We also found a positive association between student success in the quizzes and their performance in the final exam in both courses. The frequent Kahoot! quizzes supplied continuous feedback to the students and lecturers, and the students became more engaged in the course content. Our findings indicate that technology-enriched strategies, specifically gamification techniques, contribute to increased student engagement, motivation, and performance in content-intensive engineering courses.

 

Keywords – Engineering education, game-based learning, student engagement, immediate feedback, continuous assessment, Kahoot!

To cite this article:

Chernov, V., Klas, S., & Furman Shaharabani, Y. (2021). Incorporating Kahoot! in core engineering courses: Student engagement and performance. Journal of Technology and Science Education, 11(2), 486‑497. https://doi.org/10.3926/jotse.1269

 

----------

    1. 1. Introduction

Technology-enriched lessons can contribute to improving students’ engagement and learning in engineering courses (Plump & LaRosa, 2017; Vial, Nikolic, Ros, Stirling & Doulai, 2015). Recently, a massive observational study of undergraduate Science, Technology, Engineering, and Mathematics (STEM) education found that conventional lectures are prevalent throughout the curriculum (Stains, Harshman, Barker, Chasteen, Cole, DeChenne-Peters et al., 2018). There is evidence of the positive effect of active learning on student gains in STEM disciplines (Freeman, Eddy, McDonough, Smith, Okoroafor, Jordt et al., 2014). Yet, most lectures in engineering courses still lack active learning experiences and the continuous feedback required for enhancing students’ learning and developing professional competencies needed for their future employment.

Technological tools involving gamification may enrich STEM lessons and enhance students’ engagement and learning. Gamification is “the use of game design elements in non-game contexts” (Deterding, Dixon, Khaled & Nacke, 2011). Gamification techniques are a means to increase student engagement and motivation (Goshevski, Veljanoska & Hatziapostolou, 2017; Leaning, 2015) by integrating game elements in educational activities. For example, students’ perceptions of motivation, attention, and learning performances were more positive when their lectures included gamified student response systems activities (Barrio, Muñoz-Organero & Soriano, 2016). Excitement and enjoyment of game-like activity act as a leverage for student engagement and motivation. A review of gamified learning in higher education (Subhash & Cudney, 2018) indicated improved student engagement and performance as the most significant benefits, and revealed the lack of research of gamification in engineering education.

Kahoot! is a game-based response system that is designed for promoting motivation in the classroom setting (Wang & Lieberoth, 2016). The students use their mobile devices to answer short quizzes that incorporate participation-enhancing elements, such as on-screen questions and distribution of answers and points, limited response time, immediate feedback, and music. Previous studies demonstrate that Kahoot! quizzes engage students in learning and provide them with immediate feedback, thereby improving the quality of learning (Barrio et al., 2016; Licorish, Owen, Daniel & George, 2018). The Kahoot! answers convey the level of students’ knowledge and understanding and immediately draw the lecturer’s attention to shortcomings. The immediate feedback to the students offers formative ongoing assessment.

Student engagement is an evolving concept taking many forms (Groccia, 2018; Mandernach, 2015). Earlier definitions highlighted different aspects of engagement, while recent works composed an integrated definition of engagement as a multidimensional concept encompassing behavioral, emotional, and cognitive components (Groccia, 2018; Mandernach, 2015). Specifically, Groccia (2018) described student engagement as participation, effort and persistence (behavioral), experiencing interest, enjoyment, and motivation (affective), and processing the experience and linking it to previous experiences (cognitive). He indicated that it is possible for a student to be positively engaged on one or more levels, but negatively engaged on other dimensions. Hodges (2020) suggested that behavioral and emotional engagement promotes motivation, therefore is necessary for cognitive engagement. Kahoot! quizzes may engage students behaviorally, affectively, and cognitively.

To the best of our knowledge, there is limited research on students’ engagement and learning in technology-enriched gamification teaching in general and specifically in engineering courses. In this study we systematically incorporated Kahoot! in two content-intensive engineering courses. Two main research questions were examined:

  1. A)How do engineering students perceive the influence of the quizzes on their learning experience? In particular, how do the students view aspects of their motivation, engagement, and understanding?  

  2. B)How do participation and success in the quizzes relate to students’ performance in the final course tests? 

2. Methods

2.1. Course Description

Kahoot! quizzes were introduced in two mandatory content-intensive thermodynamics undergraduate (B.Sc.) courses at ORT Braude College of Engineering. The semester was 14 weeks long, and attendance was not mandatory for either course. The final grades in both courses were on a 100-point scale, with 55 being the passing grade. The first course was fourth-semester Thermodynamics at the Department of Mechanical Engineering (Thermo-M). The course consisted of two weekly sessions: a lecture of three academic hours (where an academic hour is 50 minutes long) and a tutorial of two academic hours. Eighty-nine students were enrolled and could choose between two lecture sessions and three tutorial sessions. The second course was third-semester Thermodynamics and Kinetics at the Department of Biotechnology Engineering (Thermo-B). The course consisted of two weekly sessions: a lecture of two academic hours and a tutorial of two academic hours. Twenty-eight students were enrolled (Table 1).

2.2. Kahoot! Quiz Incorporation

The courses had different lecturers, each of whom incorporated Kahoot! in their own way. In both courses, the short quizzes were incorporated regularly at the beginning of the lesson. The questions were mostly about concepts, and students could gain bonus points by answering correctly.

Thermo-M. Kahoot! quizzes consisted of three questions and were incorporated twice a week at the beginning of each lesson (both lectures and tutorials). Two questions addressed the content of the previous lesson and one addressed an earlier topic. The questions were mostly about concepts, but occasionally an easy calculation was required (several examples are provided). A quiz winner was the student who received the highest score amongst all players, i.e., answered all questions correctly and quickly. A quiz winner was granted a bonus of one point added to the final grade. Additionally, in order to reduce student speed as a major success factor, the number of correct answers given by each student was calculated at the end of the semester, and the top ten students were granted five bonus points to their final grade. A student was eligible for up to ten bonus points.

Examples of the Kahoot! questions:

  • When can the (gravitational) potential energy of a gas be neglected? 

    1. 1.Never 

    2. 2.For high temperatures 

    3. 3.For high pressures 

    4. 4.Always, except for height differences of at least a kilometer 

  • For a liquid, what value does k take? 

    1. 1.k=1 

    2. 2.k=0 

    3. 3.Depends on the liquid 

    4. 4.Does not exist in liquids 

  • What is the order of operation of a four-stroke engine? 

    1. 1.Intake -> Ignition -> Compression -> Exhaust 

    2. 2.Exhaust -> Intake -> Ignition -> Compression 

    3. 3.Compression -> Ignition -> Intake -> Exhaust 

    4. 4.Intake -> Compression -> Ignition -> Exhaust 

Thermo-B. The Kahoot! quizzes consisted of four to six questions and were incorporated typically once a week at the beginning of the lecture. The questions addressed concepts and definitions discussed in the previous lesson. All the students who answered all the questions in a quiz correctly were granted one bonus point for their final grade up to a maximum of five points. Examples of the questions:

  • A reversible process is a process  

    1. 1.In which the system is restored to its initial state 

    2. 2.In which no changes occur in the surroundings 

    3. 3.That is carried out under equilibrium conditions 

    4. 4.That occurs in the lab only under stringent conditions 

  • All processes in the Carnot cycle are 

    1. 1.Adiabatic 

    2. 2.Isothermal 

    3. 3.Real 

    4. 4.Reversible 

  • In a formation reaction 

    1. 1.The products are carbon dioxide and water 

    2. 2.There are no products, since it is theoretical 

    3. 3.There must be two reactants only 

    4. 4.The reactants can only be elements 

Table 1 summarizes the courses’ main characteristics and grading methods.

 

Thermo-M

(Mechanical Engineering)

Thermo-B

(Biotechnology Engineering)

Class size
(no. of students enrolled)

89

28

Weekly class hours

150-minute lecture +

100-minute tutorial

100-minute lecture +

100-minute tutorial

Attendance

Not mandatory

Grade composition

100% final exam +

Kahoot! bonus points

80% final exam +

20% midterm +

Kahoot! bonus points

Kahoot! bonus

1 point per win; 5 points for the
10 students with most correct answers

Max. bonus: 10 points

1 point for every perfectly
answered quiz

Max. bonus: 5 points

Table 1. Summary of the courses’ main characteristics and grading

Since success in the Kahoot! quizzes added to the final grade in both courses, the students had to identify themselves either by their true name or by a consistent nickname. The quizzes usually introduced an “action” atmosphere to the lesson. The students requested further clarifications in the case of a distribution of answers to a question, thus provoking discussion in class that clarified difficult topics or concepts.

2.3. Data Sources and Analysis

At the end of the semester the students were asked to answer an anonymous survey regarding the Kahoot! part of the course. This survey included six questions: five closed-ended and one optional openended. The survey questions were:

  1. 1.On a scale between 1 and 10, how satisfied were you with Kahoot! quizzes, where 1 is extremely dissatisfied and 10 is extremely satisfied. 

  2. 2.Did you review the course content before the lesson to increase your chances of success in the Kahoot! quiz?  

    Only one answer could be chosen. The possible answers were: 

    • Yes, before every class, except perhaps once or twice 

    • I made an effort to review the course content before each class, but didn’t always have time 

    • Occasionally  

    • No 

  1. 3.Was the Kahoot! quiz a consideration in your decision whether to attend the class?  

    Only one answer could be selected. The answers were: 

    • Yes, it was the primary consideration 

    • Yes, it was a secondary consideration 

    • No 

  1. 4.Has your participation in the quizzes made you understand the course content better? 

    Only one answer could be selected. The answers were: 

    • Yes, and it was the primary factor in understanding 

    • Yes, and it was a secondary factor in understanding 

    • No, there was no influence on understanding 

    • No, and the quizzes only confused me a little 

    • No, and the quizzes significantly hampered my understanding 

  1. 5.The option for Kahoot! bonus points led mostly to:  

  1. Students could choose as many answers as they saw fit. This question tried to assess the reaction of the students to the quizzes in a more detailed way.

    The options were:

    • Motivation to study 

    • Enjoyment 

    • Competition between students 

    • Stress 

    • I would rather not have bonus points at all 

    • Other (here the student could fill their own response) 

  1. 6.Optional: Students were invited to add remarks about the incorporation of Kahoot! in the course. 

In addition, the students’ grades in the courses’ final test were compared to their gains in the Kahoot! quizzes. We performed two ANOVA tests. The first ANOVA test was on the final test grades, taking group variable Kahoot! points (without points vs. with Kahoot! points), as a between-subject variable. A second ANOVA test was performed on the final test grades, taking the group variable number of Kahoot! points (under 60% of maximum points vs. 60% of maximum points and above) as a between-subject variable. Supplementary to the above, the lecturers’ personal insights and notions about students’ participation and cooperation concerning Kahoot! were collected. The students’ voice (survey question #6) and lecturers’ insights are incorporated in the description of the findings.

3. Findings and Discussion

This section is divided into two subsections, each addressing a research question posed in the introduction.

3.1. Student Perceptions of the Kahoot! Quizzes’ Influence on their Learning Experience

This section presents findings regarding the first research question: How do engineering students perceive the influence of the quizzes on their learning experience? In particular, how do the students view aspects of their motivation, engagement, and understanding?

At the end of the semester students completed a survey of five questions regarding the Kahoot! part of the course. The survey was voluntary, and the participation rate was 28% (25 students) and 43% (12 students) for Thermo-M and Thermo-B respectively.

3.1.1. Students’ General Satisfaction

Students’ answers to the survey question #1 indicated their general positive view of the Kahoot! quizzes. The average ranking in both courses was high. For Thermo-M the average ranking was 8.36, with 14 students ranking the quizzes at 10. Two graded this question with 1 and one student with 5. For Thermo‑B, the average ranking was 8.58 with the lowest being 7. The high average rankings in both courses reflect the positive appreciation of the Kahoot! in the courses. These findings are similar to findings in a recent study of gamification in a problem-based learning software engineering course, in which most of the students rated the use of games in the course at the highest possible rank (Ivanova, Kozov & Zlatarov, 2019).

Students had the option to add remarks about the incorporation of Kahoot! in their course (survey question #6). Several students who had negative views of the Kahoot! quizzes pointed out two issues. The first was that they experienced this activity as stressful. They felt there was pressure on them to perform well in the quizzes and felt uncomfortable about this. The second issue regarded the timing aspect of the quizzes. The timing aspect was twofold. The first was that the winner in Thermo-M was determined by giving correct answers faster than other participants. The slower students were the reason for the five bonus points given to the best 10 students at the end of the course (Thermo-M), regardless of their speed. We assume that this remark was probably an expression of frustration by students who were less successful in the Kahoot! quizzes. The second aspect is that every question had a time limit (for most questions thirty seconds, although some questions were allotted two minutes). Some students regarded this as stressful and/or unfair. While this frustration is understandable, the time limit was inherent to the game. A conscious effort was made by the lecturers to allocate sufficient time for everyone to answer. An example of a positive remark is: “Some of the Kahoot! questions indicated possible mistakes I would have done in a stressful situation”. This remark demonstrates a positive aspect in giving the students ongoing feedback during their studies. The repetitious Kahoot! quizzes provided a continuous formative assessment.

3.1.2. Students’ Motivation to Review the Content and Attend Classes

Survey questions #2 and #3 indicated students’ motivation to review the course content prior to the lesson and to attend the lessons (attendance was optional) in relation to the Kahoot! quizzes.

Figure 1 presents the answers to survey question #2 by Thermo-M and Thermo-B students. As can be seen, most of the students made at least some effort to review the course content before the class (76% of the students in Thermo-M, 66% of the students in Thermo-B). The systematic Kahoot! quizzes motivated the students to devote time to content reviewing, thus they came to class better prepared. One of the students remarked in answering survey question #6: “It’s a great way to encourage reviewing the course content before class. Kahoot! at the beginning of every class is a brain warm-up towards the lesson”.

The students’ answers to survey question #3 showed that in both the Thermo-M and Thermo-B courses the quizzes were a secondary consideration in the decision to attend class for about half the students (Thermo-M: 40%, Thermo-B: 50%). For the rest, it was not a consideration at all. No one considered it as the primary reason to attend the class. Since class attendance in the courses was optional, it was not recorded or compared with previous years, but the personal impression of the lecturers was that many students made an effort to attend on time in order to not miss the quiz.

The quizzes seemed to influence the students’ behavior, according to the lecturers’ observations. Some students declared that they came to class because of the quizzes. In addition, there were very few late arrivals (the quiz began precisely at the start of class). Furthermore, students would arrive at class with their phones open and ready to enter the game. Success in answering a quiz question was, in many cases, celebrated. Occasionally, the winning student took a picture of the winning result to post on social media. Some students even asked (and received) a selfie with the lecturer and the scores.

Students’ answers to survey question #5 indicated mostly motivation and enjoyment, hence supporting the data gathered from survey questions #2 and #3. The answer chosen most by Thermo-M students was “Enjoyment” (chosen by 13 students), followed by “Motivation to study” (11 students) and “Competition between students” (10 students). Thermo-B students gave similar answers; the leader was “Motivation to study” (7 students), followed by “Enjoyment” (6 students) and “Competition between students” (4 students). Hodges (2020) stressed the importance of behavioral and emotional engagement as a pre‑requisite for cognitive engagement. Our findings demonstrate that the consistent use of Kahoot! makes it a beneficial tool for promoting behavioral and emotional engagement.

 

Figure 1. Students’ course content review before class

It is important to mention that we have not stated whether the competition between students was positive or negative. It appears that most of the students viewed the competition inherent in Kahoot! as positive, since the “Stress” and “I would rather not have bonus points at all” options were only chosen by a few students (6 in Thermo-M and 2 in Thermo-B).

3.1.3. Student Perception of Kahoot! Contribution to Course Content Understanding

Students’ answers to survey question #4 are presented in Figure 2. The findings show that, by and large, students in both courses felt that Kahoot! quizzes improved their understanding of the course content. In both courses most students perceived the influence of Kahoot! as a secondary factor. However, in Thermo-B more positive answers were given (83%), compared to Thermo-M (60%).