Email this article 
Hide
Show all
THE EFFECT OF GENERATIVE AI TOOLS (CHATGPT, GEMINI, ETC.) ON STUDENTS’ ACHIEVEMENT AND THEIR MOTIVATION
TOWARDS LEARNING
, Maimoona Al Abri1
, Ahmed Mohamed Fahmy Yousef2
1Sultan Qaboos University (Sultanate of Oman)
2Instructional and Learning Technologies, College of Education, Sultan Qaboos University
(Sultanate of Oman) & Educational Technology Department, Faculty of Specific Education,
Fayoum University (Egypt)
Received March 2025
Accepted October 2025
Abstract
This study aimed to explore the effect of Generative Artificial Intelligence (GenAI) tools (ChatGPT, Gemini, etc.) on students’ achievement and their motivation towards learning. The participants of this study were 47 students enrolled in the course of “Educational Technology” in the Department of Instructional Technology at Sultan Qaboos University. The participants represented two sections (Sections 40 and 50), which were randomly selected. Section 40 consisted of 23 students, which represent the experimental group, while Section 50 consisted of 24 students, representing the control group. The study employed a quasi‑experimental design to investigate whether the use of GenAI applications (e.g., ChatGPT, Gemini) in teaching affects students’ performance in learning educational and information technology and their motivation to learn. A quantitative approach based on pre-post-tests and a questionnaire was adopted. The results of the study showed that the mean score value obtained in the post-test is 14.73, which is higher than the mean score value obtained in the pre-test (12.33). This result indicates that the use of GenAI in teaching had significant positive effect on students’ performance in learning educational and information technology. In addition, the findings showed that there is no statistically significant difference in students’ motivation scores between the experimental group and control group (t= 0.044, 0.97, p > 0.05). This indicates that the use of the GenAI tools in the learning activities and learning strategies applied to the experimental group did not have a significant effect on students’ motivation to learn compared to the control group. The findings of this study can have important implications for the promotion of GenAI tools, such as ChatGPT and Gemini. For example, future research should focus on exploring how using different GenAI tools into learning activities and instructional strategies may affect specific learning outcomes (e.g., creativity, critical thinking or problem solving in higher education).
Keywords – Generative artificial intelligence, Traditional teaching, Educational technology, Information technology, Academic achievement.
To cite this article:
|
Elhag, A., Al Abri, M., & Yousef, A.M.F. (2025). The effect of generative AI tools (ChatGPT, Gemini, etc.) on students’ achievement and their motivation towards learning. Journal of Technology and Science Education, 15(3), 746-759. https://doi.org/10.3926/jotse.3410 |
-
-
1. Introduction
-
1.1. Conceptualisation of Generative AI Tools
After the announcement of the Fourth Industrial Revolution at the World Economic Forum, emerging technology has rapidly evolved and reshaped education regarding how people learn and work (Schwab, 2016). As a result, these new technologies have been extensively integrated into teaching and learning practices, specifically Artificial Intelligence (AI) technologies. According to Collins, Dennehy, Conboy and Mikalef (2021) AI refers to a simulation of human intelligence processes by machines, particularly computer systems. It is a multi-use technology, with many different applications (McDonald, Shainock, O’Flaherty & Codio, 2024). The concept behind AI is to create technology that is capable of mimicking human behavior and thinking (Fetzer, 1990). AI systems can be classified based on different ways. The first one can be classified based on the way they learn. This includes machine learning and deep learning. The second way to categorize it, is based on its capability, which means that “the AI system delivers either artificial narrow intelligence (ANI), artificial general intelligence (AGI), or artificial super intelligence (ASI)” (McDonald et al., 2024: page 21). The third way to classify it is based on its functionality, which includes generative AI technologies (GenAI). Generative AI (GenAI) is regarded as a subset of AI and is focused on generating new content by using advanced machine learning techniques (Ramnarain, Ogegbo, Penn, Ojetunde & Mdlalose, 2024).
GenAI is a manifestation of this concept. However, it expands its role of completing complex tasks to be capable of generating new content (e.g., videos, text, images, stories, etc.) (Batista, Mesquita & Carnaz, 2024). As a result, these new technologies have a lot of potential in Higher Education and have been extensively integrated into teaching and learning practices. According to Hsu and Ching (2023) GenAI tools, defined as deep learning models trained on diverse datasets, such as large language models (LLMs), to process user prompts and create human-like outputs, such as ChatGPT, Gemini and Curipod. These technologies are increasingly used by instructors and students in educational settings and other sectors (Luckin & Cukurova, 2019).
1.2. Benefits of Using Generative AI as an Educational Tool
According to Erduran (2023), there is increasing interest in the use of AI in education. AI tools such as GenAI applications provide several services in educational context based on their capabilities, including personalizing education, generating content (e.g., text, images, educational videos, graphics, etc.), languages editing, translation, and providing adaptive learning (Gond, Upadhyay, Mishra & Bhardwaj, 2024; Nikolopoulou, 2024). This can make learning more personalized, interactive, and engaging, and help students build key skills for the 21st-century workforce (Ramnarain et al., 2024). It can also help to grade student work, give quick feedback, spot learning problems, and suggest ways to improve (Owan, Abang, Idika, Etta & Bassey, 2023). For instant, ChatGPT application is great at restructuring information and can help instructors in assessment tasks, such as producing rubrics (Estrada & Fabián, 2024). Moreover, these applications can be used by pre-service teachers to enhance their teaching strategies, engage students, prepare lesson plans, generate scenarios for educational videos, and adapt to the ever-changing educational landscape (Kehoe, 2023).
1.3. Previous Work
A wide range of GenAI tools such as ChatGPT and Gemini AI have been used in higher education for different educational objectives. Their effectiveness is being assessed through various studies that focus on their impact academic performance, engagement, motivation, and overall learning outcomes. More recent attention has focused on the effectiveness of GAI tools in teaching and learning practices. For instance, Gond et al., (2024) conducted a quantitative method to examine the effectiveness of ChatGPT and Gemini AI tools in students’ learning performance and engagement levels across various disciplines. Specifically, the study evaluated ChatGPT as personal assistance and support, and Gemini AI as content and assessment creation tool. This study revealed that both these AI technologies enhanced students’ understanding of complex concepts, sharing knowledge, and collaborative learning. In addition, students’ engagement and motivation toward learning have improved. Despite the above benefits, the study shed light on challenges such as students’ data privacy, accessibility, and adequate infrastructure. In addition, an extensive literature review that assessed Google Gemini’s features and its applications in education highlighted several utilizations for its use in education settings (Imran & Almusharraf, 2024). These utilizations reside in its potential to design personalized learning that is tailored to student’s learning needs. Furthermore, it can be used as a tutoring system that provides instant feedback to support students’ learning; and help educators with educational materials generation.
In a collaborative effort across multiple higher education institutions, Nikolic, Sandison, Haque, Daniel, Grundy, Belkina et al. (2024) evaluated the effectiveness of a variety of generative AI tools (ChatGPT-4, Copilot, Gemini, SciSpace, and Wolfram) across various assessment types that were commonly used in engineering education. This study aimed to select suitable tools for each type of assessment, including oral presentations, quizzes, visual assessments, programming, reflective essays, reports, project documentation, etc.). The findings of this study reported that based on the capabilities of the generative AI tools, they have identified their appropriateness for each type of assessment. In addition, the study appraised the potential integration of these tools in enhancing teaching and learning practices. However, concerns regarding academic integrity, mostly the issue of cheating, have been highlighted (Nikolic et al., 2024). Likewise, several studies highlighted some challenges and ethical considerations with the use of Google Gemini in teaching and learning such as the issue of protecting students’ information, accuracy of obtained information, and digital equity to these smart technologies (Ali, Murray, Momin, Dwivedi & Malik, 2024; Imran & Almusharraf, 2024). Sun and Zhou (2024) conducted to investigate the effectiveness of Gen-AI in improving the students’ achievement. Their findings found that Gen-AI significantly improved students’ achievement. Ironsi and Ironsi (2024) also conducted mixed method research aims to investigate the effectiveness of GenAI in improving students writing skills. They concluded that although ChatGPT may assist students in providing ideas in writing lessons, it may not improve their overall writing skills. Recent evidence reported that generative AI writing tools enhanced students’ creativity and innovation in writing skills. For example, these GAI tools supported students to generate new ideas and learn different writing styles. Furthermore, it helped students to brainstorm ideas and suggest new ways of writing the arguments in writing tasks. Besides, it gave students suggestions for vocabulary and improved sentence structure. As a result, these GAI contributed to enhance the overall students’ writing quality (Gasaymeh, Beirat & Abu-Qbeita, 2024). Other benefits of these tools were: Saving students time in generating content, helping non-native speakers to refine their work and learn from their mistakes to improve the quality of their work, helping in summarizing the research paper, and creating accurate citations. Surprisingly, this study showed that some students seek these tools to be useful because they encouraged them to critically analyze the generated content instead of only using them as it is. Accordingly, Yusuf, Nordin and Annamalai (2024) argued that responsible usage of GenAI applications can improve learning processes; however, addressing associated concerns may require strong policy that is responsive to cultural expectations.
The use of GenAI in teaching and learning is still emerging, however research has already shown a positive relationship between GenAI and student motivation (e.g., Shata & Hartley, 2024; Ramnarain et al., 2024; Mishra & Bhardwaj, 2024; Huang & Mizumoto, 2024; Song & Song, 2023). For example, Ramnarain et al., (2024) examined the factors influenced by pre-service Science teachers’ intention to integrate generative AI tools in an inquiry-based teaching approach. Their findings showed that pre-service teachers were motivated to use these smart technologies. That is, receiving significant support from their mentors positively affected their motivation to adopt GAI for their teaching practices, and thereby, these factors increased their confidence to integrate GAI tools into their teaching practices. Moreover, Mishra and Bhardwaj (2024) conducted study to examine the effectiveness of ChatGPT and Gemini AI tools in students’ learning performance and engagement levels across various disciplines. Specifically, the study evaluated ChatGPT as personal assistance and support, and Gemini AI as content and assessment creation tool. This study revealed that both these AI technologies enhanced students’ understanding of complex concepts, sharing knowledge, and collaborative learning. In addition, students’ engagement and motivation toward learning have improved. In addition, Huang and Mizumoto (2024) conducted study to examine the effect of generative AI on students’ motivation and writing self-efficacy. Their results demonstrated that ChatGPT was effective GenAI tool in maintaining student motivation. Song and Song (2023) conducted study to assess the efficacy of ChatGPT in enhancing students’ academic writing skills and motivation. They concluded that ChatGPT may enhance students’ motivation and writing skills. Cai, Lin and Yu (2023) also investigated the factors influencing learner attitudes towards ChatGPT-assisted language learning in higher education. Their findings showed that ChatGPT- assisted language learning encourages pleasure-based motivation. With the evolvement of these GenAI tools, the education landscape has been influenced by these technologies by innovating different pedagogy approaches. However, the effectiveness of these emerging technologies in teaching and learning is unclear. Questions have been raised regarding what the potential impacts of these GenAI technologies in teaching and learning are, how they should be used in classroom settings and how to adapt learning and assessment environments, what are the students and academics perceptions to integrate AI in education, as well as what are the regulations to make AI use more appropriate in this new age of GenAI (Alshamy, Al-Harthi & Abdullah, 2025; Shata & Hartley, 2024). However, empirical evidence to explore its effectiveness in teaching and learning in Omani context is required. Consequently, the current study investigates the effectiveness of GenAI tools (ChatGPT, Gemini, etc.) in teaching educational technology topics, their impact on student achievement, and their motivation towards learning.
1.4. Research Objectives and Questions
Based on the above reported literature, the main objective of this study to analyze the effect of GenAI tools (e.g., ChatGPT, Gemini) on students’ achievement and motivation towards learning educational and information technology course. More specifically, the study aimed to answer the following research objectives and questions.
1.4.1. Research Objectives
-
•To measure the effect of GenAI applications (e.g., ChatGPT, Gemini) on students’ academic performance in learning educational and information technology courses.
-
•To determine there any statistically significant differences in the mean scores of the students’ motivation to learn between the experimental and control groups.
1.4.2. Research Questions
-
•To what extent does the use of Gen AI applications (e.g., ChatGPT, Gemini) in teaching affect students’ performance in learning educational and information technology?
-
•Are there any statistically significant differences in the mean scores of the students’ motivation to learn between the experimental and control group?
2. Methodology
2.1. Research Design
The research design adopted in this study was a quasi-experimental design to investigate whether the use of the use of Gen AI applications (e.g., ChatGPT, Gemini) in teaching affect students’ performance in learning educational and information technology and their motivation towards learning. A quantitative approach also was used, based on implementing pre-posttests and questionnaires. This approach will enable the systematic collection of numerical data and allowing for measuring the objective of the effect of GenAI applications on students’ achievement and motivation. According to Creswell and Creswell (2017), quantitative methods are particularly suitable when the aim is to test hypotheses, compare groups, and determine the significance and magnitude of differences through statistical inference. Students enrolled in section (40) in the “Educational and Information technology” course was randomly allocated into the two groups of the treatment, with 23 students in the experimental group studied the course where GenAI technologies were integrated into the course curriculum (learning activities). While the other students in section (50) students allocated to the control group who studied through the traditional method without integrating GenAI tools into teaching activities and strategies.
2.2. Sample of the Study
The target population of this study consisted of all Education Qualification diploma students enrolled in the Educational and Information Technology course at the college of Education, Sultan Qaboos University in Oman) (N = 192). The total of the students was 192, divided into seven sections, including females and males, each section contains participants from 23 to 30 individuals. The sampling technique applied for this study was randomly selected, which is a probability sampling method that selects participants randomly to ensure the generalizability of findings by minimizing the potential for bias in selection and to control for the potential influence of known and unknown confounders (Creswell & Creswell, 2017; Palinkas, Horwitz, Green, Wisdom, Duan & Hoagwood, 2015). The sample of this study was 47 participants representing two sections (sections 40 and 50), which were randomly selected. Section 40 consisted of 23 students, both male and female while section 50 consisted only 24 students all are females. The researchers randomly assigned one section to represent the control group, while the other section represented the experimental group. Figure 1 presents the percentage distribution of the participants’ computer skills across the two groups.
Figure 1. Participants’ Computer Skill Levels in Control and Experimental Groups
2.3. Study Instruments
2.3.1. Achievement Test (Pre-and Post-tests)
Two instruments were adopted by the researchers in this study. The first tool is an achievement test (pre and post-test), which comprised 18 questions. The test concentrates on technological concepts knowledge and understanding, which was developed based on the course readings. The achievement test was presented to a group of expert judges from the Department of Educational Technology to ensure its validity and clarity of the questions. Then the test was modified based on the reviewers’ comments to include 15 items in its final format. The criteria that were used to review the test were as follows: (1) misalignment with the course learning outcomes and objectives and (2) the level of the difficulty, easy or difficult to the students, and (3) the clarity of the questions in terms of the wording. The reliability coefficient of the test was found 0.79, which was considered sufficient to use the instrument for this study.
2.3.2. Motivation Questionnaire
The study used the motivation questionnaire established by Keller (Keller & Kopp, 1987) which classifies motivation into four elements, Attention, Relevance, Confidence, and Satisfaction (ARCS model). These elements demonstrate how instructional design influences student motivation and provide strategies to enhance both motivation and active student engagement in learning. This scale has been used by several researchers (Afjar, Musri & Syukri, 2020). Loorbach, Peters, Karreman and Steehouder (2015) indicated that, the IMMQ questionnaire is a 36-item situational measures of people’s reactions to instructional materials in the light of the ARCS model. Therefore, in the context of this study we used and administered the IMMQ questionnaire as a posttest instrument to measure the students’ reaction to the instructional material after at the end of the treatment, after 9 weeks of studying in the Fall2024 semester. Cronbach’s alpha reliability was measured for the total questionnaire, and it was 0.89, and for each dimension were also calculated. Alpha Cronbach reliability for attention was 0.87, 0.82 for relevance, 0.72 for confidence, and 0.65 for satisfaction.
2.4. Procedures
The study treatment took about 8 weeks, with 4 teaching hours per week, and both groups were taught the same content. The first lecture was given to students in the first two weeks as students do not have prior knowledge of the concepts of educational technologies. Then, the pre-test was carried out for students in week 4 for both groups. From Week 4 forward, GenAI technologies were integrated into the course curriculum and incorporated with the experimental group into their learning based on the designed activities (e.g., scriptwriting, video production, and GenAI application).
In the first stage, students were required to design an interactive lesson plan using the ADDIE phases, by going through the phases of the Analysis (analyzing and identifying the learners’ needs), Design (stating their objectives and structuring the activities), Development (producing the learning materials), Implementing (presenting the lesson) and Evaluation (assessing and giving feedback). Then, they were asked to move this lesson plan to an actual project using presentation features. They were exposed to utilizing the CANVA AI features in developing the interactive presentation whether to use PowerPoint with utilizing CANVA AI or they can use CANVA to create the entire presentation.
The students designed AI generated mind maps by selecting a topic related to their major using CANVA AI and Mapify. As part of their assignment, they also requested to produce an interactive educational video. Students followed the following steps:
-
a)Script writing stage using ChatGPT
-
b)Ask ChatGPT to write a script suitable for the chosen topic. Be specific about the elements you want to include such as: Video length (e.g., 2 minutes), Language style (e.g.: simple, aimed at children), general structure (introduction, body, conclusion), make sure the script is interactive, so that it contains questions or challenges directed to viewers to get them involved in thinking about the topic.
-
c)Review and modify the script: A basic requirement for the assignment is to review the script provided by ChatGPT. Add your ideas and modify the parts to suit your educational style and add more details.
-
d)You need to state the aspects that were modified.
The second stage focused on video design and production, where the students created and edited their educational videos using CANVAI.
In the final stages, the students exported and published their video in high quality and used some AI tools such as Edpuzzle - NearPod to add interactivity into their video before sharing their final products. Moreover, students were asked to select GAI technology and prepare a presentation to share the selected technologies with the class. Students should present the GAI tool that can be used in their related major, how it works, and how it can be integrated to engage students in the learning process. At the end, students will have a toolkit of GAI technologies that can be implemented in teaching their subjects.
At the end of the experimental treatment, a post-test was conducted on the experimental group to assess students’ academic achievement. Additionally, a motivation questionnaire was also administered both groups at the end of the experiment.
2.5. Data Analysis
The data collected in this study from the pre-test and post-test and the motivation survey was analysed using both descriptives statistics (means and standard deviations) and inferential statistics (independent samples t-tests and paired sample t-test). The inferential statistics were utilized to determine whether there are statistically significant differences for the performance scores for the experimental group and between the experimental and control groups for the motivation. Moreover, effect size was calculated using Cohen’s d to assess the effect size of the level of GenAI’s effectiveness in improving student performance.
3. Results
3.1. The Students’ Academic Performance
The researchers conducted a pertest with the two groups before the starting of the experiment to ensure the equivalency of the groups’ performance scores between the two groups. The arithmetic means and the standard deviations were calculated. The mean and standard deviation of the scores were 11.48 and 1.47 for the control group, and 12.33 and 1.43 for the experimental group, respectively. The students’ performance scores were measured to test the mean differences between the two groups using an independent sample t-test. Table 1 shows the findings.
|
Group |
N |
df |
Mean |
SD |
t |
Sig. |
|
Control |
24 |
45 |
11.48 |
1.47 |
-2.016 |
0.150 |
|
Experimental |
23 |
|
12.33 |
1.43 |
|
|
Table 1. Descriptive statistics and t-test values of performance scores for the groups (pre-test)
As shown in Table 1, no statistically significant differences were found the two groups in their performance scores, with a value of (t = -2.016, (0.05)). Therefore, the control and experimental groups were deemed equivalent prior to the treatment.
To address the first research question “To what extent does the use of Gen AI applications (e.g., ChatGPT, Gemini) in teaching affect students’ performance in learning educational and information technology?”, a Paired Sample T-test was conducted to compare means scores of the achievement tests before and after the treatment for the experimental group. Table 2 below illustrates the mean differences between the pre-post-test performance scores in the experimental group.
|
Test |
N |
df |
Mean |
Std. Deviation |
t |
Sig. (2-tailed) |
|
Pre-test |
23 |
23 |
12.33 |
1.44 |
-4.78 |
0.001 |
|
Post-test |
23 |
|
14.73 |
1.97 |
|
|
Table 2. Descriptive statistics and Paired Sample T-test values of performance scores for the experimental group
Table 2 shows that there are statistically significant differences in the means score of the students’ performance in the post-test between the pre-post-test performance scores in the experimental group (t = 4.78, 0.001, p < 0.05) in favor of the post test score. The table illustrated that, the mean score value obtained by the post-test is (14.73), which is higher than the mean score value obtained by the pertest mean (12.33). This result demonstrated that the use of GenAI tools in teaching positively contributes to students’ performance in learning educational and information technology.
To assess the effect size of the level of GenAI’s effectiveness in improving student performance. Cohen’s d was applied to determine the standardized difference between the means of the two tests, using the following formula:
 |
(1) |
The analysis revealed that the differences between the pre-test and post-test of the experimental group had a large effect size (Cohen’s d = 0.9), indicating that the use of GenAI tools had a strong impact on students’ performance.
3.2. Students’ Motivation
To address the second research question, which asked: “Are there any significant differences in the mean scores of the students’ motivation to learn between the experimental and control group?”, a questionnaire was applied for the study groups to measure their motivation towards learning using. An Independent T‑test was conducted to compare means scores of the motivation for the control and experimental groups. Table 3 shows the descriptive statistics and t-test values of motivation scores for the groups.
|
Group |
N |
df |
Mean |
Std. Deviation |
t |
Sig. (2-tailed) |
|
Control |
17 |
31 |
3.83 |
0.48 |
0.04 |
0.975 |
|
Experimental |
16 |
|
3.82 |
0.46 |
|
|
Table 3.Descriptive Statistics and T-test for Groups Differences in Motivation Scale
Table 3 indicates that there is no statistically significant difference in the average motivation scores among students between the experimental group and control group (t= 0.044, 0.97, p > 0.05). This indicates that the GenAI tools, activities or strategies used to the experimental group did not have significant effect on students’ motivation to learn educational and information technology compared to the control group. This could be justified by the fact that both sections (the experimental and control groups) may have already had similar levels of motivation to learn educational and information technology before the of the study began.
Although the previous results of the first question showed that the use of GenAI tools in instructional activities and teaching methods contributed positively to the learning performance of the experimental group students, the results of the second question indicate that the use of GenAI tools in instructional activities and the teaching strategies applied to the experimental group did not have a significant impact on students’ motivation to learn compared to the control group.
4. Discussion
4.1. The Students’ Academic Performance
This result demonstrated that the use of GenAI tools in teaching positively contributes to students’ performance in learning educational and information technology. This could be explained by the fact that using GenAI tools in teaching educational and information technology through the various activities applied with students in the experimental group helped them to understand the course topics and concepts related to instructional media design. The learning activities that have been conducted with the students were: designing an interactive lesson plan using “Curipod”, creating interactives presentation using “CANVA AI”; creating mind maps using “Mapify”; producing an interactive educational video using “ChatGPT & CANVA”. Overall, these interactive activities appear to have a significant effect on students’ performance in the post test. In addition, these GenAI tools helped the students to promote interactive learning, for example, ChatGPT helped them to generate new ideas and video scripts. Moreover, These GenAI tools provided the students with customized feedback and explanations on the course topics which facilitated their understanding and empowered them to engage more deeply to better support their learning needs (Baidoo‑Anu & Ansah, 2023). This finding is supported by (Yusuf et al., 2024; Gond et al., 2024) who investigated the use of GenAI with similar tools and activities (e.g., ChatGPT and Gemini) in students’ learning performance and engagement levels across various disciplines and concluded that both GenAI tools enhanced students’ understanding of complex concepts, sharing knowledge, and collaborative learning. Also, this result is in alignment with the results of (Nikolic et al., 2024; Ayobami, Rodney, Ayomide & Kayode, 2024; Sun & Zhou, 2024) who reported that GenAI significantly improved students’ academic achievement and provided many opportunities in enhancing teaching and learning. However, this finding contradicts the findings of Ironsi and Ironsi (2024) which showed that although GenAI like ChatGPT may assist students in providing ideas in writing lessons, it may not improve their overall performance in writing skills.
4.2. Students’ Motivation
Motivation plays an essential role in driving student learning and engagement. The findings of this study indicate that the use of GenAI tools in learning activities and teaching strategies used to the experimental group did not have significant effect on students’ motivation to learn compared to the control group across the four different dimensions. These findings could be attributed to other external reasons that may have influenced the diploma students’ motivation levels to learn such as their personal interests in the subjects, which may assist them integrating technology as teachers in their classrooms, or the labs environments which supported with emerging technologies such as smart screens. These results also could be justified by the fact that both sections (the experimental and control groups) may have already had similar levels of motivation to learn educational and information technology before the study began. This finding seems to be contradicted to several studies related to the effects of GenAI models on teaching and learning (e.g., Cai et al., 2023; Song & Song, 2023; Huang & Mizumoto, 2024; Esfandiari & Ghamari, 2025) who also concluded that GenAI tools such as ChatGPT positively affect students’ motivation to learn, although sustained interest was related to prior human interactions (Shata & Hartley, 2024).
5. Conclusion
This examined the effect of using GenAI tools (e.g., ChatGPT, Gemini, CANVA) on students’ performance and learning motivation in learning educational and information technology. The findings showed a significant positive effect on students’ performance in learning educational and information technology, as the students experimental group achieved higher post-test scores compared to the students in the control group. However, the results indicated no statistically significant differences in students’ motivation scores between the two groups, suggesting that other external factors (e.g., students’ personal interests and available resources) may have affected motivation levels. By integrating GenAI applications in teaching, teachers can enhance their teaching strategies, foster students engagement, design effective lesson plans, generate scenarios for educational videos, and adapt more effective to the evolving educational landscape.
6. Implications
As our findings showed statistically significant improvement in the scores of students’ performance in learning educational and information technology, this indicates that GenAI applications, such as ChatGPT, Canva AI, Gemini, Mapify AI and Curipod, can positively enhance the teaching and learning process. The utilization of these GenAI tools in learning activities assisted the students in understanding the course concepts. This indicates that GenAI-based learning activities and instructional strategies may successfully be integrated in instructional technology courses to enhance students’ understanding and comprehension. Despite the improvements in the students’ achievement, the findings showed no statistically significant differences in students’ motivation levels between the control and experimental groups. This implies that although GenAI applications can positively enhance the teaching and learning process, they may certainly not enhance intrinsic students’ motivation. Therefore, future research may focus on exploring how to design and develop learning environment supported by GenAI-based interventions that foster students’ motivation and engagement, probably by incorporating gamification strategies. Moreover, teachers may focus on using additional teaching strategies by using GenAI applications, such as adaptive learning strategies, AI-based personalized learning strategies and cooperative AI based learning activities, to better enhance students’ engagement and satisfaction.
7. Limitations
In this study, we had two primary limitations. First, we depended on small size of participants which may influence the generalizability of the results. Therefore, future research could expand the sample size to enhance the reliability generalizability of the findings. Second, this study was carried and implemented over a short period of time (8 weeks). Therefore, future research should increase the period of the intervention and implementing it within enough time, so the participants of the study might have enough time to develop significant changes on the study variable. Third, this study entirely used quantitative data to examine the relationship between the study’s factors. Future research should include qualitative data from the participants to gain more complete understanding into their prospects and primary sources motivation of learning.
Acknowledgments
Special thanks to all the Diploma students who were participated in this study and responded to the questionnaires.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
References
Afjar A., Musri, C., & Syukri, M. (2020). Attention, relevance, confidence, satisfaction (ARCS) model on students’ motivation and learning outcomes in learning physics, Journal of Physics Conference Series, 1460, 012119. https://doi.org/10.1088/1742-6596/1460/1/012119
Ali, O., Murray, P.A., Momin, M., Dwivedi, Y.K., & Malik, T. (2024). The effects of artificial intelligence applications in educational settings: Challenges and strategies. Technological Forecasting and Social Change, 199, 123076. https://doi.org/10.1016/j.techfore.2023.123076
Alshamy, A., Al-Harthi, A.S.A., & Abdullah, S. (2025). Perceptions of Generative AI Tools in Higher Education: Insights from Students and Academics at Sultan Qaboos University. Education Sciences, 15(4), 501. https://doi.org/10.3390/educsci15040501
Ayobami, O.A., Rodney, E.O., Ayomide, S.O., & Kayode, E.O., (2024). AI-driven adaptive learning platforms: Enhancing educational outcomes for students with special needs through user-centric, tailored digital tools. World Journal of Advanced Research and Reviews, 22(03), 2253-2265. https://doi.org/10.30574/wjarr.2024.22.3.0843
Baidoo-Anu, D., & Ansah, L.O. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52-62. https://doi.org/10.61969/jai.1337500
Batista, J., Mesquita, A., & Carnaz, G. (2024). Generative AI and higher education: Trends, challenges, and future directions from a systematic literature review. Information, 15(11), 676. https://doi.org/10.3390/info15110676
Cai, Q., Lin, Y., & Yu, Z. (2023). Factors influencing learner attitudes towards ChatGPT-assisted language learning in higher education. International Journal of Human-Computer Interaction, 40(22), 7112-7126. https://doi.org/10.1080/10447318.2023.2261725
Collins, C., Dennehy, D., Conboy, K., & Mikalef, P. (2021). Artificial intelligence in information systems research: A systematic literature review and research agenda. International Journal of Information Management, 60, 102383. https://doi.org/10.1016/j.ijinfomgt.2021.102383
Creswell, J.W., & Creswell, J.D. (2017). Research design: Qualitative, quantitative, and mixed methods approach. Sage Publications.
Erduran, S. (2023). AI is transforming how science is done. Science education must reflect this change. Science, 382, 6677. https://doi.org/10.1126/science.adm9788
Esfandiari, S., & Ghamari, P. (2025). A Systematic Review of the Effects of Generative Artificial Intelligence Models (ChatGPT& Gemini) on English Language Teaching (ELT): Opportunities & Challenges. Journal of Foreign Language Research, 14(4), 611-641. https://doi.org/10.22059/jflr.2025.386926.1173
Estrada, B.O., & Fabián, A.D.H. (2024). Generación de rúbricas con herramientas de inteligencia artificial para la evaluación de aprendizajes en educación superior. DIDAC, 84. https://doi.org/10.48102/didac.2024..84_JUL-DIC.211
Fetzer, J.H. (1990). What is Artificial Intelligence? In Artificial intelligence: Its scope and limits. Studies in cognitive systems (3-27). Springer. https://doi.org/10.1007/978-94-009-1900-6_1
Gasaymeh, A.M.M., Beirat, M.A., & Abu-Qbeita, A.A.A. (2024). University Students’ Insights of Generative Artificial Intelligence (AI) Writing Tools. Education Sciences, 14(10), 1062. https://doi.org/10.3390/educsci14101062
Gond, S.K., Upadhyay, A., Mishra, S.M. & Bhardwaj, S. (2024). Leveraging Artificial Intelligence in Education: Assessing the Effectiveness of ChatGPT and Gemini AI Tools Among Youth. Educational Administration: Theory and Practice, 30(1), 3086-3096. https://doi.org/10.53555/kuey.v30i1.6994
Hsu, Y.C., & Ching, Y.H. (2023). Generative artificial intelligence in education, Part two: International perspectives. TechTrends, 67(6), 885-890. https://doi.org/10.1007/s11528-023-00913-2
Huang, J., & Mizumoto, A. (2024). Examining the effect of generative AI on students’ motivation and writing self-efficacy. Digital Applied Linguistics, 1, 102324. https://doi.org/10.29140/dal.v1.102324
Imran, M., & Almusharraf, N. (2024). Google Gemini as a next generation AI educational tool: a review of emerging educational technology. Smart Learning Environments, 11(22), 1-8. https://doi.org/10.1186/s40561-024-00310-z
Ironsi, C.S., & Ironsi, S.S. (2024). Experimental evidence for the efficacy of generative AI in improving students’ writing skills. Quality Assurance in Education, 33(2), 237-252.
https://doi.org/10.1108/QAE-04-2024-0065
Kehoe, F. (2023). Leveraging generative AI tools for enhanced lesson planning in initial teacher education at post primary. Irish Journal of Technology Enhanced Learning, 7(2), 172-182. https://doi.org/10.22554/ijtel.v7i2.124
Keller, J.M., & Kopp, T.W. (1987). An application of the ARCS model of motivational design. In Instructional Theories in Action (289-319). Hillsdale, NJ: Lawrence Erlbaum Associates.
Loorbach, N., Peters, O., Karreman, J., & Steehouder, M. (2015). Validation of the Instructional Materials Motivation Survey (IMMS) in a self-directed instructional setting aimed at working with technology. British Journal of Educational Technology, 46, 204-218. https://doi.org/10.1111/bjet.12138
Luckin, R., & Cukurova, M. (2019). Designing educational technologies in the age of ai: A learning sciences driven approach. British Journal of Educational Technology, 50(6), 2824-2838. https://doi.org/10.1111/bjet.12861
McDonald, J., Shainock, J., O’Flaherty, S., & Codio, S. (2024). Introduction To Artificial Intelligence (AI) Technology: Guide for Travel & Tourism Leaders. World Travel & Tourism Council. Suidia Arabia.
Mishra, K.S., & Bhardwaj, M.S. (2024). Leveraging artificial intelligence in education: Assessing the effectiveness of ChatGPT and Gemini AI tools among youth. Educational Administration: Theory and Practice, 30(1), 3086-3096. https://doi.org/10.53555/kuey.v30i1.6994
Nikolic, S., Sandison, C., Haque, R., Daniel, S., Grundy, S., Belkina, M. et al. (2024). ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering. Australasian journal of engineering education, 29(2), 126-153. https://doi.org/10.1080/22054952.2024.2372154
Nikolopoulou, K. (2024). Generative Artificial Intelligence in Higher Education: ExploringWays of Harnessing Pedagogical Practices with the Assistance of ChatGPT. International Journal of Changes in Education, 1(2), 103-111. https://doi.org/10.47852/bonviewIJCE42022489
Owan, V.J., Abang, K.B., Idika, D.O., Etta, E.O., & Bassey, A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. EURASIA Journal of Mathematics, Science and Technology Education, 19(8), 1-15. https://doi.org/10.29333/ejmste/13428
Palinkas, L.A., Horwitz, S.M., Green, C.A., Wisdom, J.P., Duan, N., & Hoagwood, K. (2015). Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533-544. https://doi.org/10.1007/s10488-013-0528-y
Ramnarain, U., Ogegbo, A.A., Penn, M., Ojetunde, S., & Mdlalose, N. (2024). Pre-service science teachers’ intention to use generative artificial intelligence in inquiry-based teaching. Journal of Science Education and Technology, 1(14). https://doi.org/10.1007/s10956-024-10159-z
Schwab, K. (2016). The Fourth Industrial Revolution. Crown Business.
Shata, A., & Hartley, K. (2024). Artificial intelligence and communication technologies in academia: faculty perceptions and the adoption of generative AI. International Journal of Educational Technology in Higher Education, 22, 14. https://doi.org/10.1186/s41239-025-00511-7
Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology, 14, 1260843. https://doi.org/10.3389/fpsyg.2023.1260843
Sun, L., & Zhou1, L. (2024). Does Generative Artificial Intelligence Improve the Academic Achievement of College Students? A Meta-Analysis. Journal of Educational Computing Research, 62(7), 1896-1933. https://doi.org/10.1177/07356331241277937
Yusuf, B., Nordin, N., & Annamalai, S. (2024). ChatGPT in higher education: Revolution or risk? Computers and Education: Artificial Intelligence, 6, 100197. https://doi.org/10.1016/j.caeai.2023.100197
Annex 1. IMMQ Questionnaire
|
Please Put a check mark (√) around the option that represents the extent to which the statement applies to you, using the following scale: (1) Strongly disagree, (2) Disagree, (3) Neutral, (4) Agree, (5) Strongly agree |
1 |
2 |
3 |
4 |
5 |
|
Attention |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Relevance |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Confidence |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Satisfaction |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This work is licensed under a Creative Commons Attribution 4.0 International License
Journal of Technology and Science Education, 2011-2026
Online ISSN: 2013-6374; Print ISSN: 2014-5349; DL: B-2000-2012
Publisher: OmniaScience