Journal of Technology and Science Education

• Home
• About
• Login
• Archives
• Submissions
• Publication Fee
• Indexing & Statistics

Home > Vol 14, No 3 (2024) > Palacios Ortega

METHODOLOGICAL DESIGN IN THE USE OF VIRTUAL SIMULATIONS IN CHEMISTRY: A SYSTEMATIC REVIEW

Universidad Internacional de La Rioja (Spain)

Received July 2023

Accepted January 2024

Abstract

Virtual simulations are a very useful educational resource to improve the teaching of chemistry. Their use makes it possible to facilitate the comprehension of concepts, promotes the development of scientific competences and even improves student attitudes toward chemistry. However, it is important to point out that a simulation by itself is not enough to improve student learning. Methodological design is a crucial aspect in order for its classroom use to be significant. Simulations must form part of an instructional sequence that promotes said learning, and thus the role of the instructor is key. This work conducts a systematic review with the aim of analyzing how to apply the use of simulations in the teaching of chemistry in secondary schools, delving deeper into the way in which simulations are used from a methodological perspective, in order to improve the learning processes and results. In this sense, it was found that there is an improvement in learning on both the level of content comprehension and in scientific competences. Guided research is also identified as the most effective methodology for the application of simulations in the classroom. With a look to the future, it is suggested that there is a need to conduct research on the continued use of simulations in the classroom, as well as their characteristics and the instructional methods used.

 

Keywords – Educational technology, Chemistry teaching, Teaching method, Secondary school teaching.

To cite this article:

Palacios, A., Pascual, V., & Moreno-Mediavilla, D. (2024). Methodological design in the use of virtual simulations in chemistry: a systematic review. Journal of Technology and Science Education, 14(3), 701-719. https://doi.org/10.3926/jotse.2357

 

----------

1. 1. 1. Introduction

For years now, both reports by the PISA (Programme for International Student Assessment) (OECD, 2017) and the analyses by the National Research Council (National Research Council, 2012) have proposed the need to introduce important changes in science education, in light of the lack of motivation and poor comprehension of scientific phenomena in secondary education. With these focuses in mind, science education is not only centered on content knowledge, but also on the learning of scientific processes (Bybee, 2011; Osborne, 2014). The growing accessibility and development of new technologies in recent decades has multiplied the options to facilitate and enrich learning focused on scientific practice (Oliveira, Feyzi-Behnagh, Ni, Mohsinah, Burgess & Guo, 2019). Furthermore, the use of these instructional tools has garnered special interest as the result of the Covid-19 pandemic, in which they have been especially helpful in complementing distance education or blended learning scenarios (Callaghan & Collins, 2021). One of the technological resources that provides these possibilities is virtual simulations, understood as an interactive application that shows a virtual representation of a phenomenon or system (De Jong & Van Joolingen, 1998). Thus, the characteristic that differentiates these tools from other similar ones, such as animation, is precisely their interactive nature. The simulations on which this study is focused make it possible for users to observe and interact with a phenomenon, modifying some of the variables that define it. For the study of Chemistry, in the same way as for classroom-based laboratories (Bretz, 2019), virtual laboratories allow students to work in environments similar to those of basic or applied scientific research work (Lynch & Ghergulescu, 2017), and also delve deeper into concepts and applications related to their specific learning (Fiad & Galarza, 2015; Ortiz, Álvarez & Sánchez, 2017). In recent decades, the number of accessible simulations has increased exponentially. There are multiple open access simulations that are very versatile and cover most of the scientific contents typically addressed in secondary education (Alkhaldi, Pranata & Athauda, 2016; D’Angelo, Rutstein, Harris, Bernard, Borokhovski & Haertel, 2014). Simulations provide a great deal of flexibility with regard to where and when we use them, they are easy to use, they can be used simultaneously by a large number of students and the phenomenon can be viewed and interacted with as many times as we wish (Correia, Koehler, Thompson & Phye, 2019). These characteristics make them useful for presenting science as a process and improving the contextualized comprehension of scientific concepts and phenomena (Samur & Evans, 2011). Students are able to observe and manipulate objects, variables and phenomena, and even view the changes that occur through different representations (Blake & Scanlon, 2007). This helps them develop scientific research competences, such as observe, ask questions, develop hypotheses, verify results and draw conclusions (De Jong & Van Joolingen, 1998; Fan & Geelan, 2013; Perkins, Moore, Podolefsky, Lancaster & Denison, 2012). It is even possible that the students might come to understand the conceptual model on which a certain simulation is based, which could help identify and correct their mistaken preconceived notions (Ronen & Eliahu, 2000; Trudel & Metioui, 2011).

Much of the research on simulations is focused on the benefits of their use as opposed to other tools, such as traditional instruction. These studies indicate that simulations are effective in improving the performance of science students (Rutten, Van Joolingen & Van Der Veen, 2012; Velasco & Buteler, 2017). Likewise, they indicate that virtual laboratories can even be more effective than real laboratories, depending on the concept being studied (Finkelstein, Adams, Keller, Kohl, Perkins, Podolefsky et al., 2005) and that the combination of virtual and real laboratories is more successful than the exclusive use of real laboratories alone (Zacharia, 2007; Zacharia, Olympiou & Papaevripidou, 2008). Besides these benefits, it is important to stress that the success of simulations in improving student learning depends in part on the design of the simulation itself. Research in this regard mainly indicates that learning improves with high levels of interactivity and realism, and that the simulations must have an intuitive design that encourages exploration (Adams, Reid, LeMaster, McKagan, Perkins, Dubson et al., 2008; Podolefsky, Perkins & Adams, 2010). Specifically, in the study of chemistry, they make it possible to boost the learning achievements, self-sufficiency and confidence of the students (Peechapol, 2021). They also allow chemistry students to work while connecting general chemistry concepts to more advanced ones (Karayilan, Vakil, Fowler, Becker & Cox, 2021).

The other crucial aspect that conditions the success of simulations is context. The research in this regard shows that a simulation by itself is not enough to improve student learning (Velasco & Buteler, 2017). The simulations must form part of an instructional sequence that encourages exploration, investigation, comprehension of the phenomenon and the development of critical thinking, and thus the role of the instructor is key. The students must have appropriate prior knowledge and a theoretical basis in order to be able to interpret phenomena, and the instructor must help students in the decoding and interpretation of the information and of the model that is represented by the simulation (Fuentes, López & Pozo, 2019; Rizvi & Nabi, 2021). The bibliography studying these aspects is not as revealing, as it is focused on the analysis of the learning results, without paying much attention to the context. Therefore, there is still a great deal of uncertainty regarding the nature of the instructional sequences of the simulations, how the teachers should support this learning and at what times in the teaching process they should be used (Rutten et al., 2012; Trudel & Metioui, 2011; Velasco & Buteler, 2017).

This systematic review analyzes the research conducted between the years 2011 and 2021 on the use of virtual simulations in chemistry in order to determine whether there is any improvement in the learning of the contents and in the fostering of scientific competences, as well as which methodologies and contexts of application are most suitable for the effective use of the same in the context of secondary education.

2. Methodological Design

2.1. Previous Systematic Reviews

In order to carry out the review work, a search was initially proposed that would make it possible to know previous systematic review works.

The search was carried out using the SCOPUS and WOS (Web of Science) databases for systematic reviews published between 2011 and 2021. A total of 15 initial systematic review articles were obtained; following the elimination of topics not specifically focused on chemistry or virtual simulations, 5 works remained.

The works found emphasize the importance of integrating the simulations, but without specifying how they are applied in the classroom, how the instructor’s work unfolds (Alzahrani, 2020) or what type of materials are designed to integrate these simulators in the classroom (Hamilton, McKechnie, Edgerton & Wilson, 2020). In this sense, D’Angelo et al. (2014) highlight the need to analyze the qualitative studies in order to find out how the different simulators are applied. The work by Chan, Van Gerven, Dubois and Bernaerts (2021) focuses on the usefulness of virtual laboratories in replacing or complementing on-site laboratory work. Their work stresses that virtual laboratories can be more effective than conventional teaching methods; however, they recommend combining virtual laboratories and traditional methods. Furthermore, they emphasize the importance for future studies to be oriented towards learning results, pointing out the lack of consideration for instructional design in virtual chemistry laboratories. Knowing the methodologies and strategies that have been developed is necessary in order to be able to make decisions about their application. Moreover, the definition of the teacher’s role becomes a fundamental aspect (Rutten et al., 2012). Based on these premises, the present systematic review intends to provide answers to some of the questions that have been left unanswered in the previous reviews.

2.2. Research Questions

The aim of this work is to analyze how virtual simulations are used and what benefits they provide in chemistry classrooms for secondary education students. To achieve this, the following research questions are proposed:

RQ1: Is there any improvement in the learning of chemistry contents over the short/long term with the use of virtual simulations in secondary education? Why does this occur?

RQ2: Within the framework of formal secondary education, what methodologies guide the learning by means of virtual chemistry simulations?

RQ3: In the secondary school classroom, must computational chemistry simulations be used in a very guided manner or not?

RQ4: Are there any advantages to the use of simulations through cooperative learning as opposed to individual learning with secondary school chemistry students?

RQ5: What competences do secondary school students acquire with the use of computational simulations in chemistry?

2.3. Databases and the Search Strategy

The SCOPUS and WOS databases were used for the systematic review. The entire process, from the general primary review until the final selection of the articles, was carried out according to PRISMA 2020 methodology, as indicated in Figure 1 (Page, Mckenzie, Bossuyt, Boutron, Hoffmann, Mulrow et al., 2021).

First, the key words were defined to delimit the definition of simulation and focus it on the field of work defined by the research questions. The following search sequence was used:

“virtual lab*” OR “simulation” OR “interactive learning environment” AND educat* OR learn* OR train* OR teach* AND “secondary education” OR “post-secondary education” OR “K-12” OR “K-16” OR “elementary secondary education” OR “postsecondary education” OR “secondary school*” OR “tertiary education” OR “tertiary school*” OR “middle school*” OR “high school*” AND “chemist*”

For the search sequence, the most common words were included that are used to define interactive virtual simulations, such as: virtual lab, simulation and interactive learning environment. Some of these terms are also used to refer to other resources, and thus the section on the exclusion criteria specifies those resources that are not the subject of this research.

The search was used to obtain articles and reviews from peer-reviewed scientific journals, written in English and published between 2011 and 2021, which include as a main topic simulations applied to the teaching of chemistry to students from 11 to 18 years of age (6th to 12th grade). The last search was conducted on July 20, 2021.

The aggregate results of the initial search included 685 articles. The lists obtained in the 2 databases consulted were compared to eliminate any possible duplicates that were found, and thus the selection ultimately consisted of 521 articles.

Figure 1. PRISMA flow chart

2.3.1. Inclusion Criteria:

• •Peer-reviewed articles.  

• •Written in English. 

• •Describe the use of interactive simulations. 

• •Include quasi-experimental statistical analysis. 

• •Include qualitative analysis on the benefits of using the simulation. 

2.3.2. Exclusion Criteria:

• •Articles based on simulation software or design from the perspective of computer programming. 

• •The curricular contents are not focused on chemistry. 

• •They are not focused on secondary and/or baccalaureate students. 

• •Articles that are focused on simulations that do not correspond to the definition of interactivity (e.g., animations) 

• •Focused on remote laboratories, e-books, web-based learning environments, games, virtual reality, augmented reality and computer-assisted laboratories. 

• •Instructional proposals are excluded that are not related to the specific analysis of simulations. 

• •Grey material (conference communications, theses, etc.) 

2.4. Encoding

To classify the articles in order to answer the research questions, a coding system was established, using as a reference that proposed by Wang and Tahir (2020) (Table 1).

Code	Description	Research Questions
L	Studies that include in their results an analysis of the effect that the use of simulations has on the learning of the students who participate.	RQ1
C	Studies that analyze the development of student competences when using simulations.	RQ5
SP	Studies that describe the student’s appreciation of the use of simulations. These may be quantitative or qualitative.	RQ4
TP	Studies that describe the instructor’s appreciation of the use of simulations. These may be quantitative or qualitative.	RQ3, RQ4
QL	Studies with qualitative analysis.	RQ2, RQ3, RQ4
QT	Studies with quantitative analysis.	RQ1, RQ2, RQ3, RQ4, RQ5

Table 1. Coding of the characteristics of the articles selected and their relation to the research questions

3. Results

First a general analysis of the 31 articles that make up the review is considered. Figure 2 shows the cumulative publications on the topic in the years of study. It is observed that the growth in the number of publications on the use of virtual chemistry simulations in secondary education shows an upward trend, which is indicative of the ongoing interest in the topic.

Figure 2. Cumulative publications selected from 2011-2021

Table 2 presents the results of the synthesis of the studies accepted, taking into account the coding shown in Table 1.

Reference	Simulation content	L	C	SP	TP	QL	QT
(Amin & Ikhsan, 2021)	Chemical equilibrium		C				QT
(Chang & Linn, 2013)	Chemical thermal dynamics	L				QL	QT
(Chen, Chang, Lai & Tsai, 2014)	Boyle’s Law	L	C	SP		QL	QT
(Chien, Tsai, Chen, Chang & Chen, 2015)	Boyle’s Law	L					QT
(Correia et al., 2019)	Behavior of gases at a sub-microscopic level	L		SP		QL	QT
(Davenport, Rafferty & Yaron, 2018)	Solutions (the mole)	L					QT
(Donnelly, O’Reilly & McGarr, 2013)	Unspecified		C		TP	QL	QT
(Gambari, Gbodi, Olakanmi & Abalaka, 2016)	Periodicity and chemical equations	L		SP		QL	QT
(Gambari, Kawu & Falode, 2018)	Identification of cations	L					QT
(Hale-Hanes, 2015)	Acid-base reactivity	L		SP		QL	QT
(Homer & Plass, 2014)	Kinetic molecular theory and ideal gas law	L					QT
(Karlsson, Ivarsson & Lindstrom, 2013)	Gas solubility	L		SP	TP	QL
(Lamb & Annetta, 2013)	Mole, chemical equations and stoichiometry	L		SP			QT
(Levy, 2013)	States of aggregation and phase change		C			QL	QT
(Olakanmi, 2015)	Chemical reactions	L				QL	QT
(Osman & Lee, 2013)	Electrochemistry	L					QT
(Papadimitropoulos, Dalacosta & Pavlatou, 2021)	Acids and bases (properties, Arrhenius theory and pH)	L					QT
(Plass, Milne, Homer,  Schwartz, Hayward, Jordan et al., 2012)	Kinetic molecular theory	L	C				QT
(Pratidhina, Pujianto & Sumardi, 2019)	Gas laws	L					QT
(Ryoo, Bedell & Swearingen, 2018)	Properties of matter and chemical reactions	L					QT
(Smetana & Bell, 2014)	Atomic structure	L		SP	TP	QL	QT
(Stieff, 2019)	Nature of matter, reactivity and chemical equilibrium	L					QT
(Tatli & Ayas, 2012)	Chemical changes	L		SP		QL	QT
(Tatli & Ayas, 2013)	Chemical changes	L					QT
(Udo & Etiubon, 2011)	Chemical combination	L					QT
(Ullah, Ali & Rahman, 2016)	Pure substances and mixtures	L		SP			QT
(Vandenplas, Herrington, Shrode & Sweeder, 2021)	Force and energy, in intermolecular bonds and attractions	L					QT
(Waight, Liu, Gregorius, Smith & Park, 2014)	General chemistry	L		SP	TP	QL
(Waight & Gillmeister, 2014)	General chemistry	L		SP	TP	QL
(Wen, Liu, Chang, Chang, Chang, Chiang et al., 2020)	Chemistry of fluids		C				QT
(Zohar & Levy, 2019)	Chemical bonds	L		SP		QL	QT

Table 2. Results of the analysis of the 31 articles identified in the systematic review.
L: Effect on learning; C: Effect on the development of competences; SP: Students’ perception of the use of simulations; TP: Teachers’ perception of the use of simulations; QL: Studies with qualitative analysis; QT: Studies with quantitative analysis

By way of general results (Figure 3), it was found that 87% of the articles include an analysis of the effect that the use of simulations has on the learning of chemical concepts or models. 19% of the works set out to specifically analyze the development of student competences. With regard to the analysis of the teachers’ or students’ perception of the use of virtual laboratories, 39% take into account the opinion of the students, while that of the teachers is only analyzed in 16% of the studies. Most of the works analyzed include the quantitative analysis of the results (90%). On the other hand, there are fewer qualitative analyses among the works analyzed (44%). They all include the design of instructional proposals, specifying in greater or lesser detail the instructional methodologies or strategies, as well as a description of the work performed by the students. 91% of the works analyzed are focused on the Secondary Education level, from grades 9-12.

Figure 3. Percentage of publications with each assigned attribute

RQ1: Is there any improvement in the learning of chemistry contents over the short/long term with the use of virtual simulations in secondary education? Why does this occur?

This section presents the studies related to the learning of scientific concepts or models through virtual simulations, and how they affect learning as compared to other approaches and tools.

Of the 31 works studied, 27 are dedicated to analyzing the learning outcomes after using virtual simulations in the classroom. Twenty-three of these works showed a significant improvement in the comprehension of concepts; three works (Homer & Plass, 2014; Waight et al., 2014; Waight & Gillmeister, 2014) conducted a qualitative analysis, and thus they were not taken into account as either positive or negative results. Only the work by Karlsson et al. (2013) found no significant improvement in conceptual comprehension, which was associated with the lack of prior knowledge by the participating students.

Many different characteristics of the virtual laboratories were highlighted as being responsible for learning. In Plass et al. (2012), it was observed that the use of simulations improves the comprehension of chemistry, as well as the students’ ability to transfer their knowledge to new contexts. They related this to the fact that the simulation design is focused on showing the relationships between the different levels of representation: the observable, through a contextualized narrative; the explanatory, through an interactive visualization on the molecular level; and the symbolic, through graphs based on the manipulation of the model. This argument coincides with that proposed by Lamb and Annetta (2013) and Osman and Lee (2013). The positive effect on the transfer of knowledge is also observed in Homer and Plass (2014). Smetana and Bell (2014) associate the improvements with two factors: 1) the simulations have the potential to make learning interactive, authentic and significant; 2) the role of the instructor, fostering cooperation and student-focused learning. Other works (Waight et al., 2014; Waight & Gillmeister, 2014) suggest that fragmented knowledge and mistaken ideas by instructors act as obstacles, as does the use of different levels of representation and graphics on the same screen (Waight & Gillmeister, 2014).

On the other hand, some works (Correia et al., 2019; Hale-Hanes, 2015; Levy, 2013; Ryoo et al., 2018; Zohar & Levy, 2019) suggest that the use of interactive simulations that allow for a dynamic visualization and representation make the comprehension of abstract molecular concepts and model building more accessible. Furthermore, they propose the importance of starting with the prior knowledge of the students for both the design of the simulations and the guidance process by the instructor (Zohar & Levy, 2019). The use of simulations, in turn, facilitates the identification of erroneous concepts and problems in the representation of ideas by the students (Olakanmi, 2015).

The possibilities for interaction offered by simulations (Homer & Plass, 2014), and the possibility of repeating the experiments as many times as necessary, in a short time and at no additional cost (Amin & Ikhsan, 2021), are other characteristics identified as promoting learning. Udo and Etiubon (2011) recommend the use of virtual laboratories due to their powerful facilitating effect, since they make it possible to work in a dynamic interactive environment that facilitates the reformulation of knowledge and mastery of concepts.

Of the works finding positive learning results, some compare traditional instruction to the use of simulations (Gambari et al., 2016; Lamb &Annetta, 2013; Udo & Etiubon, 2011; Ullah et al., 2016; Vandenplas et al., 2021). For example, Lamb and Annetta (2013) found that the use of online simulations from a constructivist teaching perspective increases the comprehension of chemistry as compared to conventional teaching methods. Gambari et al. (2016) and Udo and Etiubon (2011) showed that the use of virtual simulations is significantly better than traditional instruction and just as effective as teaching employing discovery instruction.

Other works compare the improvement in comprehension when we use virtual simulations as opposed to real laboratories (Tatli & Ayas, 2012, 2013). Tatli and Ayas (2013) observed that virtual simulations are at least as effective in improving learning about chemical changes and laboratory material recognition as real laboratories. This same result was obtained in the work by Tatli and Ayas (2012), in which they suggested that virtual simulations help instructors in the design of constructivist learning environments that place students in an active, central position in their own learning process.

Hale-Hanes (2015) proposes an improved understanding and the construction of mental models with the combined use of virtual and real laboratories, as the virtual format allows for the use of an inquiry-based methodology, while eliminating the danger of a real laboratory and the need for previous knowledge of the concepts being studied. Ullah et al. (2016) propose the use of these virtual laboratories as a step prior to practice in a real laboratory, observing far fewer errors.

On the other hand, works have been found that also compare whether there are any differences in the improved learning with regard to the use of computer assisted laboratories. Chen et al. (2014) and Chien et al. (2015) conclude that virtual manipulation is as effective as physical manipulation with computer‑assisted laboratories. Papadimitropoulos et al. (2021) observe that both the use of virtual laboratories and the use of computer-assisted laboratories have the same effect on improving learning.

The vast majority of the articles commented on so far focus on one-off interventions, which some authors point to as a difficulty when it comes to extrapolating results (Stieff, 2019). Furthermore, the improvements were more evident in simple concepts than in complex ones, which suggests the need for a longer-lasting intervention. In this sense, Ryoo et al. (2018) showed evidence that using interactive presentations to explain unobservable molecular phenomena has the potential to help students develop knowledge that persists at least three month after learning. Stieff (2019) obtained significant improvements in the learning of basic chemistry concepts over the long term by developing three course topics through investigation using virtual simulations. Lamb and Annetta (2013) observed sustained and continuous growth in the learning of students throughout the course of the school year by using virtual simulations.

RQ2: Within the framework of formal secondary education, what methodologies guide the learning by means of virtual chemistry simulations?

Learning through simulations can be implemented through different teaching methods and with differing levels of guidance from the instructor or the simulation itself. These different approaches can prove key to obtaining improvements in learning and the acquisition of competences.

Figure 4 shows a classification of the works analyzed according to the type of instruction concerned. A clear trend is observed towards constructivist-based methodologies, such as open inquiry (9.4%), guided inquiry learning (also referred to as discovery learning) (40.6%), POE (Prediction‑Observation‑Explanation) strategies (6.3%) and Problem-Based Learning (PBL) (3.1%), as opposed to step-by-step guided instruction (25%). Furthermore, it is also interesting to observe that all the works which measure the development of competences use inquiry-type instruction, either open or guided, finding an improvement in reasoning competences (Levy, 2013), graphic competences (Plass et al., 2012), inquiry competences (Wen et al., 2020) and higher level thinking competences (Amin & IIkhsan, 2021).

Figure 4. Relationship between the type of learning and the teaching method applied

Below are the details of those works that support the use of a particular teaching method according to the learning objective. Homer and Plass (2014) compared the impact on learning and knowledge transfer when a virtual simulation is used with guided investigation and when it is used with step-by-step guided instruction. It is observed that with simple simulation, students in both groups showed improved comprehension, but only those who used guided inquiry improve in terms of knowledge transfer. Chang and Linn (2013) add that the improvement increases when the students tackle experiments that involve analysis to determine what is correct and incorrect, and searching for possible solutions. This critical focus favors constructive learning and improves the connection between the microscopic level and the observable phenomenon.

On the other hand, the work by Donelly et al. (2013) attempted to determine the keys that could favor guided investigation by means of simulations. According to Donelly (2012) and Amin and Ikhsan (2021), this method facilitates active learning, focused on problem solving, experimental design and decision-making, which favors the development of scientific inquiry. Along the same lines, Tatli and Ayas (2012) and Correia, et al. (2019) found that virtual simulations facilitate the possibility of constructivist student-centered learning through POE experiences.

The use of open inquiry is possible thanks to the use of virtual simulation, since it allows students to experiment freely without any risk, promotes critical thinking skills and laboratory design skills (Hale‑Hanes, 2015).

Certain barriers are also detected in terms of applying inquiry methods with little guidance. Donelly et al. (2013) observed difficulties in students when it came to decision-making, since the instruction is normally focused on obtaining results and not on their discussion. Karlsson et al. (2013) describe serious problems students have in connecting the concepts with the simulation. For this reason, they recommend giving additional resources to the simulations that allow them to use their own prior experiences and to establish connections with related phenomena. Karlsson et al. (2013) and Hale-Hanes et al. (2015) coincide on the positive effect added by the use of cooperative work in the discussion.

With regard to when to use virtual simulations, Davenport et al. (2018) show that learning significantly improves both before the explanation of a topic and during or at the end of it, but where this most occurs is when they are used to end an instructional sequence.

RQ3: In the secondary school classroom, must computational chemistry simulations be used in a very guided manner or not?

Several works are concerned with determining whether any mode of instruction joined with a virtual simulation is appropriate for any type of students. The work by Ryoo et al. (2018) shows that the use of virtual simulations via guided inquiry boosts the comprehension of scientific phenomena when students do not master the language. On the other hand, Plass et al. (2012) found that the simulation used via a guided inquiry procedure is appropriate for students in both rural and urban settings, and even for students with poor academic performance. Along the same lines, Wen et al. (2020) found that by putting into practice a virtual simulation with guided inquiry, students improved in their development of scientific inquiry competences.

Finally, Homer and Plass (2014) go further in their work, attempting to find out whether different modes of instruction (guided inquiry vs. step-by-step guided instruction) generate different learning outcomes according to the students’ skills (thinking skills, planning, control, memory, etc.) and the complexity of the content being dealt with in the virtual simulation. The authors observe that with the simplest simulation, all the students improved in terms of comprehension. However, when complex simulations are used, they found that the guided inquiry approach is appropriate for students with better thinking skills, while the step-by-step guided instruction approach is better suited to those with poorer thinking skills.

RQ4: Are there any advantages to the use of simulations through cooperative learning as opposed to individual learning with secondary school chemistry students?

In the works analyzed, the virtual simulations were used primarily through individual work by the student (Figure 5). Of the 24 proposals in which individual work was carried out, Ullah et al. (2016) suggest that although the work is done individually, cooperation is allowed among the students via chat. In other works (Amin & Ikhsan, 2021; Hale-Hanes, 2015; Papadimitropoulos et al., 2021), a group or pair discussion phase is suggested after the performance of the individual experiment with the virtual simulation.

In these works, the importance of cooperative work was shown in order to pool the work performed and draw conclusions, and thereby developing inquiry skills. The importance of cooperation is also stressed, considering it as a crucial part of pair work in learning by discovery (Karlsson et al., 2013; Levy, 2013).

Figure 5. Types of groupings for work with virtual simulations

The comparison between different groupings in the same work is studied by Smetana and Bell (2014), who compare the effectiveness of work with virtual simulations in pairs as opposed to large group work. In both cases, a clear improvement in learning was obtained, but they suggest that large group work can promote more classroom interactions, which is considered essential according to the perspective of social‑constructivist learning. In contrast to this, Davenport et al. (2018) observe that students obtain better results when they tackle work with simulations individually, as opposed to when they work in pairs.

In terms of how to form the groups, specifically, whether to seek homogeneity or heterogeneity in them, most of the works fail to provide any information in this regard. Only Gambari et al. (2018) evaluate the learning, finding better results for homogeneous groups.

RQ5: What competences do secondary school students acquire with the use of computational simulations in chemistry?

The acquisition of science-related competences is a current need within scientific education (National Research Council, 2012; OECD, 2017). In spite of this, of the works studied, only five (Amin & Ikhsan, 2021; Chen et al., 2014; Donnelly et al., 2013; Levy, 2013; Plass et al., 2012; Wen et al., 2020) analyze these competences and only one of them (Amin & Ikhsan, 2021) gives priority to this matter in the analysis. In all the works analyzed, an improvement in scientific competences is observed on behalf of students, although there are some nuances, as shown below.

Levy (2013) analyzes the development of the scientific competence applied to the atomic scale, highlighting a significant improvement in the competence of molecular reasoning following the use of guided research activities with virtual simulations. Plass et al. (2012) showed that graphic competences significantly improve thanks to the use of simulations that contain different levels of representation, i.e., that attempt to connect observable or macroscopic aspects with the molecular model through symbols and graphics.

With regard to scientific competences related to inquiry, Wen et al. (2020) found that the scientific literacy of the students improved and remained in place longer when using virtual simulations through guided inquiry as opposed to the use of a textbook. Performance on research tasks was also analyzed by Chen et al. (2014), finding that the inquiry competences improved with virtual simulations, although it was more evident when using computer-assisted laboratories. This may be related to how the students approach the task: in a real experiment, the students think before acting, while in the simulation, just the opposite occurs; they tend to act by trial and error (Chien et al., 2015). Amin and Ikhsan (2021) compared the use of real laboratories with the use of simulations in the acquisition of higher order thinking skills (analysis, evaluation and creation), observing better results with the use of simulations. This is explained based on the three levels of chemical representation (macroscopic, symbolic and submicroscopic), which favor the use of simulations, and due to the fact that by using simulations, students can make mistakes, start again and waste less time on this process, which promotes inquiry.

4. Discussion

The majority of the works analyzed show positive results in learning chemistry-related concepts (Lamb & Annetta, 2013; Plass et al., 2012; Smetana & Bell, 2014) and the development of competences related to scientific work (Chen et al., 2014; Donnelly et al., 2013; Levy, 2013).

The improvements in conceptual learning have been detected over both the short and long term (Lamb & Annetta, 2013; Ryoo et al., 2018; Stieff, 2019), observing significant learning that, in addition, does not decline with continued use, and thus the novelty effect can be ruled out as being behind this improved learning (Lamb & Annetta, 2013).

The learning results with simulations are at least just as positive (Tatli & Ayas, 2012, 2013) as when real experiments are used. This does not mean that the works suggest substituting one type of practice with the other, rather they show the real usefulness of simulations and propose the undeniable instructional advantages of implementing a combination of both laboratory types, real and simulated (Hale-Hanes, 2015; Ullah et al., 2016), in the proposal design. In this way, real experimentation, with intrinsic motivation and a hands-on approach to the material (Chen et al, 2014), is combined with the instructional advantages provided by the use of simulations, such as their dynamic, interactive and easy to use features (Udo & Etiubon, 2011); being able to experiment without the fear of making a mistake, as many times as you wish (Amin & Ikhsan, 2021); being able to see what happens on a macroscopic, microscopic and submicroscopic level with different types of representation (Lamb & Annetta, 2013; Plass et al., 2012); and being able to explore abstract concepts in a simpler manner (Correia et al., 2019; Hale-Hanes, 2015; Levy, 2013).

In this improvement in learning that comes with the use of virtual simulations, the use of a particular instructional method or another plays an important role. When the intent is to develop competences related to scientific work, virtual simulations must be accompanied by an instructional method that involves a process of inquiry (Donelly, 2013; Amin & Ikhsan, 2021; Wen et al., 2020). Accordingly, improvements are also seen in graphic competences (Plass et al., 2012), the molecular reasoning competence (Levy, 2013) and inquiry competences (Wen et al., 2020; Chen et al., 2014), as well as in higher order thinking skills (Amin & Ikhsan, 2021).

This inquiry method, in turn, can be guided to varying degrees and its selection must depend on the level of the students’ knowledge (Wen et al., 2020, Homer & Plass, 2014), their preconceptions (Correia et al., 2019; Zohar & Levy, 2019), the teacher’s skills (Homer & Plass, 2014) and the difficulty associated with the type of simulation used (Homer & Plass, 2014), which can be related to the complexity of the content being taught. Even though the use of simulations has been demonstrated to be useful for all types of students (Ryoo et al., 2018; Plass et al., 2012; Wen et al., 2020), in general, if the simulation is very complex, the students have little previous knowledge or have a low overall academic level, it is recommended to use simulations through guided inquiry methods, or even in direct step-by-step instructional methods in extreme cases (Wen et al., 2020; Homer & Plass, 2014).

It is important to keep in mind that the simulation must make an analysis of the situation possible on a microscopic, macroscopic and graphic level (Plass et al., 2012), and the instructional proposals for practice with simulations must be based on the student’s prior knowledge (Zohar & Levy, 2019). It should be pointed out that the works fail to show any significant improvement in the results, and the authors themselves indicate problems related to the prior knowledge of the students (Karlsson et al., 2013) in order to improve the conceptual learning, as well as the need to redesign the computational simulation used (Osman & Lee, 2013). With regard to the instructional material that accompanies the simulation, whether inserted in the simulation itself or developed by the instructor, it must be based on the prior knowledge of the students (Zohar & Levy, 2019), include questions that guide the learning (Ryoo, 2018) and offer feedback to the students (Correia et al., 2019). Finally, as something critical for the correct implementation of the instructional proposals that include virtual simulations, it has been suggested that emphasizing discussion among the students is key to improving the comprehension of complex concepts and the development of inquiry competences (Hale-Hanes, 2015; Karlson et al., 2013).

5. Limitations

The limitations of the study are determined by the two conditioning factors. On the one hand is the nature of the search, selection and filtering process, and on the other is the information collected from the analyzed articles.

The search was limited to articles and was conducted in two databases of recognized prestige, namely Scopus and Web of Science, which could mean the loss of interesting works published in conferences or indexed journals in other databases, such as ERIC. The analysis was focused on works targeting K-12 students, disregarding applications in primary and university education. The classroom implementation has very different characteristics when used in other educational stages, which makes them difficult to compare. Furthermore, the topic was limited to chemistry contents, given the importance this has on scientific training and the difficulty of using and managing chemistry laboratories in secondary education. This emphasizes the usefulness and applicability of the results of this systematic review, but in turn it limits its application to the field of chemistry.

The work has been focused on analyzing articles that included the use of simulations or virtual laboratories, but it excludes the analysis of those that used non-interactive animations, virtual reality, augmented reality or mixed reality. It would be interesting for future works to attempt to analyze the full panorama from the perspective of how they are implemented in the classroom. Another important aspect in the selection of articles was the exclusion of both those works that were focused solely on simulation design and those that did not provide information about the instructional method.

With regard to the information provided about the instructional method, this was not as detailed in all the articles. In some articles, this information is very brief and not very detailed, or does not even appear in the text, with a general explanation being given in different sections of the article. This greatly complicated the extraction and interpretation of the information. Finally, it is important to mention that in many cases, the innovation introduced between the control group and the experimental group is not just the use of the simulation, rather the instructional method is also modified, and thus the results must be understood as a whole, as the result of the instructional method and the use of the simulation.

6. Conclusions

This article describes a systematic review carried out on how to use virtual simulations and the benefits they can have in chemistry classrooms at the K-12 level, which included 31 studies. The aim of the article was to answer the following research questions:

The first research question was focused on analyzing whether there was an improvement in the conceptual learning of chemistry in the short and/or long term with the use of computational simulations; it included 28 studies. Only one of the works found no statistically significant improvements in the comprehension of chemical concepts over the short term. In all cases, significant improvements are observed when comparing virtual simulations and traditional instruction, and learning outcomes are at least as good as when real laboratories or computer-assisted laboratories are used. Long-term learning was studied in three works, observing that the use of interactive simulations helps students to develop significant learning that lasts over time, and this learning does not decline when simulations are used continuously in the classrooms.

The second research question was focused on analyzing the instructional methodologies that guided the learning through virtual simulations and their implication in the learning. There is a clear trend towards using constructivist-based methodologies in learning through simulations, highlighting the use of guided inquiry. Furthermore, it is concluded that there is an improvement in the learning outcomes when the simulations are used by means of guided instruction as opposed to step-by-step guided instruction, and this change is more acute when knowledge transfer is analyzed. The use of inquiry methods involving little guidance, while of great interest for learning, created difficulties for both the students and the instructors.

The third research question was focused on analyzing whether the virtual simulations are appropriate for all types of students and whether this is regardless of the instructional method used. Virtual simulations improve learning of both students in different contexts and students with learning difficulties, such as when English is not their native language. With regard to the educational level, students of all levels improve their learning with the use of simulations with guided inquiry. However, if the simulation has a high level of complexity, it might be necessary to use step-by-step guided instruction with low-level students.

The fourth research question focused on analyzing whether there were any advantages in using simulations via cooperative learning as opposed to individual learning. There are works that have obtained the opposite results regarding the use of simulations individually or cooperatively. However, a consensus is observed regarding the importance of including a pair or group discussion phase within the teaching methodology, regardless of whether the simulation has been carried out individually or not. This is especially important when developing inquiry-based learning.

The fifth research question attempted to analyze whether the acquisition of competences improves thanks to the use of computational simulations in chemistry. Only six works analyzed the acquisition of competences, and they all concluded that an improvement in the development of competences relevant for scientific work occurred after using virtual simulations via inquiry learning. Specifically, graphic competences improved, as did the molecular reasoning competence, inquiry competences and higher order thinking skills.

In short, this systematic review confirms observations made in reviews carried out in the first decade of the twenty first century and provides relevant information about the improvement in competence learning achieved through the use of simulations. It also sheds light on future research questions proposed in previous reviews (Rutten et al., 2012), such as whether there is improved learning in both the short and long term and an improvement in the acquisition of scientific competences. Likewise, the constructivist methodologies of guided inquiry and the promotion of discussion have been described as the main methods for the effective development of simulations, highlighting the importance of matching the complexity of the simulation with the methodology according to the type of student. In this sense, more future research is necessary, making specific comparisons with concrete changes between the control sample and the experimental sample, in order to draw clearer conclusions about the effects of different instructional modes, groupings and types of learning simulations, according to the type of content being taught or the type of students that are targeted. Furthermore, more studies would be interesting on the long-term use of simulations in the classroom, and how the characteristics of the simulation and the instruction method affect the acquisition of competences related to the scientific work.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

Work financed by UNIR RETO 2022-2024 projects.

References

Adams, W.K., Reid, S., LeMaster, R., McKagan, S., Perkins, K., Dubson, M. et al. (2008). A Study of Educational Simulations Part II – Interface Design. Journal of Interactive Learning Research, 19(4), 551-577. https://www.learntechlib.org/p/24364

Alkhaldi, T., Pranata, I., & Athauda, R.I. (2016). A review of contemporary virtual and remote laboratory implementations: Observations and findings. Journal of Computers in Education, 3(3), 329-351. https://doi.org/10.1007/s40692-016-0068-z

Alzahrani, N.M. (2020). Augmented reality: A systematic review of its benefits and challenges in e-learning contexts. Applied Sciences (Switzerland), 10(16). https://doi.org/10.3390/app10165660

Amin, D.I., & Ikhsan, J. (2021). Improving higher order thinking skills via semi second life. European Journal of Educational Research, 10(1), 261-274. https://doi.org/10.12973/eu-jer.10.1.261

Blake, C., & Scanlon, E. (2007). Reconsidering simulations in science education at a distance: Features of effective use. Journal of Computer Assisted Learning, 23(6), 491-502.
https://doi.org/10.1111/j.1365-2729.2007.00239.x

Bretz, S.L. (2019). Evidence for the Importance of Laboratory Courses. Journal of Chemical Education, 96(2), 193-195. https://doi.org/10.1021/acs.jchemed.8b00874

Bybee, R.W. (2011). Scientific and engineering practices in K-12 classrooms. The Science Teacher, 78(9), 34‑40.

Callaghan, D., & Collins, H. (2021). Adaptive action learning in an online community: facilitating a large cohort to deliver results. Action Learning: Research and Practice, 18(1), 20-37. https://doi.org/10.1080/14767333.2020.1854681

Chang, H., & Linn, M.C. (2013). Scaffolding learning from molecular visualizations. Journal of Research in Science Teaching, 50(7), 858-886. https://doi.org/10.1002/tea.21089

Chan, P., Van Gerven, T., Dubois, J.C., & Bernaerts, K. (2021). Virtual chemical laboratories: A systematic literature review of research, technologies and instructional design. Computers and Education Open, 2, https://doi.org/10.1016/j.caeo.2021.100053

Chen, S., Chang, W., Lai, C., & Tsai, C. (2014). A Comparison of Students’ Approaches to Inquiry, Conceptual Learning, and Attitudes in Simulation-Based and Microcomputer-Based Laboratories. Science Education, 98(5), 905-935. https://doi.org/10.1002/sce.21126

Chien, K., Tsai, C., Chen, H., Chang, W., & Chen, S. (2015). Learning differences and eye fixation patterns in virtual and physical science laboratories. Computers & Education, 82, 191-201. https://doi.org/10.1016/j.compedu.2014.11.023

Correia, A., Koehler, N., Thompson, A., & Phye, G. (2019). The application of PhET simulation to teach gas behavior on the submicroscopic level: secondary school students’ perceptions. Research in Science & Technological Education, 37(2), 193-217. https://doi.org/10.1080/02635143.2018.1487834

D’Angelo, C., Rutstein, D., Harris, C., Bernard, R., Borokhovski, E., & Haertel, G. (2014). Simulations for STEM learning: Systematic review and meta-analysis. SRI Education, 58. Available at: https://www.sri.com/wp-content/uploads/2021/12/simulations-for-stem-learning-brief.pdf

Davenport, J.L., Rafferty, A.N., & Yaron, D.J. (2018). Whether and How Authentic Contexts Using a Virtual Chemistry Lab Support Learning. Journal of Chemical Education, 95(8), 1250-1259. https://doi.org/10.1021/acs.jchemed.8b00048

De Jong, T., & Van Joolingen, W.R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68(2), 179-201. https://doi.org/10.3102/00346543068002179

Donnelly, D., O’Reilly, J., & McGarr, O. (2013). Enhancing the Student Experiment Experience: Visible Scientific Inquiry Through a Virtual Chemistry Laboratory. Research in Science Education, 43(4), 1571-1592. https://doi.org/10.1007/s11165-012-9322-1

Fan, X., & Geelan, D.R. (2013). Enhancing students’ scientific literacy in science education using interactive simulations: A critical literature review. J.Computer in Mathematics & Science Teaching, 32(2), 125‑171.

Fiad, S.B., & Galarza, O.D. (2015). El Laboratorio Virtual como Estrategia para el Proceso de Enseñanza‑Aprendizaje del Concepto de Mol. Formación Universitaria, 8(4), 3. https://doi.org/10.4067/s0718-50062015000400002

Finkelstein, N.D., Adams, W.K., Keller, C.J., Kohl, P.B., Perkins, K.K., Podolefsky, N.S. et al. (2005). When learning about the real world is better done virtually: A study of substituting computer simulations for laboratory equipment. Physical Review Special Topics-Physics Education Research, 1(1). https://doi.org/10.1103/physrevstper.1.010103

Fuentes, A., López, J., & Pozo, S. (2019). Analysis of the digital teaching competence: Key factor in the performance of active pedagogies with augmented reality. [Análisis de la competencia digital docente: Factor clave en el desempeño de pedagogías activas con realidad aumentada] REICE.Revista Iberoamericana Sobre Calidad, Eficacia Y Cambio En Educacion, 17(2), 27-42. https://doi.org/10.15366/reice2019.17.2.002

Gambari, A.I., Kawu, H., & Falode, O.C. (2018). Impact of Virtual Laboratory on the Achievement of Secondary School Chemistry Students in Homogeneous and Heterogeneous Collaborative Environments. Contemporary Educational Technology, 9(3), 246-263. https://doi.org/10.30935/cet.444108

Gambari, I.A., Gbodi, B.E., Olakanmi, E.U., & Abalaka, E.N. (2016). Promoting Intrinsic and Extrinsic Motivation among Chemistry Students using Computer-Assisted Instruction. Contemporary Educational Technology, 7(1), 25. https://doi.org/10.30935/cedtech/6161

Hale-Hanes, C. (2015). Promoting Student Development of Models and Scientific Inquiry Skills in Acid Base Chemistry: An Important Skill Development in Preparation for AP Chemistry. Journal of Chemical Education, 92(8), 1320-1324. https://doi.org/10.1021/ed500814n

Hamilton, D., McKechnie, J., Edgerton, E., & Wilson, C. (2020). Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design. Journal of Computers in Education, 10(8),1-32. https://doi.org/10.1007/s40692-020-00169-2

Homer, B.D., & Plass, J.L. (2014). Level of interactivity and executive functions as predictors of learning in computer-based chemistry simulations. Computers in Human Behavior, 36, 365-375. https://doi.org/10.1016/j.chb.2014.03.041

Karayilan, M., Vakil, J., Fowler, D., Becker, M.L., & Cox, C.T. (2021). Zooming in on polymer chemistry and designing synthesis of high sulfur-content polymers for virtual undergraduate laboratory experiment. Journal of Chemical Education, 98(6), 2062-2073. https://doi.org/10.1021/acs.jchemed.1c00105

Karlsson, G., Ivarsson, J., & Lindstrom, B. (2013). Agreed discoveries: students’ negotiations in a virtual laboratory experiment. Instructional Science, 41(3), 455-480. https://doi.org/10.1007/s11251-012-9238-1

Lamb, R.L., & Annetta, L. (2013). The Use of Online Modules and the Effect on Student Outcomes in a High School Chemistry Class. Journal of Science Education and Technology, 22(5), 603-613. https://doi.org/10.1007/s10956-012-9417-5

Levy, D. (2013). How Dynamic Visualization Technology can Support Molecular Reasoning. Journal of Science Education and Technology, 22(5), 702-717. https://doi.org/10.1007/s10956-012-9424-6

Lynch, T., & Ghergulescu, I. (2017). Review of virtual labs as the emerging technologies for teaching STEM subjects. Paper presented at the INTED2017 Procedings, 11th International Technology, Education and Development Conference (6082-6091). https://doi.org/10.21125/inted.2017.1422

National Research Council (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. The National Academies Press. https://doi.org/10.17226/13165

OECD (2017). PISA 2015 Assessment and Analytical Framework Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving

Olakanmi, E.E. (2015). The Effects of a Web-Based Computer Simulation on Students’ Conceptual Understanding of Rate of Reaction and Attitude Towards Chemistry. Journal of Baltic Science Education, 14(5), 627-640. https://doi.org/10.33225/jbse/15.14.627

Oliveira, A., Feyzi-Behnagh, R., Ni, L., Mohsinah, A.A., Burgess, K.J., & Guo, L. (2019). Emerging technologies as pedagogical tools for teaching and learning science: A literature review. Human Behavior and Emerging Technologies, 1(2), 149-160. https://doi.org/10.1002/hbe2.141

Ortiz, C.P.M., Álvarez, J.S.O., & Sánchez, L.T.M. (2017). Laboratorio virtual de química: una experiencia de diseño interdisciplinar. Revista Virtual Universidad Católica Del Norte, 51, 98-110.

Osborne, J. (2014). Teaching Scientific Practices: Meeting the Challenge of Change. Journal of Science Teacher Education, 25(2), 177-196. https://doi.org/10.1007/s10972-014-9384-1

Osman, K., & Lee, T.T. (2013). Impact of Interactive Multimedia Module with Pedagogical Agents on Students’ Understanding and Motivation in the Learning of Electrochemistry. International Journal of Science and Mathematics Education, 12(2), 395-421. https://doi.org/10.1007/s10763-013-9407-y

Page, M.J., Mckenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D. et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372(71). https://doi.org/10.1136/bmj.n71

Papadimitropoulos, N., Dalacosta, K., & Pavlatou, E.A. (2021). Teaching Chemistry with Arduino Experiments in a Mixed Virtual-Physical Learning Environment. Journal of Science Education and Technology, 30(4), 550-566. https://doi.org/10.1007/s10956-020-09899-5

Peechapol, C. (2021). Investigating the Effect of Virtual Laboratory Simulation in Chemistry on Learning Achievement, Self-efficacy, and Learning Experience. International Journal of Emerging Technologies in Learning, 16(20), 196-207. https://doi.org/10.3991/ijet.v16i20.23561

Perkins, K., Moore, E., Podolefsky, N., Lancaster, K., & Denison, C. (2012). Towards research-based strategies for using PhET simulations in middle school physical science classes. AIP Conference Proceedings, 1413, 295-298. https://doi.org/10.1063/1.3680053

Plass, J.L., Milne, C., Homer, B.D., Schwartz, R.N., Hayward, E.O., Jordan, T. et al. (2012). Investigating the effectiveness of computer simulations for chemistry learning. Journal of Research in Science Teaching, 49(3), 394-419. https://doi.org/10.1002/tea.21008

Podolefsky, N.S., Perkins, K.K., & Adams, W.K. (2010). Factors promoting engaged exploration with computer simulations. Physical Review Special Topics - Physics Education Research, 6(2). https://doi.org/10.1103/PhysRevSTPER.6.020117

Pratidhina, E., Pujianto, P., & Sumardi, Y. (2019). Developing Computer Program as a Learning Resource on Gas Law Topics for High School Students. International Journal of Instruction, 12(2), 133-146. https://doi.org/10.29333/iji.2019.1229a

Rizvi, Y.S., & Nabi, A. (2021). Transformation of learning from real to virtual: an exploratory-descriptive analysis of issues and challenges. Journal of Research in Innovative Teaching and Learning, 14(1), 5-17. https://doi.org/10.1108/jrit-10-2020-0052

Ronen, M., & Eliahu, M. (2000). Simulation – a bridge between theory and reality: the case of electric circuits. Journal of Computer Assisted Learning, 16(1), 14-26. https://doi.org/10.1046/j.1365-2729.2000.00112.x

Rutten, N., Van Joolingen, W.R., & Van Der Veen, J.T. (2012). The learning effects of computer simulations in science education. Computers and Education, 58(1), 136-153. https://doi.org/10.1016/j.compedu.2011.07.017

Ryoo, K., Bedell, K., & Swearingen, A. (2018). Promoting Linguistically Diverse Students’ Short-Term and Long-Term Understanding of Chemical Phenomena Using Visualizations. Journal of Science Education and Technology, 27(6), 508-522. https://doi.org/10.1007/s10956-018-9739-z

Samur, Y., & Evans, M.A. (2011). Learning science through computer games and simulations – Edited by Margaret A Honey & Margaret Hilton. British Journal of Educational Technology, 42(6), E171-E172 https://doi.org/10.1111/j.1467-8535.2011.01247_6.x

Smetana, L.K., & Bell, R.L. (2014). Which setting to choose: Comparison of whole-class vs. small-group computer simulation use. Journal of Science Education and Technology, 23(4), 481-495. https://doi.org/10.1007/s10956-013-9479-z

Stieff, M. (2019). Improving Learning Outcomes in Secondary Chemistry with Visualization-Supported Inquiry Activities. Journal of Chemical Education, 96(7), 1300-1307. https://doi.org/10.1021/acs.jchemed.9b00205

Tatli, Z., & Ayas, A. (2012). Virtual Chemistry Laboratory: Effect Of Constructivist Learning Environment. Turkish Online Journal of Distance Education, 13(1), 183-199.

Tatli, Z., & Ayas, A. (2013). Effect of a Virtual Chemistry Laboratory on Students’ Achievement. Journal of Educational Technology & Society, 16(1), 159-170. Available at: http://www.jstor.org/stable/jeductechsoci.16.1.159

Trudel, L., & Metioui, A. (2011). Conception of a computer-aided physics laboratory to facilitate the understanding of kinematical concepts. IMSCI’11: The 5th International Multi-Conference on Society, Cybernetics and Informatics (186-191).

Udo, M.E., & Etiubon, R.U. (2011). Computer-Based Science Simulations, Guided-Discovery and Students’ Performance in Chemistry. Modern Applied Science, 5(6). https://doi.org/10.5539/mas.v5n6p211

Ullah, S., Ali, N., & Rahman, S.U. (2016). The Effect of Procedural Guidance on Students’ Skill Enhancement in a Virtual Chemistry Laboratory. Journal of Chemical Education, 93(12), 2018-2025. https://doi.org/10.1021/acs.jchemed.5b00969

Vandenplas, J.R., Herrington, D.G., Shrode, A.D., & Sweeder, R.D. (2021). Use of Simulations and Screencasts to Increase Student Understanding of Energy Concepts in Bonding. Journal of Chemical Education, 98(3), 730-744. https://doi.org/10.1021/acs.jchemed.0c00470

Velasco, J., & Buteler, L. (2017). Computational simulations in physics education: A critical review of the literature. [Simulaciones computacionales en la enseñanza de la física: una revisión crítica de los últimos años]. Ensenanza De Las Ciencias, 35(2), 161-178. https://doi.org/10.5565/rev/ensciencias.2117

Waight, N., Liu, X., Gregorius, R.M., Smith, E., & Park, M. (2014). Teacher Conceptions and Approaches Associated with an Immersive Instructional Implementation of Computer-Based Models and Assessment in a Secondary Chemistry Classroom. International Journal of Science Education, 36(3), 467-505. https://doi.org/10.1080/09500693.2013.787506

Waight, N., & Gillmeister, K. (2014). Teachers and Students’ Conceptions of Computer-Based Models in the Context of High School Chemistry: Elicitations at the Pre-intervention Stage. Research in Science Education, 44(2), 335-361. https://doi.org/10.1007/s11165-013-9385-7

Wang, A.I., & Tahir, R. (2020). The effect of using Kahoot! for learning – A literature review. Computers and Education 149, 103818. https://doi.org/10.1016/j.compedu.2020.103818

Wen, C., Liu, C., Chang, H., Chang, C., Chang, M., Chiang, S.F. et al. (2020). Students’ guided inquiry with simulation and its relation to school science achievement and scientific literacy. Computers & Education, 149, 103830. https://doi.org/10.1016/j.compedu.2020.103830

Zacharia, Z.C. (2007). Comparing and combining real and virtual experimentation: An effort to enhance students’ conceptual understanding of electric circuits. Journal of Computer Assisted Learning, 23(2), 120‑132. https://doi.org/10.1111/j.1365-2729.2006.00215.x

Zacharia, Z.C., Olympiou, G., & Papaevripidou, M. (2008). Effects of experimenting with physical and virtual manipulatives on students’ conceptual understanding in heat and temperature. Journal of Research in Science Teaching, 45(9), https://doi.org/10.1002/tea.20260

Zohar, A.R., & Levy, S.T. (2019). Attraction vs. repulsion - learning about forces and energy in chemical bonding with the ELI-Chem simulation. Chemistry Education Research and Practice, 20(4), 667-684. https://doi.org/10.1039/c9rp00007k

---