Journal of Technology and Science Education

• Home
• About
• Login
• Archives
• Submissions
• Publication Fee
• Indexing & Statistics

Home > Vol 11, No 2 (2021) > Rahmawati

THE INTEGRATION OF DILEMMAS STORIES WITH STEM‑PROJECT‑BASED LEARNING: ANALYZING STUDENTS’ THINKING SKILLS USING HESS’ COGNITIVE RIGOR MATRIX

1Universitas Negeri Jakarta (Indonesia)

2SMAN 4 Tambun Selatan (Indonesia)

Received April 2021

Accepted June 2021

Abstract

This study aimed to analyze students’ thinking skills through integrating dilemmas stories with a Science, Technology, Engineering, and Mathematics-Project-Based Learning (STEM-PBL) in polymers topic. The participants were 47 Grade 12 students from a public senior high school in West Java Province. The research employed a qualitative method to obtain data through classroom observations, reflective journals, interviews, and a concept test. Students’ thinking skills were assessed using Hess’ Cognitive Rigor Matrix, which combines Bloom’s taxonomic and Webb’s depth of knowledge. The results show that most students reach level C2 of Bloom’s taxonomy and level 1 of Webb’s depth of knowledge, meaning that most students understand and recall the information. The results indicate that integrating dilemmas stories with STEM-PBL enables students to explain using simple sentences and prior knowledge. In addition, this approach provides an opportunity for students to develop their critical thinking, creativity, and argumentation skills through problem-solving and project making.

 

Keywords – Polymer learning, Thinking skills, Hess’ cognitive rigor matrix, Dilemmas stories, STEM, Polymer.

To cite this article:

Rahmawati, Y., Afrizal, A., Dwi Astari, D., Budi Utami, D., & Muhab, S. (2021). The integration of dilemmas stories with stem-project-based learning: Analyzing students’ thinking skills using hess’ cognitive rigor matrix. Journal of Technology and Science Education, 11(2), 419-439. https://doi.org/10.3926/jotse.1292

 

----------

1. 1. 1. Introduction

Educational institutions have responded to the rapid increase of science and technology by focusing on improving students’ thinking skills to ensure they can compete globally in the 21st century. The shift of the industrial economy toward an era of information technology is forcing the education system to prepare graduates with the necessary skills and competencies (AlMaadeed & Ponnamma, 2020; Lewis, 2018; Sima, Gheorghe, Subić & Nancu, 2020). Basic knowledge and skills are no longer enough; students must have the future skills such as information and data processing skills (Fleaca & Stanciu, 2019), creative and critical thinking (Qadir, Yau, Ali Imran & Al-Fuqaha, 2020; Rodríguez, Díez, Pérez, Baños & Carrió, 2019), and thinking skills such as system thinking and integrating traditional thinking to future thinking (Pacis & VanWynsberghe, 2020). Therefore, the main objective of education to develop and improve students’ high-level thinking skills (Altanis, Retalis & Petropoulou, 2018) and deeper levels of learning (Lourie, 2020). Thus, across boundaries learning as contextual learning needs to be carried out in order to students do not only recall the information but also put conceptual understanding into context (Lourie, 2020).

In their study on learning model to develop students’ higher-order thinking skills, Fanani and Kusmaharti (2018) state that many Indonesian students have a low ability to understand complex information, theorize, analyze, solve problems, use tools, follow procedures, and conduct investigations. In 2013, The Government of Indonesia sought to improve students’ thinking skills by replacing the Kurikulum Tingkat Satuan Pendidikan (KTSP) curriculum that had been in place since 2006. The 2013 curriculum was designed to develop students’ character traits and 21st-century skills, such as critical thinking and problem‑solving, collaboration, creativity, and communication (Ministry of Education and Culture, 2018). However, Indonesia’s national exam results of 2015 indicated low levels of thinking skills among Indonesian students in areas such as reasoning, analyzing, and evaluating (Ministry of Education and Culture, 2017). In addition, the demand for mastering sufficiently dense learning material is also a problem in the Indonesian education system (Sukasni & Efendy, 2017). Based on the results of teacher interviews conducted at State Senior High School (SMAN) 4 Tambun Selatan, West Java, researchers found that the development of higher-order thinking skills was not a top priority due to competing demands; the students are required to master 15 subjects and prepare for the national exam. The national exam is a requirement of the Indonesian education system, and is a key focus, particularly for Grade 12 students (usually aged 17-18); studying national exam questions takes precedence over the development of students’ thinking skills in mastering and applying concepts to solve problems in daily life.

Students’ thinking skills can be developed through different approaches to the learning process, such as the use of dilemmas stories. Based on a previous study, dilemmas stories provide students the opportunity to make decisions and solve problems (Taylor, Taylor & Chow, 2013). The dilemmas approach can spur students to think critically, provide critical judgments on various arguments, and transform their perspectives; it is part of transformative learning (Settelmaier, 2004). Dilemmas stories can be applied to science learning, especially when discussing socio-cultural issues related to sustainable development (Taylor, Taylor & Hill, 2019).

Another contextual approach to learning is project-based Science, Technology, Engineering, and Mathematics (STEM). STEM integrated PBL can increase students’ interest in learning (Campbell, Speldewinde, Howitt & MacDonald, 2018; Wan, So & Zhan, 2020; Ziaeefard, Miller, Rastgaar & Mahmoudian, 2017), provide meaningful learning experiences (Awad & Barak, 2018; Campbell et al., 2018; EL-Deghaidy, Mansour, Alzaghibi & Alhammad, 2017), enhance students’ creativity and critical thinking (Blackley, Rahmawati, Fitriani, Sheffield & Koul, 2018; Rahmawati, Ridwan, Hadinugrahaningsih & Soeprijanto, 2019; Rahmawati, Ridwan, Mardiah & Afrizal, 2020), and give more chance for them to apply STEM knowledge to solve real-life problems (Cook & Bush, 2018; Lin, Hsiao, Williams & Chen, 2020). In addition, PBL provides challenges and motivates students to think critically and analytically, which improves their higher-order thinking skills (Capraro, 2013).

This study seeks to determine whether integrating dilemmas stories with STEM project-based learning (STEM-PBL) is effective in developing students’ thinking skills in chemistry learning.

2. Methodology

The research method used is a qualitative method. Qualitative research methods are based on text or image analysis, have certain steps in data analysis, and various research strategies. Qualitative research uses all perspectives in analyzing research data (Creswell, 2009).

2.1. Research Design

The research conducted with a group of 47 students made up of 10 male and 30 female year 12 secondary students in West Java, Indonesia with polymer as the topic of chemistry learning. Polymers are one of the topics in chemistry learning that have wide applications in everyday life. Polymer learning aims to increase students’ insight and awareness about the problems of polymer such as plastics waste. Participants were selected based on the suitability of the teaching material to the class level and the desire to participate in the research. Data collection was carried out through various techniques, such as interviews, reflective journals, observation, STEM project assessments, and open-ended test questions on polymer topics.

The research was conducted in three stages, namely the preliminary stage, implementation, and the final stage. The research flow is shown in Figure 1.

At the preliminary stage, the researcher compiled an interview protocol for teacher and students to obtain information related to chemistry learning in school and students’ thinking skills. At this stage, the researchers also compiled a lesson plan using STEM-PBL, developed dilemmas stories about biodegradable plastics, and compiling research instruments such as observation sheets, reflective journals, students’ worksheets, the preparation and validation of open-ended polymer test questions.

The research implementation was carried out through the STEM-PBL learning stages developed by Laboy-Rush (2010) and integrated dilemmas stories into it. The five learning stages are described in Figure 2.

In this study, students are given the opportunity to make two products that have differences in the design procedure and the level of reagents (glycerin, acetic acid, and chitosan) so they can analyze the function of reagents and determine the appropriate product design to get maximum results. During the lesson, students are given worksheets and reflective journals, while the teacher, assisted by two observers, made observations to analyze the effectiveness of integrating dilemma stories in the STEM project and their impact on the development of students’ thinking skills.

Figure 1. Research Flow and Data Collections

Figure 2. The five stages of integrating dilemmas stories with STEM project

At the final stage, students are given open-ended tests to assess their understanding on polymer topic and their thinking skills. Researchers conducted the interviews to get the information about students’ perspective on the integration of dilemmas stories on STEM project and students’ thinking skills development. Then researchers analyzed the research data using Hess’s Cognitive Rigor Matrix which describes the attainment of Bloom’s taxonomy level and Webb’s DOK level (Webb’s depth of knowledge). The Bloom Taxonomy level explains the level of students’ cognitive processes and Webb’s DOK level explains the depth or complexity of students’ understanding of concepts to answer a question (Hess, Jones & Walkup, 2009).

2.2. Data Collection

The research data were obtained using multiple collection such as interviews, reflective journals, observation, STEM project assessments, and open-ended test questions on polymer topics.

2.2.1. Interviews

Interviews were conducted with teachers and students. The teacher interview aimed to find out how teacher usually carried out polymer learning, the problems that occur in chemistry learning, and to find out students’ thinking skills. The student interviews were conducted with 9 people based on the results of the analysis of reflective journals and student involvement in learning. Student interviews aimed to collect information about students’ impressions of the learning that has been carried out and observe the development of students’ thinking skills by asking questions related to the learning process such as their communication and collaboration in groups to solve problems in dilemmas stories through the development of a STEM project. The following is an example of interview question asked of students.

What materials does your group use as a base for making biodegradable plastics? What is the reason?

2.2.2. Reflective Journals

The students completed reflective journals during the learning process, precisely at the end of each meeting. Reflective journal is used to collect data related to students’ reflections on the level of involvement and difficulties encountered during the application of dilemmas stories in the STEM project on polymer learning. The example of questions in a reflective journal is:

What do you think about today’s learning in relation to thinking skills development?
Write your reflection in the column provided

2.2.3. Observations Sheet

Observations were made to understand the conditions of learning in the classroom and the development of students’ thinking skills. Observations by researchers assisted by two observers were carried out during the learning process to ensure that the application of learning was by the STEM-PBL learning steps by Laboy‑Rush (2010), students’ thinking skills, conceptual understanding, and students’ involvement in learning.

2.2.4. Polymer Open-Ended Test

The open-ended test aimed to determine students’ thinking skills as an implication of integrating the dilemmas stories in the STEM project. This test consisted of five essay with 14 total item questions compiled by researchers based on polymer learning indicators in the 2013 curriculum namely analyzing the structure, nomenclature, properties and classifications of macromolecules (such as polymer) and reasoning about the manufacture of a product from macromolecules. The distribution of items based on indicators that have been compiled by researchers can be observed through the following Table 1.

In this study, open-ended questions were validated by 5 experts consisting of 4 Chemistry Lecturers and Chemistry Education Lecturers at Universitas Negeri Jakarta and 1 high school chemistry teacher in grade 12. The test was conducted at the end of the polymer topic lesson. The following is one of the questions being tested.

No.	Indicators	Question Item Number
1	Describing the monomer structure of a polymer	1a
2	Writing the names and polymerization reactions (addition and condensation) of the monomers	2a
3	Differentiating the classification of polymers on the basis of origin, type of constituent, polymerization reactions and properties to heat	2b
4	Explaining the use of synthetic polymers in everyday life	1c
5	Analyzing polymer properties based on the structure and the constituent monomers	1b, 2c, 3a, 4a, 4b
6	Evaluating the process and impact of polymer formation	3b, 3c, 5a
7	Creating a polymer impact mitigation plan	5b, 5c

Table 1. Distribution of Polymer Concept Test Question Based on Learning Indicators

Natural rubber comes from rubber tree sap which chemically contains isoprene polymer.

Natural rubber is a liquid, stiff when heated and sticky when cold. In 1839 Charles Goodyear conducted experiments on natural rubber which was heated and added with sulfur to find the Rubber Tires we use today. Following is the structure of the polymer rubber tires:

a) Write the polymerization reaction of isoprene to polyisoprene. b) Are the two polymers the same based on their classification? Explain your reasons! And c) Based on your analysis, explain the purpose of adding sulfur to the experiment!

2.3. Data Analysis

Qualitative data analysis techniques are carried out through the stages of data reduction, display data, and conclusion drawing/verification (Miles & Huberman, 1994). The data obtained during the study through interviews, observations, reflective journals, and polymer open-ended tests were reduced by selecting data that was under the focus of students’ thinking skills. At the data display stage, the data that has been reduced is then presented in the form of a matrix or coding based on the Hess’ Cognitive Rigor Matrix indicator so that data verification can be carried out and conclusions can be drawn.

To draw the conclusion and verified the data, credibility criteria are used including prolonged engagement, persistent observation, progressive subjectivity, and member checking (Miles & Huberman, 1994) to test the validity and trustworthiness of the data (Guba & Lincoln, 1989). The prolonged engagement was carried out during the study (December 2018 - July 2019) to understand the context of the study, explore incidents to explain in detail the research situation and participant involvement, and integrated dilemmas stories in the STEM project for 5 weeks in polymer learning. Persistent observation is carried out to explore various phenomena by involving researchers directly in the learning process. In making continuous and in-depth observations, the researcher was assisted by two observers to analyze the learning process and the development of students’ thinking skills. Progressive subjectivity is carried out to monitor research results with all notes obtained during the study based on the characteristics of students’ thinking skills. Member checking is done to ensure the accuracy of the data obtained from students as data providers regarding the various research results. All data obtained, both in the form of hard paper copies and interview transcripts, were kept privately by the researcher for the privacy of participants and confidentiality.

3. Results and Discussion

3.1. The Integration of Dilemmas Stories with a STEM Project in Chemistry Learning

The dilemmas stories highlight a problem to be solved using a STEM project (Laboy-Rush, 2010) in five stages as referred before.

3.1.1. Reflection

In the first stage, students are given initial questions about the concept being studied and the problems raised in the dilemmas stories read by the teacher. The dilemmas of plastic are raised in the stories because plastic is one example of polymers that are widely used and easily found in everyday life. Plastic is flexible, elastic, strong, and durable and is ubiquitous in today’s society, commonly used for food and beverage packaging, household appliances, and office equipment. People use plastic because it is cheap, practical, strong, and easy to obtain; however, the demand for plastic causes environmental problems because it is not easily biodegradable (Ganesh, Anjana, Hinduja, Sujitha & Dharani, 2020). According to statistical data, 14% (5.4 million tons/year) of waste production in Indonesia is plastic (Handayani & Wijayanti, 2015). Synthetic polymers are widely used and they are non-biodegradable; therefore, the environmental implications of their use must be considered (Aripin, Saing & Kustiyah, 2017). Therefore, the topic of plastics is raised in the stories to increase students’ interest and encourage them to solve plastic problems based on the polymer concepts that have been studied. The dilemmas stories share experiences and information providing various perceptions and perspectives so students can think according to their cultural background, increase their self-awareness, and combine thoughts, emotions, and actions (Morais, 2015). Table 2 provides a summary of the issues and descriptions raised in the dilemmas stories.

The plastic dilemmas stories are accompanied by questions that seek to involve the students in overcoming the dilemmas presented. The students work individually at this stage to encourage critical thinking, enhance their motivation to be actively involved in learning, and increase their curiousity (Adriyawati, Utomo, Rahmawati & Mardiah, 2020). Students then discuss their answers in groups and collaborate to agree on a single solution for the group to progress into the next stage.

No.	Issue	Description
1	The danger of using plastic bags on health	The story tells of a mother and child who sell cakes using black plastic for packaging. The mother and child also often re-use plastic soft drink bottles as drinking containers. The child is a student in high school and, while learning the various types of polymer material, learns of the dangers and suggested uses of polymers. The child realizes the health impacts of using plastic and begins to worry about the mother’s health; the mother is thought to have suffered from uterine cancer so her health is impaired. The child explains the issues to the mother and asks the mother to use an alternative to plastic, but the mother feels worried that it will affect their income.
2	Impact of the policy on the prohibition of using plastics	The story tells of a father who has worked in a factory for 25 years to support his wife and five children. The story is told from the perspective of one of the children who is a high school student. The child watches the news on TV about a dead whale stranded on Wakatobi Island and found with plastic waste in its stomach. Shortly after the news, political parties begin supporting the reduction and even the prohibition of the use of plastic in some areas. The Government of Jakarta is in the process of ratifying the ban on the use of plastic in the capital. The child’s friend at school explains that plastic is bad for the environment and living things; however, the child is worried about his father’s work. The father must continue to work to support his family. and is of an advanced age, which makes it difficult to find other jobs.

Table 2. Summary of the Plastic Dilemmas Stories

This is an example of student’s perception of the integration of dilemmas stories on the learning process: Problems in a plastic dilemma require problem solving that are not easy, it is necessary to consider both sides which are equally difficult to solve. But it is very effective in expressing the opinions and creative side of students.

(Interview of Student 17, 22 February 2019)

The interview excerpt shows that solving problems can hone students’ creative and critical thinking skills. Creative thinking is demonstrated by the students agreeing on a solution when different proposals are presented. Critical thinking is shown when students consider the dilemmas, understand the negative and positive impacts on different actors in the stories, and consider solutions based on their values. The use of dilemmas stories in the classroom shows that students can reflect their values on assumptions and decisions and improve their ability to think critically and apply science (Werth, 2017).

3.1.2. Research

At this stage, the students research solutions to the problem and plan their biodegradable plastic project. At first, students individually look for information about solutions to the problems presented in the dilemmas stories. The students then discuss their findings in groups (see Figure 3). The students are expected to exchange information and argue their proposal effectively with members of their group. It is also expected that the students can agree on the best solution to the problem through group discussion.

Figure 3. Classroom discussion about the plastic dilemmas

Figure 3 shows that the discussion between groups is an active process. This was noted by the classroom observer:

At the end of the dilemmas story application, students are triggered to criticize the opinions of others and look very enthusiastic when doing so. The opinion given is also related to the project to be carried out about biodegradable plastics

(Classroom Observation, 1 February 2019)

During this process, students learn to listen and respect the opinions of others. The students’ cognitive levels are reflected by being able to explain their proposals and arriving at a group solution. The students must analyze each other’s opinions. This is in accordance with previous research by Heriyanti (2015), whereby students ask each other questions and evaluate by expressing their opinions on the solution. The dilemmas story approach allows students to think critically and creatively. Students first analyze, synthesize, and evaluate information; then they identify the issue, search and organize information, generalize ideas, estimate possibilities, and determine actions; and finally, they reflect their findings through discussion and action (Werth, 2017).

In discussions, students look for deeper information about biodegradable plastics to defend their opinions. This shows that the information literacy of students has developed; that is, the ability to selecting, utilizing, applying, managing, and exchanging information (van Laar, van Deursen, van Dijk & de Haan, 2017). This was reflected in the student interviews, for example:

Student 15:	Biodegradable plastic is environmentally friendly, because it is easier and faster to decompose on the ground. I propose it because the plastic is more environmentally friendly than conventional plastic.
Teacher:	Why is biodegradable plastic easier to decompose than conventional plastic?
Student 15:	Because the main ingredient is plants or natural polymers. Plants and animals are easy to decompose by organisms.

(Student Interview, 22 February 2019)

The interviews excerpt shows that the student understands the concept of biodegradable plastics and can explain why they easily decompose. The student’s understanding aligns with the literature; biodegradable plastic made from biopolymer can be degraded by microbes (bacteria and fungi) producing carbon dioxide, methane, and water molecule that are not harmful to the environment (Roohi, Bano, Kuddus, Zaheer, Zia, Khan et al. (2017). Biodegradable plastic can be decomposed 20 times faster than conventional plastic and are derived from sustainable sources (Ebnesajjad, 2013).

3.1.3. Discovery

The students draw up plans and procedures within their group for making biodegradable plastics, with different raw materials provided to each group. The students also make a timeline for completing their projects. Figure 4 provides an example of a timeline designed by the students.

Students then explain their procedures to the class through group presentations (see Figure 5). This stage is undertaken to determine the readiness and extent to which students understand plastic manufacturing procedures.

The presentations revealed that the students used YouTube, blogs, and online articles as sources of information. Although the teacher suggests using articles, students prefer to find tutorial videos on YouTube because they find them easier to understand. The selection of various sources of information remains under the supervision of the teacher. The variety of sources shows how easily information is obtained by students utilizing technology; students are accustomed to using technology to facilitate learning, promote students’ autonomy and learning motivation (Ahmadi, 2018).

Figure 4. Project timeline for completing a biodegradable plastic project

Figure 5. Students presenting their plans for making biodegradable plastic

In the discovery stage, students also conduct trials on procedures that have been presented previously. Students are allowed to conduct trials at school or at home, which are then discussed with the teacher. The excerpt below is an example of how the teacher monitored progress throughout the trial process.

Student 23:	We have smoothed the straw, but when we add water, it does not blend, why is that, ma’am?
Teacher:	The straw is hard to smooth, I guessed it would be difficult. Why did you choose straw?
Student 23:	Because straw is rich in cellulose and cellulose can be made into plastic.
Teacher:	If you want, try using what has become powder, such as rice bran.
Student 23:	I’ll find out later, I tried blending straw with a blender since yesterday, but it’s not yet smooth, ma’am.
Teacher:	Have you tried other ways? For example, dried and pounded it?
Student 23:	I have oven dried, then I add more water to be blended with a blender.
Teacher:	Then what is the aim of the oven, if it is given more water?
Student 23:	Oh yes ma’am, I try to like others’ grouplike other groups, but I don’t know yet whether it will settle

(Classroom Observation, 9 February 2019)

In this example, the student tried various ways of smoothing straw to make biodegradable plastic. This shows a high level of interest and motivation to complete the project successfully (Rosenzweig & Wigfield, 2016); STEM-based learning can increase student motivation while providing experience in the product manufacturing process (Kuo, Tseng & Yang, 2019).

3.1.4. Application

In the fourth stage, the students apply the knowledge gained to complete the biodegradable plastic project. The students were interested and engaged, and all group members worked together effectively on the project. This can be observed through the following reflective journal entries:

My group collaboration is done well because the communication continues well and each member complements the materials

(Reflective Journal of Student 28, 22 February 2019)

I like my group because each other reminds each other and helps in evaluating each experiment for the experiment

(Reflective Journal of Student 12, 22 February 2019)

The results indicate that PBL stimulates effective group collaboration. Students communicate and work together to solve problems, test ideas, and evaluate the outcomes. The implementation of project making can be observed in Figure 6.

In addition to group collaboration, students also conduct several experiments using variations in the concentration of materials to get the best plastic results. The results of this process support previous research stating that the application of project-based STEM encourages students to be more active and improves the learning experience (Lou, Chou, Shih & Chung, 2017). Students’ motivation and interest is increased with the new experience. This is supported by Tseng, Chang, Lou and Chen (2013), who found that the activity of designing and implementing a project increases engineering students’ interest in learning.

Figure 6. Students working on a biodegradable plastic making project

3.1.5. Communication

In the final stage, students present the results of biodegradable plastic products made during the practicum through group presentations. This stage aims to improve students’ delivery of the results of their experiments. Students briefly explain how they made the plastic, display the results, and evaluate the outcome (Figure 7).

Figure 7. Students presenting their results

Jackfruit Seed	Sweet Potato Starch	Bran		Potato	Pineapple
Cassava Starch		Banana Peel	Cornstarch

Figure 8. Biodegradable plastic products produced by the students

The students presented pictures of the results of their experiments (see Figure 8) and explained the difference in the process of making the two products both in terms of the design procedure and the level of reagents (glycerin, acetic acid, and chitosan) used. This process of information sharing encourages students to build their knowledge through investigation and discussion to solve problems and achieve their goals (Tseng et al., 2013).

This process also provides new motivation for students with less-successful outcomes, as shown in the following reflective journal entries:

Not satisfied, because we have not gotten the plastic as desired. The plan is to keep trying until we get the desired plastic

(Reflective Journal of Student 3, 22 February 2019)

I am not satisfied with the result; the next step is to find more complete information about making biodegradable plastics so that we are more adept at making it and reduce non-biodegradable plastic waste

(Reflective Journal of Student 20, 22 February 2019)

STEM-PBL allows students to apply their knowledge to produce a tangible outcome. This study demonstrates that students become increasingly active in learning about the biodegradable plastic manufacturing process. Some students expressed their opinions of STEM-PBL, as shown in the following reflective journal entry:

In my opinion learning with the STEM project is very good because students besides being able to hone knowledge can also practice creativity like in plastic making, students will do a lot of experiments

(Reflective Journal of Student 13, 22 February 2019)

This entry demonstrates that the student is interested and motivated to undertake several experiments to achieve a good biodegradable plastic product. Students gain new experiences in STEM-PBL, which generates motivation and interest (Kuo et al., 2019).

3.2. Assessing Students’ Thinking Skills based on Hess’ Cognitive Rigor Matrix (CRM)

Students’ thinking skills are explained qualitatively based on the Hess’ CRM which is a matrix that combines Bloom’s taxonomy level with Webb’s depth of knowledge (DOK). Bloom’s Taxonomy level explains the level of cognitive processes, while Webb’s DOK explains the depth or complexity of understanding required to answer a problem (Hess et al., 2009). Students’ thinking skills are described through the achievement of Bloom’s taxonomy level and Webb’s depth of knowledge (DOK) level based on Hess’s Cognitive Rigor Matrix. The revised Bloom’s taxonomy consists of 6 levels in sequence, namely C1 (remembering), C2 (understanding), C3 (applying), C4 (analyzing), C5 (evaluating) and C6 (creating). Webb’s DOK consists of 4 levels in sequence, namely level 1 (recall), level 2 (skills and concepts), level 3 (strategic thinking) and level 4 (extended thinking).

Based on the results of students’ answers to the concept test with five open-ended questions, the percentage of students’ thinking skill levels that appear on each indicator (as referred before) is shown in the following Table 3.

The percentage that appears on each indicator is then analyzed using Hess’s Cognitive Rigor Matrix to find out the overall students’ thinking skills level. The analysis result is shown in the Table 4.

Level (Webb’s DOK-Bloom’s Taxonomy)	Indicators
	1	2	3	4	5	6	7
1-C1	17.02%	17.02%	19.14%	10.64%	8.91%	6.25%	5.71%
1-C2			80.85%	89.36%	27.28%	63.19%	50.48%
2-C2	82.98%				18.60%	30.56%	22.86%
1-C3		82.98%
2-C3							20.95%
1-C4					27.38%
2-C4					17.82%

Table 3. Level of Students’ Thinking Skills on Each Indicator

Bloom’s Taxonomy	Webb’s Depth of Knowledge
	Level 1 Recall	Level 2 Skills and Concepts	Level 3 Strategic Thinking	Level 4 Extended Thinking
Remembering (C1)	9.88%
Understanding (C2)	47.42%	23.56%
Applying (C3)	5.92%	1.67%
Analyzing (C4)	8.05%	3.5%
Evaluating (C5)
Creating (C6)

Table 4. Students’ thinking skills based on the Polymer Concept Test using Hess’ Cognitive Rigor Matrix

The results of the concept test demonstrate that students’ thinking skills reach four levels of Bloom’s Taxonomy: remembering, understanding, applying, and analyzing; and the level of depth of students’ understanding reaches 2 levels of Webb’s DOK: recall and skills and concepts. The level of C1 (remembering) only appears with a depth of level 1 (Recall), whereas the levels of (C2) understanding, (C3) applying, and (C4) analyzing reach levels 1 (Recall) and 2 (Skills and concepts).

The results also show the highest percentage of students were in level 1-C2 (47.42%), indicating that 47.42% of the students understand and can explain a concept that they have learned. The smallest percentage of student achievement is at level 2-C3 (1.67%), indicating that only 1.67% of students can apply a procedure or formula to answer the related test questions.

The level of thinking skills that emerged based on the analysis with the Hess’ CRM is explained in detail below.

3.2.1. Remembering

Bloom’s taxonomy level C1 (remembering) level appears only with Webb’s DOK level 1 (Recall). This level of thinking skills illustrates that students can remember, recognize, or identify knowledge sourced from their long-term memory. An example is as follows:

Problem 4a: Explain how the structure affects the polymer density.

Student Answer: Linear polymers have higher densities such as HDPE than branched polymers.

(Concept Test Answer of Student 1)

Problem 4a requires students to explain how the structure of high-density polyethylene (HDPE) and low‑density polyethylene (LDPE) affect the density of the polymer. The answer shows that the student does not explain how the structure affects the density of the polymer, so the researcher can assume that the student partially remembers the explanation in class about the possible arrangement of polymer structures. The literature states that LDPE has a branched polymer structure, and the presence of branches causes irregularity so crystallinity decreases, volume increases, and density decreases. HDPE, however, has a linear structure and the intermolecular force is greater, which increases crystallinity and density (Brown, 2012; Carraher, 2003).

The cognitive process of remembering comes from aspects of long-term memory such as knowing and mentioning (Tikkanen & Aksela, 2012). The process of recalling, recognizing, and identifying facts or characteristics is included in the evaluation of thinking skills  (Hess et al., 2009).

Level 1-C1 thinking skills can also be observed through the following extracts:

The chain starch is unbranched, while the amylopectin is branched

(Interview of Student 4, 10 March 2019)

We have learned about polymer, macromolecule, and carbohydrate

(Reflective Journal of Student 7, 1 February 2019)

These statements align with the literature; starch is composed of D-glucose with a linear structure while amylopectin is composed of D-glucose with a branched structure (Carraher, 2003) and the students’ display the thinking skills of recall and remembering. Students demonstrating level 1 (Recall) thinking skills can directly answer questions without the need for problem-solving (Hess, 2010).

3.2.2. Understanding

Based on the results, Bloom’s taxonomy level C2 (understanding) appears on all indicators except indicator of write the naming of polymers and the depiction of polymerization reaction such as addition and condensation. Webb’s DOK levels 1 (Recall) and 2 (Skill and Concepts) are evident. Level 1-C2 shows that students can explain concepts such as understanding polymers and can explain the meaning of symbols, while level 2-C2 indicates that students can explain causality, draw conclusions, and use concepts to solve a problem. Level 1-C2 thinking skills are demonstrated in the following answer:

Problem 4a: Explain how the structure affects the polymer density.

Student Answer: Because these polymers bind to one another so that it gives a density effect.

(Concept Test Answer of Student 32)

The student has shown that they understand and can explain that the bonds between polymers influence their density. This aligns with the literature; the more crosslinked, the higher the polymer density (Brown, 2012; Carraher, 2003). However, the student’s answer does not address the question’s intent, which is to explain the effect of HDPE (linear) and LDPE (branched) polymer structures. This is likely due to the student not understanding the question.

Students’ understanding at level 1-C2 is also shown in the following classroom observation and reflective journal entries:

Peptide bonds structures include condensation reactions because there are 2 functional groups and the reaction releases H2O

(Classroom Observation, 1 February 2019)

Different properties in various polymers are causes by the manufacturing material, manufacturing process, composition of the material used and the procedures performed

(Reflective Journal of Student 37, 4 February 2019)

These results indicate that students understand and can explain condensation reactions and differences in nature using knowledge retained in their memory. This is because the teacher has explained the condensation reaction and the differences in the properties of polymers with linear, branched, or crosslinked structures. The classroom observation aligns with the literature; the formation of peptide bonds includes condensation reactions because in their reactions they release small H2O molecules (Chang, 2005). The reflective journal statement suggests that the student is unable to explain how different kinds of polymers differ based on the previously taught polymer concept. This is similar to the results of previous research by Nursa’adah, Kurniawati and Yunita (2016), where students could explain the concepts but had difficulty explaining the context related to the concepts. Therefore, it can be concluded that the student’s understanding is only at level 1-C2.

However, level 2-C2 was also observed through the following concept test result:

Problem 4a: Explain how the structure affects the polymer density

Student Answer: Between linear polymers and branched polymers, the density level is higher in linear polymers because if the linear polymers are united, the structure will be denser and not hollow at all

(Concept Test Answer of Student 41)

The student answers by explaining the structural arrangement of linear polymers versus branched polymers. The answer shows that the student can identify and explain how the density of a polymer is based on a linear and branched structure, which aligns with the literature; the presence of branches on the polymer structure causes the volume to increase and reduce the density of the polymer (Carraher, 2003).

Students’ understanding at level 2-C2 can also be shown in the following reflective journals entry:

The difference in polymer properties lies in the type of polymer arrangement. When the arrangement is linear, it is elastic when the arrangement is crosslinked, then it is strong and stiff. Besides it lies in the composition of its constituent ingredients, the mixture of chitosan and glycerin affects the plastic tensile strength

(Reflective Journal of Student 17, 4 February 2019)

This student understands and can explain how structure affects the properties of polymers and can explain the effect of the addition of glycerin and chitosan, based on experiments that have been conducted. This shows that the student can make conclusions and explain the relationship between concepts, which is an indication of level 2-C2 thinking skills (Hess et al., 2009).

3.2.3. Applying

Students reach thinking level 1-C3 when they can apply formulas, simple procedures, or methods that originate from long-term memory to answering questions or solve problems, while level 2-C3 illustrates that students can answer questions by applying several steps, formulas, or concepts. Level 1-C3 thinking skills are demonstrated in the following answer:

Problem 2a: write down the isoprene polymerization reaction to polyisoprene

Student Answer: Isoprene is the basis of polyisoprene, so the reaction is as follows

(Concept Test Answer of Student 13)

The student predicted addition reactions because of the double bonds and wrote down the reaction to the formation of polyisoprene. This answer aligns with the literature; polyisoprene is formed through the addition of polymerization reaction (Chang, 2005). The student was able to identify the information needed and write the reaction according to the steps in writing the reaction, demonstrating level 1-C3 thinking skills.

Level 2-C3 thinking skills are demonstrated in the following answer:

Problem 5b: Make a stage plan that can be done by the Government to overcome the problem of plastic waste in the discourse.

Student Answer: Give tax to conventional plastic companies, then the money is used to help research bio-plastics and make plastics near corn/cassava fields so that distribution costs are lower. In addition to that, conducting counseling to the community to use bio-plastic, make sophisticated waste processing machines, and clean up trash in the Indonesian ocean.

(Concept Test Answer of Student 15)

The student demonstrates an understanding of the problem and proposes a staged plan to solve it. The student also mentions waste processing machines that are already widely used abroad; however, the student does not provide a detailed plan to overcome the problem of plastic waste in Indonesia. Answers that describe several steps or plans based on criteria or problems indicate an application of level 2-C3 thinking skills (Hess et al., 2009).

Students’ understanding at levels 1-C3 and 2-C3 are also shown in the following classroom observation and reflective journal entries:

How to identify aldehydes that are reacted with tollens will form a silver mirror

(Classroom Observation, 1 February 2019)

Based on the results of biodegradable plastic from milk that is elastic and bouncy, my future plan will be to make biodegradable plastic toys so they can be sold and produce economic value

(Reflective Journal of Student 10, 22 February 2019)

The classroom observation shows that students can apply a procedure to identify aldehydes (Tikkanen & Aksela, 2012), as per the literature on Tollens reagents containing silver (I) diamin ion or Ag(NH3)2+. Silver (I) diamin ion will oxidize aldehyde to carboxylic ion, while silver ion will be reduced to form silver deposition (silver mirror) (Solomons & Fryhle, 2011). The observation shows thinking skills at level 1-C3, meaning that the student can apply a simple procedure. The reflective journal entry shows that Student 10 can make plans based on the nature of biodegradable plastic products that result from bounce experiments. This shows that the student’s understanding is at level 2-C3, it means the student can implement steps or plans based on certain criteria.

3.2.4. Analyzing

The data show that some students reach the thinking skill level 1-C4, which illustrates in answering or solving problems, students use and identify data contained in text or graphics. Level 2-C4 illustrates that students use data processing techniques, such as classifying data based on certain criteria or comparing images with data (Hess et al., 2009). Level 1-C4 thinking skills are shown in the following answer:

Problem 2c: Based on your analysis, what causes polyisoprene cross links to form and how do these changes affect the properties of the polymer?

Student Answer: Caused by the mixing of sulfur. The sulfur forms a covalent bond with the polymer so that it forms a crosslink and becomes thermosetting

(Concept Test Answer of Student 17)

The student understands and can explain the nature of a polymer but has not explained the purpose of the sulfur addition. The literature states that the addition of sulfur to heated polyisoprene (rubber) is called vulcanization, which causes the formation of crosslinked polyisoprene (Carraher, 2003). The existence of crosslinked structures causes the polymer to be thermosetting because the crosslinked bonds hold the polymer form when heated so that when the thermosetting polymer is heated it does not melt (Brown, 2012). The student’s statement shows that they can find and use information to answer questions and can undertake a simple analysis based on information from text or images and knowledge in their long-term memory. Using information to answer or solve problems is a demonstration of level 1-C4 thinking skills (Hess et al., 2009).

Level 2-C4 thinking skills can be found in the following answer:

Problem 4a. Explain how the structure affects the polymer density!

Student Answer: Higher densities are found in linear polymers because if they are bound to one another, they will be denser such as HDPE, while branched polymers will lower density because they have branches such as LDPE

(Concept Test Answer of Student 23)

This answer shows that the student can analyze information to recognize patterns or relationships, demonstrating cognitive processes at level 2-C4 (Anderson & Krathwohl, 2001). At this level, students can identify information and use that information to answer questions. They can apply a simple analysis based on information from text or images using knowledge in their long-term memory.

The results show that integrating dilemmas stories with STEM-PBL is effective in enhancing critical thinking and in actively engaging students in learning about polymer materials. This process was a new experience for the students; they were challenged to make difficult decisions and implemented interesting solutions. The STEM approach is still not widely applied in Indonesia, but it has been proven to enhance interest in learning chemistry and develop resilience and innovation in students. STEM education encourages students to actively solve real-life problems by learning complex concepts and apply them, developed students’ thinking skills, and provide the current competitive and skills-based job market (Mater, Haj Hussein, Salha, Draidi, Shaqour, Qatanani et al., 2020).

4. Conclusions

The approach of integrating dilemmas stories with a STEM project received positive responses from students. The process of collaborating to solve the problems raised in the stories and implementing projects increased students’ skills in argumentation, critical thinking, and creativity and increases motivation. The students’ thinking skills applied to learning about polymer materials were assessed by analyzing test results using Hess’ CRM rubric. Students demonstrated four levels of Bloom’s Taxonomy, namely, C1 (remembering), C2 (understanding), C3 (applying) and C4 (analyzing). In addition, students’ depth of understanding of concepts appears at two levels of Webb’s DOK, namely, level 1 (Recall) and level 2 (Skill and concepts). The results of the analysis show that the majority of students achieved thinking skills at level 1-C2 when given five questions covering the classification of polymers, the use of polymers, the effect of structure on the properties of polymers, the evaluation of the processes and impacts of polymers, and the polymer impact management plan.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received financial support for the research, authorship, and/or publication of this article from Universitas Negeri Jakarta.

References

Adriyawati, A., Utomo, E., Rahmawati, Y., & Mardiah, A. (2020). STEAM-Project-Based Learning Integration to Improve Elementary School Students’ Scientific Literacy on Alternative Energy Learning. Universal Journal of Educational Research, 8(5), 1863-1873. https://doi.org/10.13189/ujer.2020.080523

Ahmadi, D.M.R. (2018). The Use of Technology in English Language Learning: A Literature Review. International Journal of Research in English Education, 3(2), 115-125. https://doi.org/10.29252/ijree.3.2.115

AlMaadeed, M.A.A., & Ponnamma, D. (2020). Role of Research and Higher Education on Industry 4.0, Material Science as an example. 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT) (435-439). https://doi.org/10.1109/ICIoT48696.2020.9089662

Altanis, I., Retalis, S., & Petropoulou, O. (2018). Systematic Design and Rapid Development of Motion‑Based Touchless Games for Enhancing Students’ Thinking Skills. Education Sciences, 8(1), 18. https://doi.org/10.3390/educsci8010018

Anderson, L.W., & Krathwohl, D.R. (2001). A taxonomy for learning, teaching and assessing. Pearson Education.

Aripin, S., Saing, B., & Kustiyah, E. (2017). Studi pembuatan bahan alternatif plastik biodegradable dari pati ubi jalar dengan plasticizer gliserol dengan metode melt intercalation. Jurnal Teknik Mesin, 6(2), 18. https://doi.org/10.22441/jtm.v6i2.1185

Awad, N., & Barak, M. (2018). Pre-service Science Teachers Learn a Science, Technology, Engineering and Mathematics (STEM)-Oriented Program: The Case of Sound, Waves and Communication Systems. EURASIA Journal of Mathematics, Science and Technology Education, 14(4). https://doi.org/10.29333/ejmste/83680

Blackley, S., Rahmawati, Y., Fitriani, E., Sheffield, R., & Koul, R. (2018). Using a “Makerspace” Approach to Engage Indonesian Primary Students with STEM. Issues Educational Research, 28(1), 18-42. http://www.iier.org.au/iier28/blackley.pdf

Brown, T.L. (2012). Chemistry: the central science (12th ed.). Prentice Hall.

Campbell, C., Speldewinde, C., Howitt, C., & MacDonald, A. (2018). STEM Practice in the Early Years. Creative Education, 09(01), 11-25. https://doi.org/10.4236/ce.2018.91002

Capraro, R.M. (2013). STEM Project Based Learning; An integrated science, technology, engineering, and mathematis (STEM) approach. Sense Publisher. https://doi.org/10.1007/978-94-6209-143-6

Carraher, C.E. (2003). Polymer Chemistry: Sixth Edition Revised and Expanded. Marcel Dekker.

Chang, R. (2005). Kimia Dasar: Konsep-konsep Inti Edisi Ketiga Jilid 1. Erlangga.

Cook, K.L., & Bush, S.B. (2018). Design thinking in integrated STEAM learning: Surveying the landscape and exploring exemplars in elementary grades. School Science and Mathematics, 118(3-4), 93-103. https://doi.org/10.1111/ssm.12268

Creswell, J.W. (2009). Reasearch Design; Quantitative, Qualitative and Mixed Methods Approach. SAGE Publications.

Ebnesajjad, S. (2013). Handbook of biopolymers and biodegradable plastics properties, processing and applications. Elsevier Academic Press.

EL-Deghaidy, H., Mansour, N., Alzaghibi, M., & Alhammad, K. (2017). Context of STEM Integration in Schools: Views from In-service Science Teachers. EURASIA Journal of Mathematics, Science and Technology Education, 13(6). https://doi.org/10.12973/eurasia.2017.01235a

Fanani, A., & Kusmaharti, D. (2018). Pengembangan pembelajaran berbasis HOTS (Higher Order Thinking Skill) di sekolah dasar kelas V. Jurnal Pendidikan Dasar, 1(9), 1-11.

Fleaca, E., & Stanciu, R.D. (2019). Digital-age Learning and Business Engineering Education – a Pilot Study on Students’ E-skills. Procedia Manufacturing, 32, 1051-1057. https://doi.org/10.1016/j.promfg.2019.02.320

Ganesh, K.A., Anjana, K., Hinduja, M., Sujitha, K., & Dharani, G. (2020). Review on plastic wastes in marine environment – Biodegradation and biotechnological solutions. Marine Pollution Bulletin, 150, 110733. https://doi.org/10.1016/j.marpolbul.2019.110733

Guba, E.G., & Lincoln, Y. (1989). Fourth Generation Evaluation. SAGE Publications.

Handayani, P.A., & Wijayanti, H. (2015). Pembuatan Film Plastik Biodegradeable dari Limbah Biji Durian (Durio zibethius murr). Jurnal Bahan Alam Terbarukan, 4(1), 21-26. https://doi.org/10.15294/jbat.v4i1.3770

Heriyanti. (2015). Penerapan Pendekatan Dilemmas Stories dalam Pembelajaran Kimia Berbasis Karakter pada Materi Polimer dan Karbohidrat di SMAN 10 Jakarta Kelas XII IPA 2 tahun Pembelajaran 2014/2015. FMIPA UNJ.

Hess, K.K. (2010). Applying Webb’s depth of knowledge (DOK) levels in science. National Center for Assessment.

Hess, K.K., Jones, B.S., & Walkup, J.R. (2009). Cognitive Rigor: Blending the strengths of Bloom’s taxonomy and Webb’s depth of knowledge to enhance classroom-level processes. Education Resources Information Center.

Kuo, H.C., Tseng, Y.C., & Yang, Y.T.C. (2019). Promoting college student’s learning motivation and creativity through a STEM interdisciplinary PBL human-computer interaction system design and development course. Thinking Skills and Creativity, 31, 1-10. https://doi.org/10.1016/j.tsc.2018.09.001

Laboy-Rush. (2010). Integrated STEM education through project-based learning. http://www.learning.com/stem/whitepaper/integrated-STEM-throughProject-based-Learning

Lewis, P. (2018). Globalizing the Liberal Arts: Twenty-First- Century Education. In Gleason, N.W. (Ed.), Higher Education in the Era of the Fourth Industrial Revolution (15-38). Springer Singapore. https://doi.org/10.1007/978-981-13-0194-0

Lin, K.Y., Hsiao, H.S., Williams, P.J., & Chen, Y.H. (2020). Effects of 6E-oriented STEM practical activities in cultivating middle school students’ attitudes toward technology and technological inquiry ability. Research in Science & Technological Education, 38(1), 1-18. https://doi.org/10.1080/02635143.2018.1561432

Lou, S.J., Chou, Y.C., Shih, R.C., & Chung, C.C. (2017). A Study of Creativity in CaC2 Steamship-derived STEM Project-based Learning. EURASIA Journal of Mathematics, Science and Technology Education, 13(6). https://doi.org/10.12973/eurasia.2017.01231a

Lourie, M. (2020). Recontextualising Twenty-first Century Learning in New Zealand Education Policy: The Reframing of Knowledge, Skills and Competencies. New Zealand Journal of Educational Studies, 55(1), 113-128. https://doi.org/10.1007/s40841-020-00158-0

Mater, N.R., Haj Hussein, M.J., Salha, S.H., Draidi, F.R., Shaqour, A.Z., Qatanani, N., et al. (2020). The effect of the integration of STEM on critical thinking and technology acceptance model. Educational Studies (1-17). https://doi.org/10.1080/03055698.2020.1793736

Miles, M., & Huberman, A. (1994). Qualitative Analysis: An Expanded Source Book (2nd ed.). SAGE.

Ministry of Education and Culture (2017). Penilaian untuk pembelajaran abad 21 belajar dari berbagai hasil penilaian. Ministry of Education and Culture.

Ministry of Education and Culture (2018). Modul pelathan implementasi kurikulum kimia 2013 SMA tahun 2018. Directorate General of Primary and Secondary Education.

Morais, C. (2015). Storytelling with Chemistry and Related Hands-On Activities: Informal Learning Experiences To Prevent “Chemophobia” and Promote Young Children’s Scientific Literacy. Journal of Chemical Education, 92(1), 58-65. https://doi.org/10.1021/ed5002416

Nursa’adah, E., Kurniawati, D., & Yunita. (2016). Analisis kemampuan kognitif mahasiswa pada konsep asam basa menggunakan tes berdasarkan taksonomi bloom revisi. EduChemia (Jurnal Kimia Dan Pendidikan), 2(25-35).

Pacis, M., & VanWynsberghe, R. (2020). Key sustainability competencies for education for sustainability. International Journal of Sustainability in Higher Education, 21(3), 575-592.
https://doi.org/10.1108/IJSHE-12-2018-0234

Qadir, J., Yau, K.L.A., Ali Imran, M., & Al-Fuqaha, A. (2020). Engineering Education, Moving into 2020s : Essential Competencies for Effective 21st Century Electrical & Computer Engineers. 2020 IEEE Frontiers in Education Conference (FIE) (1-9). https://doi.org/10.1109/FIE44824.2020.9274067

Rahmawati, Y., Ridwan, A., Hadinugrahaningsih, T., & Soeprijanto (2019). Developing critical and creative thinking skills through STEAM integration in chemistry learning. Journal of Physics: Conference Series, 1156, 012033. https://doi.org/10.1088/1742-6596/1156/1/012033

Rahmawati, Y., Ridwan, A., Mardiah, A., & Afrizal. (2020). Students’ chemical literacy development through STEAM integrated with dilemmas stories on acid and base topics. Journal of Physics: Conference Series, 1521, 042076. https://doi.org/10.1088/1742-6596/1521/4/042076

Rodríguez, G., Díez, J., Pérez, N., Baños, J.E., & Carrió, M. (2019). Flipped classroom: Fostering creative skills in undergraduate students of health sciences. Thinking Skills and Creativity, 33, 100575. https://doi.org/10.1016/j.tsc.2019.100575

Roohi, Bano, K., Kuddus, M., Zaheer, M.R., Zia, Q., Khan, M.F., et al. (2017). Microbial Enzymatic Degradation of Biodegradable Plastics. Current Pharmaceutical Biotechnology, 18(5). https://doi.org/10.2174/1389201018666170523165742

Rosenzweig, E.Q., & Wigfield, A. (2016). STEM Motivation Interventions for Adolescents: A Promising Start, but Further to Go. Educational Psychologist, 51(2), 146-163. https://doi.org/10.1080/00461520.2016.1154792

Settelmaier, E. (2004). Dilemmas with Dilemmas…exploring the suitability of dilemmas stories as a way of addressing ethical issues in science education. Presented at the Annual Conference of the Australian Association for Research in Education.

Sima, V., Gheorghe, I.G., Subić, J., & Nancu, D. (2020). Influences of the Industry 4.0 Revolution on the Human Capital Development and Consumer Behavior: A Systematic Review. Sustainability, 12(10), 4035. https://doi.org/10.3390/su12104035

Solomons, T.W., & Fryhle, C.B. (2011). Organic Chemistry (10th ed.). John Wiley & Sons.

Sukasni, A., & Efendy, H. (2017). The Problematic of Education System in Indonesia and Reform Agenda. International Journal of Education, 9(3), 183. https://doi.org/10.5296/ije.v9i3.11705

Taylor, E., Taylor, P.C., & Chow, M. (2013). Diverse, disengage, and reactive: A teacher’s adaptation of ethical dilemma story pedagogy as a strategy to re-angage learners in education for sustainability. In Monsour, N., & Wegerif, R. (Eds.), Science Education for Diversity. Cultural Studies of Sciense Education, 97‑117. Springer. https://doi.org/10.1007/978-94-007-4563-6_6

Taylor, E., Taylor, P.C., & Hill, J. (2019). Ethical Dilemma Story Pedagogy – A Constructivist Approach to Values Learning and Ethical Understanding. Proceeding of the Science and Mathematics International Conference (1-6). https://doi.org/10.1201/9780429461903-18

Tikkanen, G., & Aksela, M. (2012). Analysis of Finnish chemistry Matriculation Examination questions according to Cognitive Complexity. Nordic Studies in Science Education, 8(3), 257-268. https://doi.org/10.5617/nordina.532

Tseng, K.H., Chang, C.C., Lou, S.J., & Chen, W.P. (2013). Attitudes towards science, technology, engineering and mathematics (STEM) in a project-based learning (PjBL) environment. International Journal of Technology and Design Education, 23(1), 87-102. https://doi.org/10.1007/s10798-011-9160-x

van Laar, E., van Deursen, A.J.A.M., van Dijk, J.A.G. M., & de Haan, J. (2017). The relation between 21st‑century skills and digital skills: A systematic literature review. Computers in Human Behavior, 72, 577‑588. https://doi.org/10.1016/j.chb.2017.03.010

Wan, Z.H., So, W.M.W., & Zhan, Y. (2020). Developing and Validating a Scale of STEM Project-Based Learning Experience. Research in Science Education. https://doi.org/10.1007/s11165-020-09965-3

Werth, J.C. (2017). Teaching a Dilemmas story in my science classroom enabling students to make ethically aware decisions. Curtin University.

Ziaeefard, S., Miller, M.H., Rastgaar, M., & Mahmoudian, N. (2017). Co-robotics hands-on activities: A gateway to engineering design and STEM learning. Robotics and Autonomous Systems, 97, 40-50. https://doi.org/10.1016/j.robot.2017.07.013

---