The integration of dilemmas stories with stem-project-based learning: analyzing students’ thinking skills using hess’ cognitive rigor matrix


Yuli Rahmawati1 , Afrizal Afrizal1 , Devina Dwi Astari1 ,
Alin Mardiah
1 , Dyah Budi Utami2 , Sukro Muhab1

1Universitas Negeri Jakarta (Indonesia)

2SMAN 4 Tambun Selatan (Indonesia)

Received April 2021

Accepted June 2021


This study aimed to analyze students’ thinking skills through integrating dilemmas stories with a Science, Technology, Engineering, and Mathematics-Project-Based Learning (STEM-PBL) in polymers topic. The participants were 47 Grade 12 students from a public senior high school in West Java Province. The research employed a qualitative method to obtain data through classroom observations, reflective journals, interviews, and a concept test. Students’ thinking skills were assessed using Hess’ Cognitive Rigor Matrix, which combines Bloom’s taxonomic and Webb’s depth of knowledge. The results show that most students reach level C2 of Bloom’s taxonomy and level 1 of Webb’s depth of knowledge, meaning that most students understand and recall the information. The results indicate that integrating dilemmas stories with STEM-PBL enables students to explain using simple sentences and prior knowledge. In addition, this approach provides an opportunity for students to develop their critical thinking, creativity, and argumentation skills through problem-solving and project making.


Keywords – Polymer learning, Thinking skills, Hess’ cognitive rigor matrix, Dilemmas stories, STEM, Polymer.

To cite this article:

Rahmawati, Y., Afrizal, A., Dwi Astari, D., Budi Utami, D., & Muhab, S. (2021). The integration of dilemmas stories with stem-project-based learning: Analyzing students’ thinking skills using hess’ cognitive rigor matrix. Journal of Technology and Science Education, 11(2), 419-439.



    1. 1. Introduction

Educational institutions have responded to the rapid increase of science and technology by focusing on improving students’ thinking skills to ensure they can compete globally in the 21st century. The shift of the industrial economy toward an era of information technology is forcing the education system to prepare graduates with the necessary skills and competencies (AlMaadeed & Ponnamma, 2020; Lewis, 2018; Sima, Gheorghe, Subić & Nancu, 2020). Basic knowledge and skills are no longer enough; students must have the future skills such as information and data processing skills (Fleaca & Stanciu, 2019), creative and critical thinking (Qadir, Yau, Ali Imran & Al-Fuqaha, 2020; Rodríguez, Díez, Pérez, Baños & Carrió, 2019), and thinking skills such as system thinking and integrating traditional thinking to future thinking (Pacis & VanWynsberghe, 2020). Therefore, the main objective of education to develop and improve students’ high-level thinking skills (Altanis, Retalis & Petropoulou, 2018) and deeper levels of learning (Lourie, 2020). Thus, across boundaries learning as contextual learning needs to be carried out in order to students do not only recall the information but also put conceptual understanding into context (Lourie, 2020).

In their study on learning model to develop students’ higher-order thinking skills, Fanani and Kusmaharti (2018) state that many Indonesian students have a low ability to understand complex information, theorize, analyze, solve problems, use tools, follow procedures, and conduct investigations. In 2013, The Government of Indonesia sought to improve students’ thinking skills by replacing the Kurikulum Tingkat Satuan Pendidikan (KTSP) curriculum that had been in place since 2006. The 2013 curriculum was designed to develop students’ character traits and 21st-century skills, such as critical thinking and problem‑solving, collaboration, creativity, and communication (Ministry of Education and Culture, 2018). However, Indonesia’s national exam results of 2015 indicated low levels of thinking skills among Indonesian students in areas such as reasoning, analyzing, and evaluating (Ministry of Education and Culture, 2017). In addition, the demand for mastering sufficiently dense learning material is also a problem in the Indonesian education system (Sukasni & Efendy, 2017). Based on the results of teacher interviews conducted at State Senior High School (SMAN) 4 Tambun Selatan, West Java, researchers found that the development of higher-order thinking skills was not a top priority due to competing demands; the students are required to master 15 subjects and prepare for the national exam. The national exam is a requirement of the Indonesian education system, and is a key focus, particularly for Grade 12 students (usually aged 17-18); studying national exam questions takes precedence over the development of students’ thinking skills in mastering and applying concepts to solve problems in daily life.

Students’ thinking skills can be developed through different approaches to the learning process, such as the use of dilemmas stories. Based on a previous study, dilemmas stories provide students the opportunity to make decisions and solve problems (Taylor, Taylor & Chow, 2013). The dilemmas approach can spur students to think critically, provide critical judgments on various arguments, and transform their perspectives; it is part of transformative learning (Settelmaier, 2004). Dilemmas stories can be applied to science learning, especially when discussing socio-cultural issues related to sustainable development (Taylor, Taylor & Hill, 2019).

Another contextual approach to learning is project-based Science, Technology, Engineering, and Mathematics (STEM). STEM integrated PBL can increase students’ interest in learning (Campbell, Speldewinde, Howitt & MacDonald, 2018; Wan, So & Zhan, 2020; Ziaeefard, Miller, Rastgaar & Mahmoudian, 2017), provide meaningful learning experiences (Awad & Barak, 2018; Campbell et al., 2018; EL-Deghaidy, Mansour, Alzaghibi & Alhammad, 2017), enhance students’ creativity and critical thinking (Blackley, Rahmawati, Fitriani, Sheffield & Koul, 2018; Rahmawati, Ridwan, Hadinugrahaningsih & Soeprijanto, 2019; Rahmawati, Ridwan, Mardiah & Afrizal, 2020), and give more chance for them to apply STEM knowledge to solve real-life problems (Cook & Bush, 2018; Lin, Hsiao, Williams & Chen, 2020). In addition, PBL provides challenges and motivates students to think critically and analytically, which improves their higher-order thinking skills (Capraro, 2013).

This study seeks to determine whether integrating dilemmas stories with STEM project-based learning (STEM-PBL) is effective in developing students’ thinking skills in chemistry learning.

2. Methodology

The research method used is a qualitative method. Qualitative research methods are based on text or image analysis, have certain steps in data analysis, and various research strategies. Qualitative research uses all perspectives in analyzing research data (Creswell, 2009).

2.1. Research Design

The research conducted with a group of 47 students made up of 10 male and 30 female year 12 secondary students in West Java, Indonesia with polymer as the topic of chemistry learning. Polymers are one of the topics in chemistry learning that have wide applications in everyday life. Polymer learning aims to increase students’ insight and awareness about the problems of polymer such as plastics waste. Participants were selected based on the suitability of the teaching material to the class level and the desire to participate in the research. Data collection was carried out through various techniques, such as interviews, reflective journals, observation, STEM project assessments, and open-ended test questions on polymer topics.

The research was conducted in three stages, namely the preliminary stage, implementation, and the final stage. The research flow is shown in Figure 1.

At the preliminary stage, the researcher compiled an interview protocol for teacher and students to obtain information related to chemistry learning in school and students’ thinking skills. At this stage, the researchers also compiled a lesson plan using STEM-PBL, developed dilemmas stories about biodegradable plastics, and compiling research instruments such as observation sheets, reflective journals, students’ worksheets, the preparation and validation of open-ended polymer test questions.

The research implementation was carried out through the STEM-PBL learning stages developed by Laboy-Rush (2010) and integrated dilemmas stories into it. The five learning stages are described in Figure 2.

In this study, students are given the opportunity to make two products that have differences in the design procedure and the level of reagents (glycerin, acetic acid, and chitosan) so they can analyze the function of reagents and determine the appropriate product design to get maximum results. During the lesson, students are given worksheets and reflective journals, while the teacher, assisted by two observers, made observations to analyze the effectiveness of integrating dilemma stories in the STEM project and their impact on the development of students’ thinking skills.


Figure 1. Research Flow and Data Collections


Figure 2. The five stages of integrating dilemmas stories with STEM project

At the final stage, students are given open-ended tests to assess their understanding on polymer topic and their thinking skills. Researchers conducted the interviews to get the information about students’ perspective on the integration of dilemmas stories on STEM project and students’ thinking skills development. Then researchers analyzed the research data using Hess’s Cognitive Rigor Matrix which describes the attainment of Bloom’s taxonomy level and Webb’s DOK level (Webb’s depth of knowledge). The Bloom Taxonomy level explains the level of students’ cognitive processes and Webb’s DOK level explains the depth or complexity of students’ understanding of concepts to answer a question (Hess, Jones & Walkup, 2009).

2.2. Data Collection

The research data were obtained using multiple collection such as interviews, reflective journals, observation, STEM project assessments, and open-ended test questions on polymer topics.

2.2.1. Interviews

Interviews were conducted with teachers and students. The teacher interview aimed to find out how teacher usually carried out polymer learning, the problems that occur in chemistry learning, and to find out students’ thinking skills. The student interviews were conducted with 9 people based on the results of the analysis of reflective journals and student involvement in learning. Student interviews aimed to collect information about students’ impressions of the learning that has been carried out and observe the development of students’ thinking skills by asking questions related to the learning process such as their communication and collaboration in groups to solve problems in dilemmas stories through the development of a STEM project. The following is an example of interview question asked of students.

What materials does your group use as a base for making biodegradable plastics? What is the reason?

2.2.2. Reflective Journals

The students completed reflective journals during the learning process, precisely at the end of each meeting. Reflective journal is used to collect data related to students’ reflections on the level of involvement and difficulties encountered during the application of dilemmas stories in the STEM project on polymer learning. The example of questions in a reflective journal is:

What do you think about today’s learning in relation to thinking skills development?
Write your reflection in the column provided

2.2.3. Observations Sheet

Observations were made to understand the conditions of learning in the classroom and the development of students’ thinking skills. Observations by researchers assisted by two observers were carried out during the learning process to ensure that the application of learning was by the STEM-PBL learning steps by Laboy‑Rush (2010), students’ thinking skills, conceptual understanding, and students’ involvement in learning.

2.2.4. Polymer Open-Ended Test

The open-ended test aimed to determine students’ thinking skills as an implication of integrating the dilemmas stories in the STEM project. This test consisted of five essay with 14 total item questions compiled by researchers based on polymer learning indicators in the 2013 curriculum namely analyzing the structure, nomenclature, properties and classifications of macromolecules (such as polymer) and reasoning about the manufacture of a product from macromolecules. The distribution of items based on indicators that have been compiled by researchers can be observed through the following Table 1.

In this study, open-ended questions were validated by 5 experts consisting of 4 Chemistry Lecturers and Chemistry Education Lecturers at Universitas Negeri Jakarta and 1 high school chemistry teacher in grade 12. The test was conducted at the end of the polymer topic lesson. The following is one of the questions being tested.



Question Item Number


Describing the monomer structure of a polymer



Writing the names and polymerization reactions (addition and condensation) of the monomers



Differentiating the classification of polymers on the basis of origin, type of constituent, polymerization reactions and properties to heat



Explaining the use of synthetic polymers in everyday life



Analyzing polymer properties based on the structure and the constituent monomers

1b, 2c, 3a, 4a, 4b


Evaluating the process and impact of polymer formation

3b, 3c, 5a


Creating a polymer impact mitigation plan

5b, 5c

Table 1. Distribution of Polymer Concept Test Question Based on Learning Indicators

Natural rubber comes from rubber tree sap which chemically contains isoprene polymer.


Natural rubber is a liquid, stiff when heated and sticky when cold. In 1839 Charles Goodyear conducted experiments on natural rubber which was heated and added with sulfur to find the Rubber Tires we use today. Following is the structure of the polymer rubber tires:


a) Write the polymerization reaction of isoprene to polyisoprene. b) Are the two polymers the same based on their classification? Explain your reasons! And c) Based on your analysis, explain the purpose of adding sulfur to the experiment!

2.3. Data Analysis

Qualitative data analysis techniques are carried out through the stages of data reduction, display data, and conclusion drawing/verification (Miles & Huberman, 1994). The data obtained during the study through interviews, observations, reflective journals, and polymer open-ended tests were reduced by selecting data that was under the focus of students’ thinking skills. At the data display stage, the data that has been reduced is then presented in the form of a matrix or coding based on the Hess’ Cognitive Rigor Matrix indicator so that data verification can be carried out and conclusions can be drawn.

To draw the conclusion and verified the data, credibility criteria are used including prolonged engagement, persistent observation, progressive subjectivity, and member checking (Miles & Huberman, 1994) to test the validity and trustworthiness of the data (Guba & Lincoln, 1989). The prolonged engagement was carried out during the study (December 2018 - July 2019) to understand the context of the study, explore incidents to explain in detail the research situation and participant involvement, and integrated dilemmas stories in the STEM project for 5 weeks in polymer learning. Persistent observation is carried out to explore various phenomena by involving researchers directly in the learning process. In making continuous and in-depth observations, the researcher was assisted by two observers to analyze the learning process and the development of students’ thinking skills. Progressive subjectivity is carried out to monitor research results with all notes obtained during the study based on the characteristics of students’ thinking skills. Member checking is done to ensure the accuracy of the data obtained from students as data providers regarding the various research results. All data obtained, both in the form of hard paper copies and interview transcripts, were kept privately by the researcher for the privacy of participants and confidentiality.

3. Results and Discussion

3.1. The Integration of Dilemmas Stories with a STEM Project in Chemistry Learning

The dilemmas stories highlight a problem to be solved using a STEM project (Laboy-Rush, 2010) in five stages as referred before.

3.1.1. Reflection

In the first stage, students are given initial questions about the concept being studied and the problems raised in the dilemmas stories read by the teacher. The dilemmas of plastic are raised in the stories because plastic is one example of polymers that are widely used and easily found in everyday life. Plastic is flexible, elastic, strong, and durable and is ubiquitous in today’s society, commonly used for food and beverage packaging, household appliances, and office equipment. People use plastic because it is cheap, practical, strong, and easy to obtain; however, the demand for plastic causes environmental problems because it is not easily biodegradable (Ganesh, Anjana, Hinduja, Sujitha & Dharani, 2020). According to statistical data, 14% (5.4 million tons/year) of waste production in Indonesia is plastic (Handayani & Wijayanti, 2015). Synthetic polymers are widely used and they are non-biodegradable; therefore, the environmental implications of their use must be considered (Aripin, Saing & Kustiyah, 2017). Therefore, the topic of plastics is raised in the stories to increase students’ interest and encourage them to solve plastic problems based on the polymer concepts that have been studied. The dilemmas stories share experiences and information providing various perceptions and perspectives so students can think according to their cultural background, increase their self-awareness, and combine thoughts, emotions, and actions (Morais, 2015). Table 2 provides a summary of the issues and descriptions raised in the dilemmas stories.

The plastic dilemmas stories are accompanied by questions that seek to involve the students in overcoming the dilemmas presented. The students work individually at this stage to encourage critical thinking, enhance their motivation to be actively involved in learning, and increase their curiousity (Adriyawati, Utomo, Rahmawati & Mardiah, 2020). Students then discuss their answers in groups and collaborate to agree on a single solution for the group to progress into the next stage.





The danger of using plastic bags on health

The story tells of a mother and child who sell cakes using black plastic for packaging. The mother and child also often re-use plastic soft drink bottles as drinking containers. The child is a student in high school and, while learning the various types of polymer material, learns of the dangers and suggested uses of polymers. The child realizes the health impacts of using plastic and begins to worry about the mother’s health; the mother is thought to have suffered from uterine cancer so her health is impaired. The child explains the issues to the mother and asks the mother to use an alternative to plastic, but the mother feels worried that it will affect their income.


Impact of the policy on the prohibition of using plastics

The story tells of a father who has worked in a factory for 25 years to support his wife and five children. The story is told from the perspective of one of the children who is a high school student. The child watches the news on TV about a dead whale stranded on Wakatobi Island and found with plastic waste in its stomach. Shortly after the news, political parties begin supporting the reduction and even the prohibition of the use of plastic in some areas. The Government of Jakarta is in the process of ratifying the ban on the use of plastic in the capital. The child’s friend at school explains that plastic is bad for the environment and living things; however, the child is worried about his father’s work. The father must continue to work to support his family. and is of an advanced age, which makes it difficult to find other jobs.

Table 2. Summary of the Plastic Dilemmas Stories

This is an example of student’s perception of the integration of dilemmas stories on the learning process: Problems in a plastic dilemma require problem solving that are not easy, it is necessary to consider both sides which are equally difficult to solve. But it is very effective in expressing the opinions and creative side of students.

(Interview of Student 17, 22 February 2019)

The interview excerpt shows that solving problems can hone students’ creative and critical thinking skills. Creative thinking is demonstrated by the students agreeing on a solution when different proposals are presented. Critical thinking is shown when students consider the dilemmas, understand the negative and positive impacts on different actors in the stories, and consider solutions based on their values. The use of dilemmas stories in the classroom shows that students can reflect their values on assumptions and decisions and improve their ability to think critically and apply science (Werth, 2017).

3.1.2. Research

At this stage, the students research solutions to the problem and plan their biodegradable plastic project. At first, students individually look for information about solutions to the problems presented in the dilemmas stories. The students then discuss their findings in groups (see Figure 3). The students are expected to exchange information and argue their proposal effectively with members of their group. It is also expected that the students can agree on the best solution to the problem through group discussion.