Journal of Technology and Science Education

• Home
• About
• Login
• Archives
• Submissions
• Publication Fee
• Indexing & Statistics

Home > Vol 14, No 3 (2024) > Hernàndez-Sabaté

FROM TRADITIONAL TEACHING TO FLIPPED CLASSROOM: IMPACT ON LEARNING IN ENGINEERING DEGREES

1Universitat Autònoma de Barcelona (Spain)

2Computer Vision Center (Spain)

Received December 2023

Accepted April 2024

Abstract

Computer engineering students should develop competences related to the contents of databases design and SQL queries. For this purpose, the recommendations on the convenience of changing the traditional teaching methodology to the flipped classroom are followed. In this article we present a quantitative study in which we compare the potential for the development of engineering students’ competences in the design and use of databases of the flipped classroom methodology and the traditional teaching methodology. The results obtained in the evaluation of the subject in two different courses are compared. In the first course, traditional teaching methodology was used. In the second one, flipped classroom was used when its implementation had already been tried and tested. In this article we show evidence that the implementation of the flipped classroom teaching methodology provides different results depending on the learning promoted and on the specific contents in the subject of relational databases. We have found evidence that flipped classroom improve theoretical learning outcomes for database design and in the resolution of non-reproductive activities in the SQL queries block. No conclusive benefits are reported for students’ competency development for either of the two content blocks.

 

Keywords – Computer science, Databases, Traditional teaching, Flipped classroom, Competences development, Higher education.

To cite this article:

Hernàndez-Sabaté, A., Albarracín, L., Ramos, O., Gil, D., Sánchez, C., & Martí, E. (2024). From traditional teaching to flipped classroom: impact on learning in engineering degrees. Journal of Technology and Science Education, 14(3), 798-814. https://doi.org/10.3926/jotse.2623

 

----------

1. 1. 1. Introduction

University programs in the European Higher Education Area (EHEA) are designed to promote the development of knowledge, skills, and competences to individuals and society (OECD, 2002). Since its introduction, the convergence of European university degree’s structure and learning goals promoted by the EHEA has brought challenges at different levels (Rychen & Salganik, 2003). Designing teaching methodologies that promote the development of competences in engineering studies is more necessary than ever. A competence is defined as the instance where a complex reality requires a learning at different levels (conceptual, procedural, and attitudinal) obtained from the universe of knowledge, capabilities and skills for the understanding and transformation of this reality (Mulder, 2014). Competences development in the individual requires, not only the ability to globally manage them, but also a certain degree of conjunction with attitudes and personal values. Thus, a competence is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilizing psycho-social resources (including skills, attitudes, and responsibilities), in a particular context and producing the desired result (OECD, 2005; ONU, 2015; Yániz, 2006).

The proposal of redirect teaching to promote the development of competences focuses on the student as the promoter of his or her own learning. In this study we use as a foundation the updated Bloom’s taxonomy, which will be used as a theoretical reference to characterize student learning in the direction of measuring competence development (Anderson & Krathwohl, 2001). Bloom’s taxonomy has already been used to establish differentiated levels in evaluation questions in the field of engineering (Swart, 2009). From this proposal we classify specific competences at three levels: a first level that affects the acquisition of knowledge from a conceptual point of view; a second one in which this knowledge is coordinated, integrated, and put into value in engineering practice; and a third level that allows the development and evaluation of new designs or products. However, still most of university teaching in the field of engineering is focused on the transmission of knowledge. Teaching is scheduled and delivered following the classical methodology: theory classes, seminars/problems and practical lessons. This methodological structure may not be optimal for a complete achievement of competences since, among others, the teacher directs the learning strategy leading students to act only as receivers of knowledge (Freeman, Eddy, McDonough, Smith, Okoroafor, Jordt et al., 2014). Furthermore, in large classroom groups, the pace of learning is very different among students. This means that, often, in problem and practical lessons, students are still assimilating the concepts given in the theory classes and are therefore unable to go deeper into the subject. A group of teachers from the Universitat Autònoma de Barcelona, teaching subjects related to databases in engineering studies identified that these drawbacks also promote low participation of students in the classroom, including absence of assistance and, thus, low academic performance. In consequence, a change in teaching methodology was necessary to overcome these inconveniences and lead the students to their competence development in a proactive way.

In recent years, student-centered teaching methodologies have arisen in the field of engineering where specific didactic resources are introduced and the student is the protagonist of his/her learning process, evidencing an improvement in academic results (Lage, Platt & Treglia, 2000; Hernández-Sabaté, Albarracín & Sánchez, 2020; Gren, 2020; Karabulut-Ilgu, Jaramillo-Cherrez & Jahren, 2018; O’Grady, 2012; Freeman et al., 2014). Among these methodologies, flipped classroom allows to apply active learning where the development of competences by students is naturally promoted (Pluta, Richards & Mutnick, 2013). In Flipped classroom, the focus of the face-to-face sessions is inverted, leaving the students to learn the theoretical concepts on their own and making room to the teacher during the face-to-face sessions to help students consolidate and place value on the learned knowledge (Cheng, Ritzhaupt & Antonenko, 2019; Cheng, Liu, Huang & Shyr, 2019; Keengwe, Onchwari & Agamba, 2014; Öncel & Kara, 2019). Under this teaching paradigm, the teacher provides the course materials before class (articles, book chapters, informative or pre-recorded lecture videos…), so that face-to-face contact is devoted to collaborative group discussions and problem solving (Mok, 2014; Tsai & Wu, 2020). This pre-classroom activity aims to prepare students for in-class work that focuses on active learning approaches such as problem solving to help students better understand the subject matter (Fulton, 2012; Davies, Dean & Ball, 2013). Research shows many benefits of the flipped learning approach in engineering education domains, such as increased learning outcomes (Jang & Kim, 2020), positive changes in students’ attitude towards the subject matter (Kang, 2015) and students’ engagement with their learning (Jang & Kim, 2020), improved teacher-student and student-student interaction (Della-Ratta, 2015), and problem solving and metacognition (Chun & Lee, 2016; Van Vliet, Winnips & Brouwer, 2015). However, recent studies and literature reviews show that Flipped classroom methodology challenges teachers by significantly increasing their teaching dedication, preparing videos and teaching materials of a different nature than usual, and changing their role in the classroom (Cheng, Ritzhaupt et al., 2019; Altaii, Reagle & Handley, 2017; McLaughlin, Roth, Glatt, Gharkholonarehe, Davidson, Griffin et al., 2014). But this is also a challenge for students, who must prepare for classes by engaging in prior activities (Diwanji, Hinkelmann & Witschel, 2018; Palmer, 2015; Sahin, Cavlazoglu & Zeytuncu, 2015). Given that Flipped Classroom redirects the requirements of students’ self-regulation in terms of autonomous work, the engagement of effective pre-class learning is vital to the success of flipped learning, since the model assumes that students have prepared for subsequent lessons during class by completing all assigned pre-work (Gillette, Rudolph, Kimble, Rockich-Winston, Smith & Broedel‑Zaugg, 2018). Therefore, if students assist to the class unprepared the teacher will waste valuable class time reviewing material that was already addressed through pre-class work. In addition, ill-prepared students would gain little or no benefit from active learning activities during class.

This paper analyses the impact of Flipped classroom in subjects of relational databases in engineering degrees, in contrast to the challenges posed in the state-of-the-art. In particular, the study presented in this article is part of a teaching quality improvement project initiated in the 18/19 academic year. The project purpose is to introduce active teaching dynamics by the student within a conducive environment, using the Flipped classroom methodology, to improve students’ competence in the field of engineering and on the specific contents on relational databases (Hernández-Sabate, Albarracín, Gil, Ramos, Sánchez, Valveny et al. 2021; Ramos, Albarracín, Martí, Hernandez-Sabaté & Gil, 2021). During the implementation of the changes, we observed some difficulties in adapting the activities related to the contents to the Flipped classroom dynamics. This fact led us to the need to evaluate the impact on the type of learning of students who worked under the Flipped classroom paradigm and compare them with the achievements of students in previous courses who worked under the traditional teaching paradigm. Our research objective is to determine for which content learning and competence development on the design and use of databases in a computer science subject is improved by using Flipped classroom compared to traditional instruction.

2. Research Methodology

A quantitative study is developed comparing the learning achieved by two different promotions of the same engineering studies in the subject of Databases in the 18/19 and 21/22 courses. The 18/19 course is the last one in which these studies were taught in the traditional format. In the 19/20 course, the flipped classroom methodology was introduced, just the semester before the pandemics of covid-19, but as it was the first approach to this type of methodology by the faculty, some difficulties were identified. Specifically, it was observed that a high percentage of students attended classes without having completed the previous tasks (readings, videos) and the teachers reacted by reviewing the previous materials in the classroom; or that the time needed to solve the exercises and problems increased significantly, affecting the planned teaching program (Ramos et al., 2021). The course 20/21 had many teaching constraints due to the restrictions on classroom attendance, but the flipped methodology continued being implemented and improved. The course 21/22 is considered the first one in which the Flipped Classroom methodology was adequately applied and once all teaching difficulties have been identified and overcome. Therefore, comparing student learning outcomes in terms of database content competences makes sense for years 18/19 (the last year in which the traditional methodology is implemented) and year 21/22 (the first year in which the Flipped methodology is properly applied after a period of experimentation and improvements in implementation).

The following subsections describe the research context and the methodological choices made in the study. Firstly, the main features of the two types of teaching interventions compared are explained, and secondly, the characteristics of the data collection are detailed.

2.1. Learning Contents

The contents worked in these subjects correspond to the ones are those of a subject to be treated in the subject of Databases in degrees such as Informatics Engineering, Aeronautics Engineering, Bussiness and Technology, Bioinformatics, Data Engineering, Computational Mathematics. In the field of engineering, these contents usually have two different blocks: database design and database queries, as defined in ANECA (2004).

On database design, the basic concepts of database systems and their architecture are presented. Real examples of use are used so that students can appreciate first-hand the need to implement a good model‑based database design in the basic entity-relationship model, and the phases of database design are worked on. The next step is to work on the conversion from the entity-relational model to the relational model to work on the classic concepts that every database designer should know and understand.

Regarding database queries, the formal languages for querying and generating answers on databases are introduced: relational algebra and relational calculus (SQL). The different relational algebra operators and SQL syntax are covered, from the simplest ones to subqueries or complex queries. In particular, the aggregation functions and the grouping operator are introduced. This work anticipates complex queries in which results obtained from calculations are combined with aggregation functions. Subsequently, the queries to be solved with the set operators: union, difference and intersection are worked on. In addition, the relational operator of division in relational algebra and the way to solve this type of queries in SQL are introduced.

2.2. Traditional Lecture Classroom

The classic structure of the face-to-face sessions in which the subject of Relational Databases was initially divided establishes 2 hours per week of theory, 1 hour per week of problem classes and 1 hour per week of practical classes. In the theory classes, lectures are given by the teacher with ICT support to groups of 80-100 students. The teacher chooses the theoretical contents to be covered and explains them to the students. Students can ask questions about the explanations and the teacher answers to them. Problem sessions are devoted to solving problems to a group of about 40-50 students. These problems are proposed by the teaching team and, during the session in the classroom, the teacher poses and solves them. The student can participate in their resolution by asking questions and/or answering questions asked by the teacher. In some cases, the teacher lets some minutes to the students to think about the problem posed and then (s)he solves it. Finally, practical sessions correspond to guided sessions based on the performance of practical work done by the student. Usually, these sessions are linked to the resolution of a larger practical project. The groups in these sessions are of about 20 students and are carried out in the integrated laboratories, with computers for every two students.

Besides, homework can be categorized in two levels. On the one hand, after problem sessions, the teacher proposes some exercises to solve at home. These exercises must be submitted by the student some days later. On the other hand, there is a practical project that is posed at the beginning of the course and students must solve in pairs during most of the course. There are two milestones where students must submit the work done so far.

2.3. Flipped Classroom

Flipped classroom sessions are also divided into work outside the classroom and work in the classroom. According to this methodology, it is before the classroom session that the student must carry out a previous work of study of the contents, so that in the classroom the teacher can focus on solving the doubts of the students in a personalized way (Lage et al., 2000). Therefore, we have divided our flipped classroom methodology into the following stages:

Preliminary work. The student must watch a video or read documentation related to the concepts that will be worked on during the classroom session. The contents of the material must be accessible to the students so that they can understand it autonomously. They should also give way to the more advanced concepts that will be worked on during the session. To encourage the students to do the work, a series of on-line questionnaires can be proposed that will be closed before the classroom session.

Classroom session. These are 2-hour sessions organized as explained below. During the first 10-15 minutes the teacher summarizes the contents to be worked on during the session. Then, students work in groups on a set of guided exercises designed following a constructivist model of knowledge (Ortiz, 2015). Each exercise is divided into relatively simple but incrementally difficult subtasks that students can solve autonomously. Students need to solve all the sections (subtasks) to build the final solution. For those contents that we know from experience in previous courses that students have greater difficulties or are accustomed to making the same errors, typical errors are forced to appear to promote a proper understanding of the concepts.

The initial enrolment group (80 students) has been divided into 2 groups and, for each of them, the same number of hours foreseen for the classical methodology are carried out. The changes in teaching methodology to allow the introduction of the inverse class have required institutional changes in terms of infrastructure and academic organization. At the infrastructure level, it must be considered that in most sessions students need one computer per work pair, so it is necessary to adapt the classrooms to this reality. At the level of academic organization, the weekly scheduling of the subjects concerned no longer makes any distinction between theory, problems, and practice. Table 1 shows the changes in the structuring of classes.

Theory	Problems	Practices		Without category
80 students	40 students	20 students	→	40 students
		20 students
	40 students	20 students	→	40 students
		20 students

Table 1. Sketch of the changes in classes structuring

2.4. Transformations Induced by Methodology Change

The change in teaching methodology has brought about changes at different levels. Firstly, it is important to note that we have not changed the typology of the assessment tests. The design of the exams comes from a previous teaching innovation project in which they were adapted to the needs required by the EHEA and we consider that maintaining their structure provides a clear guide to the objectives to be achieved, both for teaching staff and students.

Figure 1. Screenshot of an introductory video

However, other aspects have been modified. It has been necessary to record introductory videos for the key contents of each session. These videos are approximately 10-15 minutes long and the teaching materials (presentations) already used in previous courses (traditional methodology) were used. Figure 1 shows a screenshot of an introductory video, hung in the main channel of YouTube of the School of Engineering of Universitat Autònoma de Barcelona. Regarding the classroom activities, the fact of having more effective time in class has made it possible to scale the activities in the form of scaffolding (Melero, Leo & Blat, 2012; MacLeod & van der Veen, 2020), with the aim of improving the activities by adding sub-activities that allow students to build their knowledge.

Below is an example of an activity used in the course. This is a database design exercise. To support decision-making in database design, students are provided with an example of a database already built on the data management structure of a university degree (subjects, teaching staff, students, classrooms...) shown in Figure 2. Based on this proposal, the students are asked to extend the database according to certain requirements. The students are given a text describing the new situation. With this text, the students are asked, firstly, to identify the entities to be added, the attributes of each entity and to place them in the diagram according to the explicit requirements, and, thirdly, to identify and determine the relationships between all the entities.

According to the diagram, we know that a student (ESTUDIANTS) can have many subjects (ASSIGNATURES) and a subject can have many students. For students, we have the NIA, name, and email as relevant data. For the subjects we want to register an identifier (ID), their name and the course where it is taken. A classroom (AULES) can host many subjects and each subject can be taught in a single associated classroom. For the classroom we want to save the number that identifies it and the capacity. We have registered the teaching staff (PROFESSORAT), who are identified with the NIA, in addition to having registered their name and the department they belong to. We know that a subject can have more than one teacher, but each teacher can only be in one subject. A subject can only have one responsible teacher, but each teacher can be responsible for more than one subject.

Figure 2. Example of an activity class

From that situation, students are asked to expand the database according to a set of requirements. A text describing the new situation is given:

We know that the classrooms are distributed in different buildings, so that we want to have a record in which building each classroom is. For each building we want to have a code, the name of the building, the location and the number of floors registered. For example, classroom B123 is located at the School of Engineering, which is located in Bellaterra (Spain). This building has 3 floors. We also want to record the articles that each teacher has published. We know that an article may have been coauthored by different teachers. For each article we want to have a numerical code, its title, the topic, the date of publication and the number of words it has. To be able to contact the students, we want to know which contact phone number(s) they have. We know that a student can have more than one contact phone (mobile, home, work ...). We want to have the telephone numbers of each student registered and a comment from them (if it is about work, contact hours...) if we have it.

Then, students are asked to:

• •identify the entities to be added to the database. 

• •identify the attributes of each entity and place them in the diagram according to the explicit requirements. 

• •define the corresponding primary key for each entity and highlight it in the diagram. 

• •identify and determine the relationships between the new entities and the original ones and illustrate them into the diagram. 

To ensure comprehension of the exercise, students are asked to think about specific data to be added to the database:

You must add the necessary data that meets the following requirements: •For each student in the database, we need to know their contact telephone numbers. We know that there are at least 10 students who have 2 or more contact phones.  •We want to know which building each classroom is located in. There must be at least 3 buildings.  •We must have a record of the articles that each teacher has published. We know that there are at least 30 professors who have published articles and that at least 20 of them have published more than one.

2.5. Data collection tools

To provide evidence of the impact of the Flipped Classroom methodology on the student learning, we use the questions of the different exams as evidence of their learning. Each course has two individual written tests, one for each type of content (database design and database queries). The design test contains 5 questions, and the queries test contains 4 questions. The tests are individual and are taken in a maximum of two hours. The questions are provided in paper format and students answer them in handwritten form without any support of external material. Since we are interested in evaluating the impact of the flipped classroom methodology on the different types of student learning (knowledge, skills, and competences) (OECD, 2002), we use as a reference the updated Bloom’s taxonomy (Anderson & Krathwohl, 2001). Bloom’s taxonomy classifies learning domains from the most basic to the most complex according to the following scale:

1. 1.Remember: recall facts and basic concepts.  

2. 2.Understand: explain ideas or concepts. 

3. 3.Apply: use information in new situations. 

4. 4.Analyse: draw connections among ideas. 

5. 5.Evaluate: justify a stand or decision. 

6. 6.Create: produce new or original work. 

In our study, we use the materials developed in the teaching innovation project that supported the change of teaching towards the flipped classroom (Ramos et al., 2021). To facilitate the design of exams, it was necessary to reduce the number of question types. For this purpose, learning domains were grouped together into three large blocks since they adequately connect with the type of content worked on and the needs generated by the understanding and use of databases. In this way, the questions that make up the tests were classified in reference to whether they evaluate learning referred to:

1. A)theoretical content (remember and understand). 

2. B)non-reproductive type developments (apply and analyse). 

3. C)competence developments (evaluate and create). 

The teaching team chose the appropriate questions for each typology and content, in agreement with the second author of this study, who has a specific background in education research. Table 2 shows an example of each type of question used in the exams.

Theoretical content (remember & understand)	Q: Explain what indexing is. Explain the different types of indexes. Q: Describe graphically and explain each of the components of the ANSI-SPARC database architecture. Which of them are relational, which are not? Why?
Non-reproductive knowledge (analyse & apply)	Q: Reason if it is possible to insert the tuple (NumReclamacio001, NULL, 14/01/2020, NULL, NULL, NULL, NULL) into the CLAIM table. Would the table be updated? Why? Q: Given the E-R diagram below, describe step by step and justify how the relational diagram would look.
Competence development (create & evaluate)	Q: Use relational algebra to pose the following queries of the air-line database. Q: Regarding the relational model of airlines, raise the statement and solve the corresponding SQL query in which at least three tables intervene, and a difference must be made.

Table 2. Samples of exam questions

As explained above, two exams are used during the course, one for each content block (design and queries). Of the 5 questions contained in the database design exam, 2 are of theoretical knowledge, 1 of non-reproductive content and 2 test competence aspects. On the other hand, of the 4 questions the database queries exam contains, 1 assesses theoretical knowledge, 1 is of non-reproductive content and 2 assess competency aspects. With this organisation and distribution of questions, all the learning objectives of the course are covered.

The quantitative data that make up this study are the numerical scores for each question and exam for students in grades 18/19 and 21/22, corresponding to a traditional course (TRAD) and a flipped classroom (FLIP) course, respectively. A total of 630 exams were collected, distributed by each course, and learning content according to Table 3.

	Design	Queries
18/19 TRAD	175	189
21/22 FLIP	135	131

Table 3. Number of exams collected per course and learning content

A key aspect of this study is the nature of the evaluation criteria and their application. A grading criterion is a standard that one uses as a reference for interpreting the information gathered in the assessment, i.e. for analysing it and making a judgement (Sanmartí, 2007) according to a previously established grading system (Giménez, 1997). The grading of open-ended examinations is a complex process (Wang & Cai, 2018) and may exhibit various types of bias during grading (Fitzpatrick, Ercikan, Yen & Ferrara, 1998), with discrepancies observed between different graders when grading the same set of examinations, being of relevance to the robustness of grading (Lane, Stone, Ankenmann & Liu, 1994). In our case, grading criteria were developed and agreed by the teaching team to distribute a total of 10 points among the questions, so that, each question had a score between 0 and the maximum given grade, with an intermediate score according to the number of minor and major errors done by the student. That is, the type of criteria used in this study is by penalty for error. To avoid bias between markers, it was decided that each question would be marked by the same person throughout the study.

Table 4 and Figure 3 show an example of criteria definition and application to a question dealing with the design of a database. As shown in Figure 3, students are asked to design an entity-relationship model according to the requirements arisen from the provided statement. In the response to this question four key understandings should be assessed, which correspond to the identification of the four basic elements in these types of diagrams: i) entities, ii) attributes of entities, iii) primary keys, and iv) relationships between entities. As the question is marked out of 4 points, each of these elements is awarded a maximum of 1 point. The teaching team agreed on a set of common errors in the resolution of this activity and determined the penalty associated with each of the errors according to their relevance. All the possible mistakes are shown in Table 4. Note that even if the sum of the penalties for a particular element is greater than 1 point, the penalty will never exceed 1.

Entities	Missing or extra entities	-0.5
	Entities without attributes	-1
	Entities without PK	-0.5 (only in 1 entity)
		-1 (in more than 1 entity)
	Weak Entities	-0.5 (by requirements is clearly very strong)
		-1 (identified as strong)
		0.25 (misspelled)
	Aggregation	-0.5 (cycle or ternary instead of aggregation)
		-1 (aggregation without any sense or missing)
	Unnecessary specialization to represent an attribute that takes finite set of values	-0.5
	Unidentified specialization	-0.5
Relationships	Erroneous cardinality	0.25 for each relationship
	Weak entity with erroneous cardinality	-0.5
	Missing or extra relationships	-0.5
	Missing relationship in the design but there is the arrow	-0.5
	Relationships of grade 3 (“starfish”)	-1
Attributes	Entity without any attribute	-1
	Redundancies: misuse of an attribute instead of creating a relationship	-1
	Using Multivalued in case of an attribute that took finite set of values	0.25
	Unidentified Multivalued	0.25
	Attribute that should be in an n-n relationship, is in one of the entities	0.25
Primary Key	Missing PK	-0.5 (Only in 1 place)
		-1 (more than 1 place)
	Poorly defined PK (incomplete, on wrong attributes)	-0.5
	PK in the relationship	-1

Table 4. Criteria for a non-reproductive question

In the Entity-relationship diagram shown in Figure 3 there are three errors, two in the category of entities and another in the category of attributes. All of them are squared in red and enumerated.

In the Entity-relationship diagram shown in Figure 3 there are three errors, two in the category of entities and another one in the category of attributes. All of them are squared in red and enumerated:

1. A)The first error belongs to the second item in the attributes category of table 4, since “guanyador” is assigned as an attribute in the relationship “té” but it should be a relationship “guanya”. 

2. B)The second error is in the relationship “té” between “COMPETICIONS” and “TEMPORADA”. “TEMPORADA” should be a weak entity because it does not have a well‑defined primary key. In case the student wants to design a strong entity with primary key it should be an aggregation.  

3. C)In the third case, there is a missing entity to store the history of the clubs where each “JUGADOR” has played and then the cardinality between “JUGADOR” and “EQUIP DE FUTBOL” is n-n. 

Figure 3. Example of marking of a non-reproductive question from an exam of design

3. Results

A quantitative analysis is performed to determine the impact of the methodological change on an overall improvement in students’ effective learning. The arithmetic mean of the two exams implemented during each course is computed using the weighting proposed in the evaluation of the subject. These global marks are compared by means of a Student t-test for paired data, where the global marks of each course are a variable of the test. To enable the comparison, all the global marks are given on a scale of 0-10 and the exams from each course have been randomly selected to have the same length of samples. Table 5 shows the mean and standard deviation of all global marks obtained in each course (TRAD and FLIP), together with the confidence interval and the p-value of the hypothesis test.

18/19 TRAD	21/22 FLIP	CI	p-value
5.421 ± 1.66	5.849 ± 1.94	(-0.865, 0.008)	.055

Table 5. Comparison of final marks

Note that the mean marks obtained for course 21/22 (FLIP) is higher than the mean marks of course 18/19 (TRAD), although the null hypothesis that there is no statistical significance between the two set of marks cannot be rejected (p=.055). This result would imply that the change in teaching methodology did not completely benefit the students in terms of learning as measured by the subject assessments. However, the p-value is very close to the significance set and the confidence interval is shift to the negative values, which suggests that the different types of knowledge evaluated should be delved into deeply.

Let the success rate be the percentage of correct answers done by a student in both exams, so that each question has the same weight. In this way, the overall performance of the students can be measured without any bias caused by the weights assigned in the course evaluation. Notice that the success rate is in the range [0,1]. Table 6 shows the mean and standard deviation of success rates obtained in each course (TRAD and FLIP), together with the confidence interval and the p-value of the corresponding hypothesis test.

18/19 TRAD	21/22 FLIP	CI	p-value
0.506 ± 0.175	0.576 ± 0.196	(-0.114, -0.025)	.002*

Table 6. Success rate

Note that the average success of learning without weighing the exam questions (Table 6) for the 21/22 course (FLIP) is higher than the results of the 18/19 course (TRAD) and the difference is statistically significant (p<.05). However, this difference does not represent a substantial improvement that would justify the workload, or the institutional changes introduced to adapt teaching to the Flipped Classroom methodology. To identify more precisely the types of learning and specific content on which the change in methodology had the greatest impact, the data is explored in a dis-aggregated way. Table 7 shows the results of the performance of the two blocks of course content (database design and database queries) with the corresponding hypothesis tests.

	18/19 TRAD	21/22 FLIP	CI	p-value
Design	0.501 ± 0.200	0.587 ± 0.220	(-0.134, -0.039)	.000*
Queries	0.502 ±0.220	0.526 ± 0.212	(-0.074, 0.026)	.034

Table 7. Success by blocks of contents

These results show an uneven impact of the Flipped Classroom methodology on the two content blocks. In learning database queries, the improvement is small and not significant, whereas the database design content benefited in a statistically significant way. This suggests an imbalance between these two content areas, which makes us wonder if there are aspects of the Flipped Classroom methodology that benefit one more than the other.

Besides, the ultimate goal of implementing the Flipped Classroom is to enable students to develop competencies in database content. For that the impact of the methodological change on the different types of learning (theoretical, non-reproductive, competence) is analysed without distinguishing between types of content. Table 8 shows the mean and standard deviation of marks according to the type of learning ignoring the type of content, together with the confidence interval and the p-value of the corresponding hypothesis test.

	18/19 TRAD	21/22 FLIP	CI	p-value
Theoretical	0.454 ± 0.340	0.611 ± 0.325	(-0.200, -0.113)	.000*
Non-reproductive	0.471 ± 0.366	0.548 ± 0.321	(-0.131,0.021)	.007*
Competence	0.540 ± 0.275	0.532 ± 0.292	(-0.024, 0.040)	.623

Table 8. Success rate by type of learning

The results divided by the performance according to the type of learning show an unexpected impact of the Flipped Classroom methodology. The proposal to change the methodology was driven by the need to improve the development of the students’ competences, considering that the traditional methodology did not allow working in the classroom with open activities in which the students could develop their own solution proposals. However, the results obtained show a statistically significant positive impact of the Flipped Classroom methodology for theoretical learning and for non-reproducible problems. For competency-based learning, the average performance decreased between the two courses, but only less than one tenth of a point, without any significant difference.

To have an exhaustive picture of the situation, the results have been completely dis-aggregated, distinguishing by type of content and type of learning. Results are shown in Table 9, split in a first level by learning contents (database design in top rows and database queries in bottom rows) and in a second level by learning type (theoretical, non-reproductive, competence).

		18/19 TRAD	21/22 FLIP	CI	p-value
Design	Theoretical	0.405 ± 0.325	0.635 ± 0.335	(-0.283, -0.178)	.000*
	Non-reproductive	0.675 ± 0.291	0.594 ± 0.334	(0.011, 0.151)	.023*
	Competence	0.509 ± 0.290	0.535 ± 0.312	(-0.074, 0.021)	.281
Queries	Theoretical	0.545 ± 0.348	0.556 ± 0.298	(-0.088, 0.059)	.692
	Non-reproductive	0.282 ± 0.324	0.499 ± 0.300	(-0.288, -0.147)	.000*
	Competence	0.569 ± 0.256	0.529 ± 0.271	(-0.002, 0.021)	.060

Table 9. Success by blocks of contents and by type of learning

These results show an asymmetry in the impact of the methodology in terms of the different types of learning that the students have acquired. On the one hand, the type of learning that has been improved for the database design block concerns to the reproductive aspects, while results on non-reproductive marks are significantly worsen with the Flipped Classroom. On the other hand, the improvement for the database queries block is focused on non-reproductive activities. Notice that there is no significant improvement on the development of competences in any contents block.

4. Discussion

In this study, a study of the impact on student learning in a second-year database course in Computer Engineering by replacing the traditional teaching methodology by Flipped Classroom is presented. An essential part of the project work has been to identify the different types of learning (reproductive, non‑reproductive, competency-based) promoted by the Flipped Classroom methodology in relation to the traditional methodology in terms of database design and database queries. This has allowed for a fine‑tuning of the type of learning promoted by the change in teaching methodology. This study was possible because there are two specific features in the teaching context of this subject. Firstly, the teaching team has remained stable over time and has been able to develop various teaching improvements. A key element is that the learning objectives were defined in a previous teaching innovation project, which makes it possible to have validated correction criteria that persist over time. Another essential element is that the implementation of the Flipped Classroom was iterated until it was successful. This allows the selection of two courses for a proper comparison.

The results show that there is a small but significant improvement in overall database learning, but that this is not based on the development of database skills. This main finding may be hopeless or inconsistent with the published literature highlighting the benefits of the Flipped Classroom methodology (Freeman et al., 2014; Cheng, Ritzhaupt et al., 2019; Cheng, Liu et al., 2019; Fulton, 2012; Chun & Lee, 2016; Van Vliet et al., 2015) but must be analysed in detail in order to observe its impact on students’ learning, as it is no less true that previous studies show some difficulties in its implementation (Cheng, Ritzhaupt et al., 2019; Altaii et al., 2017; McLaughlin et al., 2014; Diwanji et al., 2018; Palmer, 2015; Sahin et al., 2015; Gillette et al., 2018).

First, not all specific content types gain in student achievement by incorporating the Flipped Classroom. This type of findings is not usually reported in studies, which tend to focus on the positive aspects of implementing the flipped classroom (Gren, 2020; Karabulut-Ilgu et al., 2018). Teaching queries has a small (non-significant) benefit from switching to the Flipped Classroom, but databases design content makes a difference. This is attributable to the fact that the increased time for discussion with students in the classroom has made easier to address in more detail the motivations and needs of database design. It is not always clear for students to understand the implications of databases design for later use. This fact opens the door for teaching teams to consider for each content block whether it is worth adapting the Flipped Classroom methodology.

An unexpected result is the large positive impact on student achievement of theoretical content. Traditional methodology relies on long sessions of lectures on theoretical content, which do not seem to have much impact on student learning. In the past, several problems that discouraged students from attending or benefiting from these sessions were identified. However, the shift to a flipped methodology, including the use of short, scripted, and edited videos to maximise their impact on students, may be responsible for the observed improvement in theoretical knowledge. The videos used are synoptic and focus on the most relevant content, allowing students to clearly identify the basic concepts to be learned and to spend sufficient time visualizing them without a timetable. Although the purpose of the methodological change does not explicitly refer to the theoretical content, it can be considered that this improvement is positive because of the importance of knowing and understanding the precise formulation of the key concepts about databases, since this level of specification is essential in engineering (Swart, 2009).

On the other hand, non-reproductive activities show a large significant improvement with Flipped Classroom methodology in the database queries block, although not in the database design block. The improvement could be done because of the increase in classroom time for discussions among students and between students and teacher for solving activities and problems involving decision-making (Cheng, Ritzhaupt et al., 2019; Cheng, Liu et al., 2019; Öncel & Kara, 2019), in particular on how to make successful databases queries. Still, the worsening in the database design, although being significant, is lower. This suggests that this fact is not attributed only to the Flipped Classroom methodology, but the nature of the tasks proposed to students should also be revised. Regarding to the questions showing competence development and connection to real-world situations in engineering, the adjustment of the Flipped Classroom had a small and not significant impact in any case. This leads us to think that to promote this type of knowledge it is necessary to introduce more extensive changes in the design of the subject. In other areas it has been documented that project-based learning can enhance competence development, but the introduction of database content, due to the need to introduce the technical characteristics of database formulation, does not allow for this. In this sense, it is important to highlight that during the 19/20 academic year this subject was the first to use Flipped Classroom in which students participated, with no other subject in the same academic year sharing this methodology. From our experience, the implementation of the Flipped Classroom requires a change in students’ habits and benefits from a context in which several subjects use the same teaching methodology.

5. Conclusions

This study confirms that the implementation of the Flipped Classroom methodology as a pedagogical methodology, replacing the traditional methodology in the subject of Databases in Engineering Studies, has an impact on the students’ learning outcomes, but not the results expected at first. Due to its characteristics, the Flipped Classroom should improve the development of students’ competences, but this impact has not been statistically confirmed. Some significant improvements can be observed in theoretical learning and in the resolution of non-reproductive activities.

Besides, the impact of the Flipped Classroom is different depending on the type of content to be taught has been proved. This suggests that, to implement a teaching change of the magnitude of switching to Flipped Classrooms, teaching teams could take this step gradually, assessing the potential of Flipped Classrooms for each type of content and focusing their efforts on those subjects where the change is expected to have a positive impact. It is also important to note that this study is only based on the consolidated learning that was demonstrated by the assessment tests. The teaching team has observed other changes that have had a positive impact on the students’ development, both in tangible aspects such as increased class attendance, and at the level of self-perception as future engineers.

This study opens the door to new lines of research. Concretely, it seems necessary to investigate whether the effects of the flipped classroom also present variations in the learning promoted for other content specific to computer science studies. On the other hand, these differences in learning outcomes could be nuanced by introducing specific activities in the courses. In this way, a teaching methodology based on the flipped classroom could be developed but tailored to each type of content or learning objective in engineering courses.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This research was supported by Oficina de Qualitat Docent, UAB (GI513783).

References

Altaii, K., Reagle, C.J., & Handley, M.K. (2017). Flipping an engineering thermodynamics course to improve student self-efficacy. In 2017 ASEE annual Conference & Exposition. American Society for Engineering Education.

Anderson, L.W., & Krathwohl, D. (2001). A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives complete edition. Longman.

ANECA (2004). Libro Blanco del título de grado en Ingeniería Informática. ANECA.

Cheng, L., Ritzhaupt, A.D., & Antonenko, P. (2019). Effects of the flipped classroom instructional strategy on students’ learning outcomes: A meta-analysis. Educational Technology Research and Development, 67(4), 793-824. https://doi.org/10.1007/s11423-018-9633-7

Cheng, P.W., Liu, P.W., Huang, C.C., & Shyr, W.J. (2019). The effects of the flipped classroom on technical high school students with low self-esteem. The International Journal of Engineering Education, 35(5), 1518‑1525.

Chun, K.H., & Lee, Y.H. (2016). Improvement of debate competence: an outcome of an introductory course for medical humanities. Korean Journal of Medical Education, 28(1), 87. https://doi.org/10.3946/kjme.2016.13

Davies, R.S., Dean, D.L., & Ball, N. (2013). Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educational Technology Research and Development, 61(4), 563-580. https://doi.org/10.1007/s11423-013-9305-6

Della-Ratta, C.B. (2015). Flipping the classroom with team-based learning in undergraduate nursing education. Nurse Educator, 40(2), 71-74. https://doi.org/10.1097/NNE.0000000000000112

Diwanji, P., Hinkelmann, K., & Witschel, H.F. (2018). Enhance classroom preparation for flipped classroom using ai and analytics. In Proceedings of the 20th international Conference on Enterprise Information Systems (1, 477-483). ICEIS. https://doi.org/10.5220/0006807604770483

Fitzpatrick, A.R., Ercikan, K., Yen, W.M., & Ferrara, S. (1998). The consistency between raters scoring in different test years. Applied Measurement in Education, 11(2), 195-208. https://doi.org/10.1207/s15324818ame1102_5

Freeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okoroafor, N., Jordt, H. et al. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410-8415. https://doi.org/10.1073/pnas.1319030111

Fulton, K. (2012). Upside down and inside out: Flip your classroom to improve student learning. Learning & Leading with Technology, 39(8), 12-17.

Gillette, C., Rudolph, M., Kimble, C., Rockich-Winston, N., Smith, L., & Broedel-Zaugg, K. (2018). A meta-analysis of outcomes comparing flipped classroom and lecture. American Journal of Pharmaceutical Education, 82(5), 6898. https://doi.org/10.5688/ajpe6898

Giménez, J. (1997). Evaluación en matemáticas: una integración de perspectivas. Síntesis.

Gren, L. (2020). A flipped classroom approach to teaching empirical software engineering. IEEE Transactions on Education, 63(3), 155-163. https://doi.org/10.1109/TE.2019.2960264

Hernández-Sabate, A., Albarracín, L., Gil, D., Ramos, O., Sánchez, C., Valveny, E. et al. (2021). Canviant la metodologia docent per a l’adquisició real de competències a Enginyeria Informàtica. Revista del Congrés Internacional de Docència Universitària i Innovació (CIDUI), 2021, Núm. 5. https://raco.cat/index.php/RevistaCIDUI/article/view/389864.

Hernández-Sabaté, A., Albarracín, L., & Sánchez, F. J. (2020). Graph-based problem explorer: a software tool to support algorithm design learning while solving the salesperson problem. Mathematics, 8(9), 1595. https://doi.org/10.3390/math8091595

Jang, H.Y., & Kim, H.J. (2020). A meta-analysis of the cognitive, affective, and interpersonal outcomes of flipped classrooms in higher education. Education Sciences, 10(4), 115. https://doi.org/10.3390/educsci10040115

Kang, N. (2015). The comparison between regular and flipped classrooms for EFL korean adult learners. Multimedia-assisted language learning, 18(3), 41-72. https://doi.org/10.15702/MALL.2015.18.3.41

Karabulut-Ilgu, A., Jaramillo-Cherrez, N., & Jahren, C.T. (2018). A systematic review of research on the flipped learning method in engineering education. British Journal of Educational Technology, 49(3), 398-411. https://doi.org/10.1111/bjet.12548

Keengwe, J., Onchwari, G., & Agamba, J. (2014). Promoting effective e-learning practices through the constructivist pedagogy. Education and Information Technologies, 19(4), 887-898. https://doi.org/10.1007/s10639-013-9260-1

Lage, M.J., Platt, G.J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31(1), 30-43. https://doi.org/10.2307/1183338

Lane, S., Stone, C.A., Ankenmann, R.D., & Liu, M. (1994). Reliability and validity of a mathematics performance assessment. International Journal of Educational Research, 21(3), 247-266. https://doi.org/10.1016/S0883-0355(06)80018-2

MacLeod, M., & van der Veen, J.T. (2020). Scaffolding interdisciplinary project-based learning: a case study. European Journal of Engineering Education, 45(3), 363-377. https://doi.org/10.1080/03043797.2019.1646210

McLaughlin, J.E., Roth, M.T., Glatt, D.M., Gharkholonarehe, N., Davidson, C.A., Griffin, L.M. et al. (2014). The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Academic medicine, 89(2), 236-243. https://doi.org/10.1097/ACM.0000000000000086

Melero, J., Leo, D.H., & Blat, J. (2012). A review of constructivist learning methods with supporting tooling in ICT higher education: Defining different types of scaffolding. The Journal of Universal Computer Science, 18(16), 2334-2360. https://doi.org/10.3217/jucs-018-16-2334

Mok, H.N. (2014). Teaching tip: The flipped classroom. Journal of Information Systems Education, 25(1), 7-11.

Mulder, M. (2014). Conceptions of professional competence. In Billett, S., Harteis, C., & Gruber, H. (Eds.), International handbook of research in professional and practice-based learning (107-137). Springer. https://doi.org/10.1007/978-94-017-8902-8_5

OECD (2002). Definition and selection of competencies: Theoretical and conceptual foundations. OECD.

OECD (2005). The definition and selection of key competencies. Executive summary. OECD.

Öncel, A.F., & Kara, A. (2019). A flipped classroom in communication systems: Student perception and performance assessments. The International Journal of Electrical Engineering & Education, 56(3), 208-221. https://doi.org/10.1177/0020720918788718

ONU (2015). Transformar nuestro mundo: la Agenda 2030 para el Desarrollo Sostenible.A/70/L.1. Available at: https://unctad.org/system/files/official-document/ares70d1_es.pdf

Ortiz, D. (2015). El constructivismo como teoría y método de enseñanza. Sophia, 19, 93-110. https://doi.org/10.17163/soph.n19.2015.04

O’Grady, M.J. (2012). Practical problem-based learning in computing education. ACM Transactions on Computing Education, 12(3), 1-16. https://doi.org/10.1145/2275597.2275599

Palmer, K. (2015). Flipping a calculus class: One instructor’s experience. Primus, 25(9-10), 886-891. https://doi.org/10.1080/10511970.2015.1050618

Pluta, W.J., Richards, B.F., & Mutnick, A. (2013). PBL and beyond: Trends in collaborative learning. Teaching and learning in medicine, 25(sup1), S9-S16. https://doi.org/10.1080/10401334.2013.842917

Ramos, O., Albarracín, L., Martí, E., Hernandez-Sabaté, A., & Gil, D. (2021). Activitats d'aula inversa en l'àmbit de l'enginyeria. Experiència en la docència de bases de dades. Experiències docents innovadores a la UAB, 111-126.

Rychen, D.S., & Salganik, L.H. (2003). Key competencies for a successful life and well-functioning society. Hogrefe Publishing.

Sahin, A., Cavlazoglu, B., & Zeytuncu, Y.E. (2015). Flipping a college calculus course: A case study. Journal of Educational Technology & Society, 18(3), 142-152.

Sanmartí, N. (2007). 10 ideas clave. Evaluar para aprender. Graó.

Swart, A.J. (2009). Evaluation of final examination papers in engineering: A case study using Bloom’s Taxonomy. IEEE Transactions on Education, 53(2), 257-264. https://doi.org/10.1109/TE.2009.2014221

Tsai, H.L., & Wu, J.F. (2020). Bibliometric analysis of flipped classroom publications from the web of science core collection published from 2000 to 2019. Science editing, 7(2), 163-168. https://doi.org/10.6087/kcse.212

Van Vliet, E.A., Winnips, J.C., & Brouwer, N. (2015). Flipped-class pedagogy enhances student metacognition and collaborative-learning strategies in higher education but effect does not persist. CBE—Life Sciences Education, 14(3), 26. https://doi.org/10.1187/cbe.14-09-0141

Wang, N., & Cai, J. (2018). An investigation of how teachers score constructed-response mathematics assessment tasks. Journal of Research in Education, 28(1), 1-29.

Yániz, C. (2006). Planificar la enseñanza universitaria para el desarrollo de competencias. Educatio Siglo XXI, 24, 17-34.

---