DEVELOPMENT AND ASSESSMENT OF GENERIC COMPETENCES IN ENGINEERING DEGREES THROUGH CREATIVITY

Diego Anguís Climent

Higher Technical School of Building Engineering. University of Seville

Spain

Received June 2011
Accepted February 2012

Abstract

Now more than ever, the social demands on the University require huge amounts of creativity to respond to the socio-economic context of today’s society. Creativity is an ability that requires teamwork as a framework for discussion of the particular ideas of its members, and a scenario for the development of necessary attitudes of tolerance.

The experience described was developed with the help received by the University of Seville, within the Call for Teaching Innovation and Improvement Projects of the University’s First Teaching Plan during the 2010-2011 academic year, in a subject corresponding to the second term of the first year of Building Engineering. The experience was designed to develop a very short video with subject-related content to motivate the development of creativity among the students, and which would allow them to understand their own strengths and weaknesses in this area. It was also designed to achieve certain transverse competences included within the learning objectives of the subject, such as the ability to communicate through word and image in the context of a project in which teamwork is promoted.

Given that such competences are always difficult to assess, the experience also sought to explore different approaches through peer assessment (using rubrics designed specifically for this case) and assessments involving agents external to the educational process.

The results obtained show that, from the context of teaching technical subjects, it is possible to develop creativity together with the specific knowledge of the subject, and this experience is proposed as a new model for its teaching.

Keywords -  creativity, formative assessment, participatory assessment.

----------

1         INTRODUCTION. context of the subject

One of the most important aspects of the recent university adaptation to the framework of the European Higher Education Area is, undoubtedly, the need to ensure that students have acquired certain professional competences upon the completion of their educational period. The various teaching programmes have been the first to establish an initial segregation between those competences which have come to be known as specific and those referred to as generic or transversal.

The traditional theoretical and practical structure of the different subjects that make up any engineering degree, coupled with the existing time constraints, is conducive to the different educational programmes primarily seeking to develop their own specific competencies. Thereby, the tools and mechanisms necessary to ensure that students acquire those generic competences are left in an ambiguous and indeterminate space, a kind of “no man's land” and, of course, without any attempt to assess those competencies.

As engineers, our most quoted value should be our ideas, but we cannot know our capabilities if we do not promote the opportunity to exercise them.

The course in which this experience was gained was Materials I, which is given within the second term of the first year of the Building Engineering degree. It is a subject with 6 ECTS credits and is allocated a total of 30 hours theoretical teaching, and another 30 hours allocated among problem solving classes and laboratory practice. The theoretical and practical teaching (problems and laboratory) are assessed independently.

The teaching project of this subject sets out the need to train the student to obtain a series of generic competences, including: “capacity for reasoning, discussion and presentation of their own ideas”, “capacity for communication through word and image”, “aptitude for oral and written communication”, and “capacity for autonomous learning”.

The project presented in this paper was applied in a group of 64 students, of which 49 participated voluntarily, and who were distributed in 19 working groups. The project was undertaken in three attended sessions of two hours each. It was also estimated that the student should dedicate 18 hours of free time to the project. The attended sessions were conducted in the theory classes and consisted of a first phase of group brainstorming, followed by a phase for the presentation of the various video production techniques by the group of experts and a final phase for the exhibition of all the work. The project was given a timeframe of eight weeks, during which specific tutorials of one hour a week were established. These tutorials were intended to monitor the work and to answer any questions, both technical and conceptual, that might arise during the project. The broad timescale of the project enabled the students to reflect and bring the content of the work to fruition, allowing them to organize their attendance at tutorials (which should be in group form) and to be able to combine the work with their other scheduled obligations.

The project teaching team consisted of the subject teacher and three professionals in the field of audiovisual production acting as a group of experts.

The initial idea for the project came from a survey, made at the end of first term, of the same group of students that would go on to undertake the project in the second term. Coinciding with a critical session held on the subject of a travelling exhibition of models that was held in the grounds of the school, the students were asked a series of questions to find out how many of them considered what they had seen, to be nothing creative, moderately creative or very creative. None of them considered the models as “nothing creative”. Only a few were identified as “very creative”. Most preferred to classify them as “moderately creative”.

However, what does being “moderately creative” mean? How can we measure our own creativity? No one asked what it meant, which showed us an interesting gap in this aspect.

The work that was proposed to students had the objective of encouraging reflection about the possibilities of light as a building material. What is it or what is it not? What might it become? What does it represent when it exists and when it does not, or when it is reflected, alone or with other materials? They were encouraged to play with light, discovering its transformations, its different states, and more.

To do this, it was suggested that they address the knowledge of the construction materials from a different perspective. Not as a series of objects in themselves, but as elements whose diversity could produce different perceptions and emotions in the observer. All this from the relationship between these materials, which students begin to know through theoretical and practical lessons, and an element that cannot be abstracted: light.

The question that was put to them as the core of the project was: Do you believe that light is a building material?

The objective of the project was the development of a video with a maximum duration of 90 seconds, through which they could express their opinion about the question, using resources typical of this medium, with verbal speech, music, images, etc. The technique for the preparation was free.

It was intended to encourage the student in finding the information and resources needed for the development of the work, using the strategy of incorporating them in an active learning methodology (Anguís, 2008; Anguís, 2010). This was hoped to establish synergies which would enable them, in the future, to improve their learning throughout life.

2         OBJECTIVES OF THE PROJECT

This project had different objectives:

·     To turn the classroom into a laboratory, i.e. a place to analyse and explore concepts whose meaning is known intuitively, but whose knowledge is unknown.

·     To awaken the imaginative creativity of the students through play, through the teaching challenge, to enable them to transform an idea into images and words.

·     To exercise this competence with the transforming challenge, with the flexibility and originality of thought, by means of an active and personalizing activity, while being socializing and cooperative, leading to new ways of thinking and feeling.

·     To develop the technical capacity of the student to be able to communicate their thoughts through word and image, with current resources and technologies.

3         METHODOLOGY DEVELOPED

From the start, this project was not conceived as an extra-curricular activity independent of the subject programme of Materials I, but as a methodological innovation to achieve the generic transverse competences included within the course syllabus.

To carry out this project, several working groups of 2 to 3 students were freely organized among themselves. The project was undertaken voluntarily by a total of 49 students, representing 70.3% of the pupils who were taking the course.

A plan was established with the students, within the fifteen teaching weeks into which the term was organized, which did not interrupt the progress of the subject teaching. The presentation of the work was made in the last week of the course, after the conclusion of the course interim exams.

The project was coordinated by the subject teacher who participated in all the theoretical sessions, in addition to attending the established complementary tutorials. Also, three professionals were selected for the team of experts whose activity was focused on the area of development of interactive visualization projects, visual experimentation and audiovisual productions.

Also, through being proposed as a voluntary activity, scoring criteria additional to the final mark obtained in the course was established.

The assessment of the work was made using a hetero-assessment system that included a peer assessment with the one made by the teacher and the group of experts invited to the project. This also allowed the analysis of possible differences between the teacher's assessments and those of the group of experts.

Throughout the whole project it was intended that the work environment would approximate more to that of play than to the scholastic. As in any game, it was proposed to develop the exercise through the promotion of peer relationships, shared thinking, enjoyable work and teamwork (Serrano, 1996), with the understanding that these strategies could provide an innovative but valid learning system.

Each group approached the proposed theme from their own point of view.

The methodology used was divided into the three parts detailed below.

3.1 Formation of the Work Groups. Brainstorming

To inculcate our students with this necessary atmosphere, we provoked them, from a creative position, to help them to open fields of thought that they could later develop, the first session was conducted using the brainstorming technique (Prado, 2000). This technique had already been used with the same group during the first term which allowed the right climate to be quickly established, allowing them to express themselves with more freedom and spontaneity.

During this first session of the project, it was important that it was the students directing debate among themselves. The teacher only ordered the ideas that emerged and re-launched the dialogue on those occasions when the subject seemed to run out of steam.

During this session, the students were questioned about aspects that might lead to understanding light as a building material. There was discussion about the qualities of natural light, the various forms of artificial light and how the same object could change its setting by changing only these aspects. How could we define, as with any building material, its essential properties related to texture, colour, brightness, etc.? How then to specify its mechanical properties?. Could we use a building material whose intrinsic qualities were so changeable, so mutant? How were our subjective and individualized emotions going to be affected by the objective qualities of this material? They were also asked about darkness: Could it also be considered a building material? Throughout this process, the action of the teacher walked the fine line between motivation and challenge, trying to push them just enough so they started to film themselves in the new scenario presented to them, so alien to the classic teaching scenario to which they were so accustomed and in which they were so comfortable.

3.2 Contribution of the Experts on various Types of Communiction through Video

Like any process of knowledge which seeks to integrate any current technological innovation, the students had to be provided with the knowledge necessary to enable them to successfully approach the development of the project. To provide these initial tools, the team of invited experts introduced them by means of a two-hour theory session on the various techniques for the capture and processing of images for video. This team of experts subsequently participated in the assessment of the presented papers (Aldarías, 2010).

3.3  Development and Preparation of the Work

Each group of students arranged 60 days for the information search and development work.

During this period an additional tutorial of one hour per week was established. This time was designed to monitor the work and resolve doubts, about both the technical shortcomings that the student might have in producing their work, and questions that had been raised related to the focus of the work (Martínez, Santos & Padilla, 2011).

4         presentation of the projects and assessment criteria

4.1 Preparation of the Work

Prior to the presentation session, all participating groups submitted their work in video format. The presentation took place in a two-hour session, attended by the students participating in the activity and the other students in the class who had not participated in the project, but wanted to attend. A total of five participating students, corresponding to two working groups, did not attend the presentation of their work, although they were equally shown and assessed. The number of students who attended the assessment was 52, including the 44 pertaining to the workgroups.

Together with the students, the works were seen by the teacher and the expert group who had participated in the initial theory session of the project.

The works submitted were numbered from 1 to 19, with the number of the work to begin the presentation being selected at random. The presentation continued from this number sequentially corresponding to the number assigned when the works were submitted.

Most work presented used the stop motion technique, with the inclusion of music and/or additional dialogue. In most cases the images were made directly by the authors, obtaining movement from the montage of successive still images shown at higher speed.

The works were shown without any prior presentation by the authors, in order to ensure, as far as possible, the fidelity of the visual message. After the presentation of the work, the attending students were asked to give their opinion on the score sheet/rubric that had been previously distributed. A small discussion was then begun, where students, teacher and the expert group expressed their views on the shown work.

4.2 Assessment Criteria

As indicated above, the subject teaching project did not have its own time space delimited for the development of such competences. Neither were parameters set that would allow them to be assessed, with the assessment systems and criteria only specified in relation to the knowledge of the student on the thematic blocks, in which the course was structured. This, coupled with the voluntary nature of the activity, made it advisable that it be assessed as an improvement in the student's overall grade.

Being aware of the difficulty of assessing such competences, we decided to conduct a participatory evaluation system that integrated the assessments of the teacher and the experts as well as the peer assessment conducted by the students themselves (Martinez & Cadenato, 2010).

Table 1. Assessment tool used to assess the presented videos

The peer assessment was carried out using a hybrid instrument that linked a work rating scale with a rubric designed specifically for this project (García, Terrán & Blanco, 2009, Torres & Perea, 2010). Some proposals of various authors (Eduteka, 2011; Borges, 2011; Núñez Molina, 2011) served as the basis for the preparation of this assessment tool, but, given the special case of this project, they had to be adapted to its particular needs. The issues assessed using the rubric included issues such as quality of the presented images, sound quality, the effects and transitions used, appropriate working time in relation to the transmitted message, originality in the manner of addressing the subject, the existence of an argument in the presentation, the interest shown in the development and presentation of the work, etc. The questionnaire allowed a score of 1-5 in each of the sections. This rating scale, as already indicated, was distributed to students before viewing the videos, in order that they knew, a priori, which aspects to assess.

All the students who attended, whether they participated in the activity or not, assessed all the work except their own. The assessment tool included the name of the student assessor.

Due to the uniqueness of the project, it was suggested at the outset that the teacher's assessment would be compared with that of the group of experts invited to the project. The assessments made by the students were also compared, to achieve greater objectivity and balance in the formal assessments of the contents.

The scores obtained for the presented work were added to the marks obtained by the students in the theory block of the subject. Two points were assigned to work that reached a high average score, calculated from the twelve items rated (3.91). One point was assigned to an average rating of the work presented (2.89). Work located between the two limits was scaled linearly. Work whose value was below the average of the ratings was scored with 0.5 points.

The assessment tool used can be seen in Table 1.

5         results

The first aspect to highlight is the low attendance of participating students at the tutorials. This indicates a lack of appreciation of this resource as part of their learning process (Ramírez, Sampedro & Martín, 2011). This fact is evidenced by the result of some of the presented work. On the contrary, it can be shown that, generally, those teams that made most use of the tutorials obtained a better evaluation of their work in the peer assessment.

The score for each work was treated mathematically to determine an average that allowed some study parameters to be obtained that could be extrapolated to the whole.

Thus, the average result obtained, on a maximum score of five points, for the different aspects evaluated are shown in Table 2 and Figure 1, where they are also compared with the evaluations obtained by the experts and teachers.

Table 2. Average score and standard deviation of the peer assessment.

The first three issues relate to more general evaluations, such as: effort, work and interest reflected in the presented work, overall score of the work undertaken, or interest in the approach to its production. The assessment by the expert group on these items was slightly higher in all cases than that obtained in the peer assessment.

Contrary to what happened in this first set of aspects, the second presented greater uniformity between the expert and peer assessments. The aspects most valued by the students were the sound quality (average 3.01) and quality of the images used (average 3.12). In contrast, the aspects less valued by them related to those questions which referred to the assessment of the "imaginative" and "own" qualities in which the work had been approached, such as "approach to the subject", "structure and organization of the project" and "overall idea". To frame the assessment of the students on the last question, the section included a brief explanation which asked the students whether, after seeing the video, they had just some idea of what the authors wanted to express, or if they understood it very well. On these same aspects the assessment by the expert group was even lower. Certainly this aspect of the project should have had more work put into it, which would have been possible if the time established for the tutorials had been properly used.

Fig. 1. Results obtained, on a maximum score of five points, for the different aspects assessed by the expert group and the teacher and the students themselves.

Regarding the assessment of the duration of the work, the expert group and teacher gave higher value to the appropriateness of the time spent than to the transmitted message.

In general, the values given by the teacher to the different aspects of the work were lower than those given by the expert group for the same items.

The standard deviations shown in Table 2 reflect a low dispersal in the results. Regarding the assessment of sound quality, the high overall score obtained in this section, with the greatest spread of results, is highlighted, and is due to the large differences in quality that the different works presented.

After the meeting, there was an interesting discussion between students, experts and teacher, with comments on some of the aspects seen in the videos.

As initially assumed, and due to the chosen theme, there were many lines of thought and above all, many forms of expression, brought about in the formalization of the conclusions of each work group, which we understand have notably enriched the group.

6         conclusions and prospects for improvement

The contents developed in this project allowed the students to discover their creative abilities and provided them, through their involvement in active learning methodologies, new training resources that will be of great usefulness in the future.

The results of the peer assessments were similar to those made by the experts and teachers. The students showed the ability to think critically about the work of their peers and take rigorous assessment activities, integrating them into those parts of teaching that until now seemed out of their reach, and making the student a participant in their own learning.

Unlike the assessment made by the teacher, we have verified that the peer assessment and that of the expert group were shown to comprise a more objective and comprehensive assessment system.

Regarding the limited use that the students made of tutorials during the project, it is necessary to analyse some proposals with a view to future projects. For future projects, we think that a possible solution might be to incorporate a planned series of tutorials into the project, at which attendance should be obligatory in this case, and the content of which should also be assessed by the tutor, so that they could be used to assess student progress, as well as guide and accompany them in their learning process.

Finally, we believe it is important to start these types of projects among first-year students to involve them in new teaching strategies from the very beginnings.

Also, we generally think that progress must be made in the coordination between subjects that include the same generic competences in their teaching projects, establishing the methodology together to ensure that students have reached their expected capabilities at the end of their training cycle.

Finally, it must be noted that these types of project, based on group work and participatory assessment systems, continue to arouse mistrust among the more traditional university sectors. Moreover, "the management process is complicated and more time consuming" which often makes strict compliance difficult in a teaching programme full of specific competencies to be achieved.

references

Aldarías, J. (2010). Portofolio Javi Aldarias. From http://issuu.com/javialdarias/docs/portfolio_javialdarias

Anguís, D. (2008). La incorporación de modelos activos en el aprendizaje, una forma de integración en el EEES. Revista de Enseñanza Universitaria, 30, 19-30. http://www.publius.us.es/reu/indice_contenidos/num_30

Anguís, D. (2010). La incorporación de metodologías activas como estrategia docente en las ingenierías. Proceedings of the Congreso Internacional de Docencia Universitaria e Innovación. CIDUI. Universitat Politécnica de Catalunya.

Borges, E. (2011). Rúbrica para evaluar presentaciones orales. Retrieved September 12, 2011, from http://www.slideshare.net/elizabethborgesocasio/rbrica-presentacin-oral

Eduteka. Matrices de Valoración. Retrieved  October 23, 2011, from http://www.eduteka.org/proyectos/RubricPresentacion.php3

García, M. J., Terrán, M. J., & Blanco, Y. (2009).  Desarrollo de recursos docentes para La evaluación de competencias genéricas. Proceedings of the XV Jornadas de la Enseñanza Universitaria de la Informática. JENUI. Barcelona.

Martínez, I., Santos, C., & Padilla, T. (2011). Autoevaluación y evaluación por pares: Participación del alumnado en la evaluación de su aprendizaje. Experiencias innovadoras en el aprendizaje a través de la evaluación. Evaltrends pp. 347-359. http://evaltrends.uca.es/index.php/publicaciones.html.

Martinez, M., & Cadenato, A. (2010). Integración y evaluación de competencias genéricas. Proceedings of the XVIII Congreso Universitario de Innovación Educativa en las Enseñanzas Técnicas. Santander: Escuela Técnica Superior de Ingenieros Industriales y de Telecomunicación. Universidad de Cantabria.

Núñez Molina, M. (2011). Rúbrica para la evaluación de presentaciones orales. Retrieved September 12, 2011, from www.vidadigital.net/blog/2007/05/11/rubrica-para-la-evaluación-de-presentaciones-orales

Prado, D. (2000). Torbellino de ideas por una Educación participativa y creativa. Santiago de Compostela: Universidad de Santiago de Compostela.

Ramírez, A., Sampedro, B. E. & Martín, V. (2011). Calidad en la planificación docente a través de la autoevaluación del profesorado universitario, Experiencias Innovadoras en la sistematización de la evaluación. Evaltrends pp. 146-157. http://evaltrends.uca.es/index.php/publicaciones.html

Serrano, J. M. (1996). El aprendizaje cooperativo. In J. L. Beltrán & C. Genovard (Eds.), Psicología de la Instrucción I. Variables y procesos básicos (pp. 217-244). Madrid: Síntesis. 

Torres, J. J., & Perea, V.H. (2010). La rúbrica como instrumento pedagógico para la autorización y evaluación de los aprendizajes en el foro on-line en educación superior. Revista de Medios y Educación, 36, 141-149.

 

AUTHOR BIOGRAPHY

Diego Anguís

Doctorate in Architecture from the University of Seville and Associate Lecturer in the Department of Architectural Constructions II, Dr Anguís Climent has been teaching since 2001, and has taught various Masters and Expert University Courses within the University of Seville.

Since 2004, Dr Anguís Climent has been Director and Coordinator of various educational innovation projects, funded by the University of Seville, the findings of which have been presented at conferences and have appeared in special publications.

Published by OmniaScience (www.omniascience.com)

Journal of Technology and Science Education, 2012 (www.jotse.org)

Article's contents are provided on a Attribution-Non Commercial 3.0 Creative commons license. Readers are allowed to copy, distribute and communicate article's contents, provided the author's and Intangible Capital journal's names are included. It must not be used for commercial purposes. To see the complete licence contents, please visit http://creativecommons.org/licenses/by-nc/3.0/es/