Journal of Technology and Science Education

• Home
• About
• Login
• Archives
• Submissions
• Publication Fee
• Indexing & Statistics

Home > Vol 6, No 3 (2016) > Cohen

ONLINE QUIZZES IN A VIRTUAL LEARNING ENVIRONMENT AS A TOOL FOR FORMATIVE ASSESSMENT

1Tel-Hai College (Israel)

2Golan Research Institute (Israel)

Received January 2016

Accepted May 2016

Abstract

Assessment in education employing web tools, also known as e-assessment, deals with the effective use of technology to support successful instruction. The aim of this study was to investigate learning outcomes and the students' attitudes to online Moodle quizzes in order to improve instructional design. The research population included 204 college students enrolled in life sciences who were participating in an introductory physics course. A blended learning model was used, based on large, traditional face-to-face lectures, practice sessions held with smaller groups of about 25 students and a rich Moodle learning environment. The students' knowledge and understanding were assessed weekly throughout the semester, using two different methods: three ordinary short written tests and online quizzes in the Moodle environment. The students' attitudes towards the online quizzes as compared to the written tests were investigated by questionnaire. Results indicate that both the average grade on written tests and the average grade on online quizzes were found to be significant predictors of the grade on the final exam. Students significantly improved their scores and greatly shortened their performance time on the last attempts of the online quiz, as compared to their first attempts. The investigation into the students’ attitudes towards online quizzes reveals a generally positive attitude. Learning outcomes and the students' attitudes to online Moodle quizzes were considered to improve instructional design, which demonstrates formative assessment in higher education.

 

 

Keywords – Formative assessment, Higher education, Online quizzes, Physics education, Virtual learning environment.

----------

1. Introduction and Theoretical Background

Higher education is changing. Most if not all students now have access to learning materials via the Internet (Brown, 2006). Students expect and need different things from their instructors. They need to be guided through their learning process. They expect the instructor to teach them how to build their knowledge and comprehension of the given subject (Shea, Pickett & Pelz, 2003; Clark, Kirschner & Sweller, 2012). Furthermore, they expect them to use technology to do this. Instructors must meet this challenge and integrate technology into their courses, effectively facilitating the students' learning process. The instructors must therefore be confident that the investment is worthwhile – that the effective use of new tools leads to better outcomes, in both students' attitudes and in their professional understanding. According to Trow (1999), studies are needed to assist teaching staffs in high schools, colleges and universities to recognize the possibilities of using technology as an effective tool in the teaching process and to enhance the positive outcomes that it may bring about. The present study falls into this category. We studied the implementation of Moodle quizzes in a physics college course for life science students. The focus was on assessing the role of the online quizzes in the learning environment in a blended learning model. Our investigation included three stages. We first examined the role of computerized assessment through online quizzes in the academic course evaluation process. In this stage, we investigated whether the online quizzes and the written tests used during the course predicted the students' scores on the final written exam. In the second stage, we focused on the students' behaviors and performance during the online quiz. In particular, we tried to understand whether many attempts on an online quiz served any benefit for the students. We examined possible correlations between the number of attempts and student outcomes. We believe that the quiz design, which combines a randomization factor and many allowed attempts, offers an opportunity for students to take responsibility for their own learning. The immediate feedback encourages the students to reflect on their understanding, to implement whatever methods are needed to gain more understanding, and to make another attempt to ascertain whether they have actually improved their knowledge. The last stage included the investigation of student attitudes towards the online quizzes. Another possible benefit we studied focused on the use of symbol manipulation when solving problems on a quiz, which encourages the students to focus on problem-solving strategies rather than finding a final numeric answer. The three stages are presented in Figure 1.

Figure 1. Investigation stages

The concept of a blended learning model refers to a learning environment that combines face-to-face learning sessions with self-paced, computer-assisted learning activities (Alonso, López, Manrique & Viñes, 2005). The learning environment is the entire space in which learning occurs. It includes all the resources (physical components, time and society) in this space, which may affect the learning that takes place. Since all learning is rooted in the conditions of the specific environment, the environment must be designed in a rich, complex and dynamic manner that invites interaction with different agents and fosters deep understanding. The learning environment should also include innovative technologies that can be used to achieve educational goals and improve the quality of the learning processes (Bransford, Brown & Cocking, 2000).

Technology has almost always been part of the teaching and learning process. The effective integration of technology in the classroom has been the subject of many studies, especially since Internet use has become so common. New terms, such as interactive learning environment (ILE), information and communication technologies (ICT) and blended learning have been introduced (Johnson, Rickel & Lester, 2000; Novak, Patterson & Gavrin, 2000; Garrison & Kanuka, 2004; Mor & Winters, 2007; Yen & Lee, 2011). Several studies have been conducted to investigate how the use of certain technologies has affected student outcomes. Belcher and Dori (2005) showed how the Technology-Enabled Active Learning (TEAL) project at MIT positively affected student outcomes in a physics course. The project improved the students' conceptual understanding, reduced the failure rate and improved other learning dimensions. Beatty and Gerace (2009) emphasized that technology alone is not enough; it must be used effectively to support successful pedagogy. They developed and tested a technology-enhanced pedagogy based on the classroom response system (CRS), also known as the clicker system. The authors' important distinction between technology itself and its effective use means that instructors who want to improve their teaching should not be satisfied with just adding technological tools to their classes. They must carefully plan the ways in which they can integrate these tools into their teaching. This should lead to the rethinking of the entire attitude towards the teaching process, encouraging more self-learning and student-centered teaching. It is not surprising, therefore, that studies have been written about the teaching staff: their attitudes towards integrating new technology and the need to prepare and guide them towards satisfactory results (e.g., Shemla & Nachmias, 2006; Pundak & Rozner, 2008; Pundak, Herscovitz, Shacham & Wiser-Biton, 2009; Liu, 2011).

Design approaches in technology-enhanced learning have become popular in educational research. This approach is concerned with the design of learning processes, taking into account the complexities involved in educational settings (Wang & Hannafin, 2005; Mor & Winters, 2007). Design-based research in education includes the systematic exploration of critical elements in learning environments in terms of their effect on the learning process (Collins, 1992). Kali (2006) used design-based methods to investigate the Design Principles Database (DPD) as a mechanism for supporting a community of learners who are exploring educational technologies. The learning environment is defined by Kali (2006) as a system that incorporates a set of artifacts, along with a navigation system and curriculum materials; the term 'feature' is used to refer to any technological artifact in the learning environment that targets advanced learning.

This research study investigated online Moodle quizzes in order to understand the role of this technology component as part of the learning environment in a blended learning model. Moodle (Modular Object Oriented Development Learning Environment) is a popular open-source course management system (CMS), also known as a learning management system (Romero, Ventura & García, 2008). Moodle was designed as a pedagogical platform to promote and assist learning in several dimensions (Katsamani, Retalis & Boloudakis, 2012). It offers numerous ways of facilitating information sharing, collaborative platforms through forums and chats, storage areas for course materials and student data, thus providing powerful and easy-to-use tools for active learning and for student assessment (Psycharis, Chalatzoglidis & Kalogiannakis, 2013).

 In our field of research, online Moodle quizzes have been used for formative assessment, which emphasizes the operations needed to improve student achievement in relation to desirable outcomes. The goal of formative assessment is to gather feedback that can be used by the instructor and the students to guide improvements in ongoing teaching and learning (Brown, Race & Smith, 1996). The integration of assessment practices embedded in instruction is critical for improving teaching and learning. Research implies that learning can be improved when teachers use instructional strategies with frequent and ongoing assessment of the students’ developing understanding (Treagust, Jacobowitz, Gallagher & Parker, 2001). This assessment tool enables students to reflect on their work over time, allows them to share their thinking, gives them meaningful feedback from their teacher and their peers, and provides teachers with a variety of evidence to support their final evaluation (McGuire, 2005).

Assessment in education using web tools, also known as e-assessment, has been the subject of several studies (Pundak, Maharshak & Rozner, 2004; McGuire, 2005; Nicol, 2009; Pachler, Daly, Mor & Mellar, 2010; Stödberg, 2012). Gikandi. Morrow and Davis (2011) presented a review of the literature related to effective formative e-assessment, and emphasized the role of feedback to both students and to teachers. Many studies have focused on e-assessment in science education, especially in higher education (Cole & Todd, 2003; Hauk & Segalla, 2005; Nirmalakhandan, 2007), while others have focused on the subject of physics (Mestre, Hart, Rath & Dufresne, 2002; Martin-Blas & Serrano-Fernandez, 2009). Physics, as a special case in upper level science education, is considered challenging for both students and teachers (Angell, Guttersrud, Henriksen & Isnes, 2004; Chandra & Watters, 2012). Online quizzes are used as a means of e-assessment in several design approaches. In some studies, they consist mainly of multiple-choice or true-false questions to test conceptual understanding (Martin-Blas & Serrano-Fernandez, 2009). Sometimes quizzes contain problems, and are mainly used as a substitute for written homework, often allowing repetition (Bonham, Deardorff & Beichner, 2003; Cole & Todd, 2003; Hauk & Segalla, 2005; Liberatore, 2011). In these studies, the online homework was found to have a negligible to moderate advantage over written homework in terms of course outcomes, while student attitudes were seen to be generally positive. Kortemeyer, Kashy, Benenson and Bauer (2008) have expressed concern that multiple attempts at online homework might lead students to adopt a trial-and-error approach.

In this study, we investigated the role of online Moodle quizzes in the learning environment of a blended learning model (based on the stages presented in Figure 1). We posed the following main research questions:

• •Are the online quiz grades related to other methods of assessment (a final written exam and written tests)?

• •What are student behaviors during the online quizzes?

  • ◦How many attempts did the students actually make on the multiple-attempts online quiz?

  • ◦Was there a noticeable change in successive attempts (grades and solving time)?

• •What are the students' attitudes towards the online quizzes?

  • ◦To what extent did the students use symbol manipulation when solving problems on the online quizzes?

  • ◦How many attempts should be allowed, according to student perceptions?

2. Methods

2.1. Context of the study

The main goal of this research was to study the learning outcomes and attitudes of students with regard to online Moodle quizzes in order to improve instructional design in an introductory physics course for life sciences at Tel-Hai College, in the Upper Galilee region of northern Israel. The physics course (algebra-based mechanics) was taught by the first author, according to a blended learning model. It was based on large, traditional face-to-face lectures, practice sessions in smaller groups of about 25 students and a rich Moodle learning environment. The lectures focused on conceptual understanding of mechanics, while the practice sessions were used to develop problem-solving skills. The Moodle environment that accompanied the course included some attractive features both for students and instructors. The course site was built with several supportive tools for learning: PowerPoint presentations, lecture videos, links to java animations that were demonstrated during classes, links to other physics sites, such as short lectures in TED format, a weekly set of problems to be used for self-assessment, full or partial solutions to these problems, another weekly set of problems that were worked out during practice sessions with tutors, a discussion forum and four quizzes that were used for formative assessment, together with written tests. In each quiz, we included some conceptual questions, often as multiple choice questions, as well as regular physics problems, where an unknown numerical value was required as the answer.

The students' knowledge and understanding were assessed weekly throughout the semester, using two different methods: ordinary, short written tests restricted to 20 minutes were given during the second, fourth and tenth weeks of the semester. In four other weeks, the students were required to answer an online quiz in a Moodle environment. The online quizzes were open for one week, and students could choose where and when to take them. Once a student started a quiz, there was a mandatory limit of one hour prior to submission. For the first three quizzes, the students did not receive their grade until the end of the week, and were allowed only one attempt. These online quizzes were a substitute for the written tests, and were similar to them in that the feedback was given after a few days, and only one attempt was allowed. The last quiz was open, with five attempts allowed. After each attempt, limited, but immediate feedback was given, indicating the overall score and which questions were answered correctly. The quizzes had a randomization factor permitted by the Moodle environment, so that in the next attempt, the student was required to answer the same questions as in the previous attempt, but with different numeric results. Fuller and richer feedback, including verbal feedback and clues about how the questions could be answered, was given at the end of the week.

2.2. Research instruments

The first stage in this research was based on the analysis of student grades. Two instruments were used in the second and the third stages in the study:

• •Data from log files that were retrieved from Moodle, to examine the students' behavior: how many attempts they made, whether there was an improvement between the first and last attempts, how much time the students spent on a single attempt, whether there was a change from the first attempt to the last, and whether there was any correlation between the quiz outcomes and the score on the final exam.

• •A questionnaire distributed towards the end of the semester to assess the students' attitudes towards the online quizzes, as compared to the written tests. The questionnaire consisted of four sections. The first included background information, specifically gender and department of study. The second comprised 14 statements that were divided into four categories: feelings when conducting the online quiz, attitudes toward the quiz and the preparation required for success, contributions to self-assessment and the use of symbol manipulation when solving the problems on the quiz (see the next paragraph). The students were required to rank each statement in each category on a scale of 1 to 5: 5. “strongly agree,” 4. “agree,” 3. “undecided,” 2. “disagree,” 1. “strongly disagree.” The Cronbach alpha reliability coefficient for this section was α = 0.887. We calculated the means and standard deviations for the scores in each category. In the third section, the students were asked whether they preferred online quizzes or written tests, and were required to explain their choice in their own words, and explicitly relating to the contribution they made to their learning and understanding. The fourth section was similar to the third, but was related to the number of attempts permitted on the online quiz, as recommended by the students.

We were particularly interested in assessing the use of algebraic symbol manipulation while solving a problem on the quiz. The quiz consisted of a few physics questions, in which a system is described, sometimes with a picture, and a specific physical entity is to be calculated. The students had to determine which physics laws applied to the described system, write the appropriate equation based on the law, and calculate the missing entity from the equation, using algebraic manipulations. Most of this process is not gauged by the learning environment, as only the numeric final answer is tested. Many students find it easier (Simon & Simon, 1978) to manipulate numbers instead of symbols, while experts manipulate symbols and only at the end substitute the symbols with the given numbers. The latter strategy has an advantage over a quiz that allows several attempts, such as the last online quiz. This is because the questions in the following attempts were the same, so that the laws and equations applied were also the same, but the numbers, and therefore the final numeric answers, were different. Using the symbol manipulation strategy could save a lot of time for questions that were answered correctly in the earlier attempt, but had to be answered again with different numbers in the following attempts. We wanted to determine whether the students were aware of this advantage, and if they tended to use this strategy.

2.3. Study participants

The analysis in the first stage in this study (as presented in Figure 1) was based on the scores of 173 students from the life sciences department (56% males, 44% females) who participated in the course. The second and the third stages were based on 120 students who answered the attitudes questionnaire that was administered towards the end of the semester. Sixty-six of them (55%) were male and 54 (45%) were female. The students came from four different departments: Biotechnology (37%), Animal Sciences (10%), Environmental Sciences (24%), and Food Sciences (23%). Eight students did not mention their department. It is important to emphasize that there is a less than absolute overlap among the participants in the different stages.

2.4. Data Analysis

The main purpose of the online quizzes was to serve as formative assessment for students, i.e., to aid students in understanding the course material. In order to determine whether our expectations were fulfilled, we analyzed the data retrieved from the Moodle log files, as the Moodle environment keeps track of every activity by every student in the environment. Some of the information we looked for was straightforward. For example, the quiz grades were exported to an excel file and later added to the SPSS tool to be statistically processed and compared to other features, such as the final grades, by means of a linear regression. Other types of information required programming tools for their retrieval. For example, the number of attempts each student actually submitted had to be extracted from a file containing the scores for every attempt. As there were more than 400 attempts – meaning more than 400 lines in the file, we used programming to extract this information. Although data mining tools do exist to deal with such problems (see for example Romero et al., 2008; Siemens & Long, 2011), we found it easier to write a simple program in the C programming language by ourselves. We used linear regression to determine whether the grades on the online quizzes predicted the grade on the final exam of the course. We examined other possible predictors of the final grade, such as gender, grades on written tests, the number of attempts on the last quiz, time spent taking the quizzes, and the attitudes of students toward the online quizzes. An important part of formative assessment is the ongoing improvement gained from the repetitive nature of solving a problem, receiving immediate feedback, reflecting, and trying again. To determine whether the students met this challenge, we looked for indications of improvement from the first attempt to the last. We examined two indicators – the grade on the attempted quiz and the time spent trying to complete the quiz during each specific attempt; to do so, we retrieved these data for the first and last attempts at the last online quiz, for every student. Note that the last attempt was the same as the first attempt for students who took the quiz only once. For each indicator, we compared the first and last attempts using a t-test. All statistical analyses were conducted using SPSS software.

3. Results

In the first stage of the data analysis, we investigated the predictors of success on the final course exam. We used a linear regression to test four predictors: average grade on written tests, average grade on online quizzes, gender and science department. Results indicate that gender was not a significant predictor in the model (p>0.05). The science department was found to be atavistically significant (F = 3.92, P=0.050), but the explained variance value (r2) was low (about only 4%), and therefore we did not include it in the model. Both the average grade on written tests and the average grade on online quizzes were found to be significant predictors of the grade on the final exam, as shown in Table 1. High scores on the written tests during the course, as well as on the online quizzes, predicted significantly high scores on the final written exam.

Standardized equation	Regression equation	Explained variance value r2	Significance	Predictor
Zy=0.541*zx	y=37.520 + 0.419x	29%	F=40.036 p=0.000	Average grade on written tests
Zy=0.450*zx	y=23.264 + 0.521x	20%	F=24.677 P<0.003	Average grade on online quizzes

Table 1. Regression models for predictors of the final grade

After identifying the role played by the online quizzes in the evaluation process, we examined student behaviors as they completed the quiz. The following results compared the first and last attempts on the last online quiz. As noted, the last attempt was the same as the first for students who attempted to complete the quiz only once. There was a significant difference in both the time taken to complete the quiz and the grade achieved (both with large effect size [Cohen’s d]), suggesting an improvement in students' performance. Students significantly improved their scores (Meanfirst attempt=53.38, S.D=31.12, Meanlast attempt =82.39, S.D=22.99, t=-10.7, P<0.001, Cohen’s d=1.380) and greatly shortened their performance time (Meanfirst attempt=38.85, S.D=15.26, Meanlast attempt =26.13, S.D=13.49, t=-7.09, P<0.001, Cohen’s d=0.915). The results are presented in Figures 2 and 3.

Figure 2. First attempt vs. last attempt on the last online quiz – Student grades

Figure 3. First attempt vs. last attempt on the last online quiz – Student response time

3.1. Student attitudes

Close inspection of the students’ attitudes towards online quizzes reveals a generally positive attitude. Table 2 presents the results.

Example of statement	S.D.	Mean Min=1, Max=5	Cronbach’s α	Category
"I feel relaxed when I complete the online quiz"	1.02	3.36	0.89	Feelings
"In preparation for an online quiz, I do exercises, just like I prepare for a written test"	0.79	4.06	0.81	Behaviors
"Completing the quiz helped me figure out my problems with the course material"	0.88	3.44	0.89	Contribution to self-assessment
"I use symbol manipulation when I complete the quiz"	1.01	3.35	0.83	Use of symbol manipulation

Table 2. Student attitudes toward online quizzes, organized by category

No differences were found with respect to gender. A positive, statistically significant relationship was found between students' attitudes and their scores on online quizzes, but the explained variance value (r2) was low (F=16.065, p=0.000, r2=13%, y = 41.938+ 8.537x, Zy = 0.361*zx).

No significant relation (as tested by Pearson tests) was found between the level of symbol manipulation use (as reported by students on the questionnaire) and the number of attempts or the time taken to complete the quiz (as measured by the Moodle system).

In answer to the summary question: "If you had the choice, which would you prefer – an online quiz or a written test?" 76% chose online quizzes. In the open-ended question that followed, asking for an explanation for the above answer, some students (7) commented that they would prefer a combination of online quizzes and written tests, as was actually given during the semester. We have left the analysis of other open-ended answers for a later study.

In order to answer research questions 3 to 6, we analyzed both the relevant question on the questionnaire and students' performance in the Moodle system.

3.2. Recommended vs. actual number of attempts

The question "How many attempts at an online quiz would you recommend?" was answered by 112 students, with the following frequency: 4 (3.57%) recommended only one attempt, 39 (34.82%) recommended 2 attempts, 45 (40.18%) recommended 3 attempts, 14 (12.50%) recommended 4 attempts and 10 (8.93%) recommended 5 attempts for each quiz (Figure 1). It is interesting to compare these results with the actual number of attempts that the students submitted for the last quiz. One hundred and seventy-three students submitted 418 attempts at the last online quiz, with the following frequency: 49 (28.32%) submitted one attempt, 52 (30.06%) submitted 2 attempts, 36 (20.81%) submitted 3 attempts, 23 (13.29%) submitted 4 attempts and 13 (7.51%) submitted 5 attempts. The most prominent difference between actual and recommended frequencies is related to one attempt. While a negligible number of students recommended 1 attempt, more than 28% actually performed only one attempt. The results are presented in Figure 4.

Figure 4. Frequency of recommended and actual number of attempts on an online quiz

4. Discussion

Our research focused on online quizzes as a learning object and investigated their role in instructional design. Wiley (2000) and Lehman (2007) defined a ‘learning object’ as a reusable digital component that can be selectively applied alone or in combination by computer software, learning facilitators, or the learners themselves to meet individual needs for learning or support their performance. The online quizzes in our instructional design serve as a practice and review component (one of the three types identified by Shepherd in 2006), with an emphasis on the formative assessment process.

The online quizzes provide information that yields feedback for making necessary modifications to improve teaching and learning. We focused on promoting a better instructional design for a physics course in higher education, demonstrating formative assessment as Daly, Pachler, Mor and Mellar (2010) have proposed. They suggested five strategies for formative assessment:

• •engineering effective learning tasks that elicit evidence of learning;

• •providing feedback that moves learners forward;

• •clarifying, understanding and sharing learning intentions and criteria for success;

• •activating students as owners of their own learning; and

• •activating students as instructional resources for one another.

We will now discuss the main results found in our study with respect to our design approach to online quizzes. We found a significant correlation between quiz grades (written and online) and final exam grades. This finding increases the sense of reliability when it comes to integrating computerized assessment tools in higher education as part of the students' assessment process. These computerized tools allow students to flexibly choose the time and place to act. No significant difference was found in relation to gender, in contrast to what was found in studies by Kortemeyer (2009) and Sanders and Morrison-Shetlar (2001).

No significant relationship was found between the level of use of symbol manipulation and the number of attempts or the time taken to complete the quiz. This might indicate that the development of generalization in problem solving requires explicit intervention by instructors. Several attempts offer the advantage of using symbol manipulation, because the questions in the following attempts were the same. Thus, the laws and equations applied were the same, but the numbers, and therefore the final numeric answers, were different. The results indicate that most of the students were probably not aware of this advantage and they did not tend to use this strategy. The results strengthen Simon and Simon’s (1978) claim that, unlike experts, students find it easier to manipulate numbers as opposed to symbols during problem solving. The results focused attention on the students’ mathematical thinking and allowed instructors to improve learning and teaching process.

We found important differences between the number of attempts the students recommended to be allowed and the actual number of attempts they made. It is interesting to focus on the restriction to one attempt. According to this approach, the quiz is used as a test that evaluates knowledge and skills, but does not allow for correcting the wrong answers. About 72% of the students attempted the quiz more than once, and significantly improved their grades. This can be explained by the fact that students do better when they feel confident (Pintrich & De Groot, 1990). Knowing that there are more attempts available gives them the confidence they need to perform better. However, that does not necessarily mean that they intend to use the available attempts to improve learning and try to achieve a better grade. This last behavior is related to responsibility, i.e., the willingness to take responsibility for one's learning. Only 4% of the students recommended that the quizzes be given with one attempt, while 28% of them actually attempted the quiz only once. Most of the students in this study recognized the potential benefits, took responsibility and attempted to achieve a better grade. It is useful to note that no significant difference was found between the average grade on the quiz and the number of actual attempts. This finding suggests that there are some students who were satisfied with their low grade, or assumed that another attempt would not help them achieve a better grade and therefore did not make any effort to improve it. Instructors can use this tool to identify these students and offer them the help they need to learn better by using alternative teaching methods. Notably, only 21% of the students who answered the questionnaire recommended more than three attempts at the quiz. A possible explanation for this is that the students were aware of the phenomenon that Kortemeyer et al. (2008) referred to as "turning thinkers into guessers", and they therefore preferred to have a small number of allowed attempts. It seems that students understand the usefulness of the suggested design approach, and do not want to be tempted by many attempts that can lead them into a trial-and-error cycle, rather than guiding them towards a better understanding.

Other useful results found in this study are in line with the literature. These include the positive attitudes of students towards online quizzes and the relevance of the online quizzes to student achievement (Sanders & Morrison-Shetlar, 2001; Liberatore, 2011). Furthermore, we found an improvement in the students' performance throughout the learning process, from the first attempt at completing the quiz to the last attempt. The mean grade improved and the time spent completing the quiz decreased. This reflects the willingness of the students to enter into the meaningful learning process offered by the specific course design.

Educational institutions are increasingly looking for effective and productive technological aids for the development of new models of teaching and learning. A wide range of design options are offered, ranging from a traditional design that includes face-to-face meeting, to hybrid or blended models and e-learning courses. Our research emphasizes the role of online quizzes as a tool for formative assessment that enables greater interaction between the teachers and their students, and allows learners to make decisions concerning their studies and build their knowledge based on their experiences. Nevertheless, efforts are still needed in this area. The physics course design presented some formative components that were investigated, for the first time, in our research. Continued research will lead to further development in the area of course design that will also consider other formative components, such as skill performance tools. In order to improve course design, we suggest using further qualitative research tools, such as interviews, in addition to those that were used in this study. This will provide greater insight into how students feel (resistance or support) about the course components.

In this article, we describe the successful implementation of online Moodle quizzes in a first-year introductory, algebra-based physics course for the life sciences. One quiz was given with five allowed, time-restricted attempts, with feedback following each attempt. This design approach using online quizzes is slightly different from the conventional design approach described in many articles. In the conventional design approach, online quizzes are used as online homework, with a due date, but there is no time restriction for an individual attempt. We believe that this difference is important, because it changes the way the student approaches the quiz. Using our approach, the student has to prepare for the quiz, cover the relevant topics, and do exercises before attempting to take the quiz.

References

Alonso, F. López, G., Manrique, D., & Viñes, J.M. (2005). An instructional model for web-based e-learning education with a blended learning process approach. British Journal of Educational Technology, 36(2), 217-235. http://dx.doi.org/10.1111/j.1467-8535.2005.00454.x

Angell, C., Guttersrud, Ø., Henriksen, E.K., & Isnes, A. (2004). Physics: Frightful, but fun. Pupils' and teachers' views of physics and physics teaching. Science Education, 88(5), 683-706. http://dx.doi.org/10.1002/sce.10141

Beatty, I.D., & Gerace, W.J. (2009). Technology–enhanced formative assessment: A research-based pedagogy for teaching science with classroom response technology. Journal of Science Education and Technology, 18, 146-162. http://dx.doi.org/10.1007/s10956-008-9140-4

Belcher, J., & Dori, Y. (2005). How does technology-enabled active learning affect undergraduate students’ understanding of electromagnetism concepts?. The Journal of the Learning Sciences, 14(2), 243-279. http://dx.doi.org/10.1207/s15327809jls1402_3

Bonham, S.W., Deardorff, D.L., & Beichner, R.J. (2003). Comparison of student performance using web and paper-based homework in college-level physics. Journal of Research in Science Teaching, 40(10), 1050-1071. http://dx.doi.org/10.1002/tea.10120

Bransford, J.D., Brown, A.L., & Cocking, R.R. (2000). How People Learn. Washington, DC: National Academy Press.

Brown, J.S. (2006). New learning environments for the 21st century: Exploring the edge. Change: The Magazine of Higher Learning, 38(5), 18-24.‏ http://dx.doi.org/10.3200/CHNG.38.5.18-24

Brown, S., Race, P., & Smith, B. (1996). 500 tips on assessment. London: Kogan Page.

Chandra, V., & Watters, J.J. (2012). Re-thinking physics teaching with web-based learning. Computers & Education, 58(1), 631-640.‏ http://dx.doi.org/10.1016/j.compedu.2011.09.010

Clark, R., Kirschner, P.A., & Sweller, J. (2012). Putting students on the path to learning: The case for fully guided instruction. American Educator, Spring, 6-11.

Cole, R.S., & Todd, J.B. (2003) Effects of web-based multimedia homework with immediate rich feedback on student learning in general chemistry. Journal of Chemical Education, 80(11), 1338‑1343. http://dx.doi.org/10.1021/ed080p1338

Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O'Shea (Eds.), New Directions in Educational Technology (pp. 15-20). Berlin: Springer-Verlag. http://dx.doi.org/10.1007/978-3-642-77750-9_2

Daly, C., Pachler, N., Mor, Y., & Mellar, H. (2010). Exploring formative e-assessment: Using case stories and design patterns. Assessment & Evaluation in Higher Education, 35(5), 619-636.‏ http://dx.doi.org/10.1080/02602931003650052

Garrison, D.R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95-105.‏        http://dx.doi.org/10.1016/j.iheduc.2004.02.001

Gikandi J.W., Morrow, D., & Davis, N.E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333-2351. http://dx.doi.org/10.1016/j.compedu.2011.06.004

Hauk, S., & Segalla, A. (2005). Student perceptions of the web-based homework program WeBWorK in moderate enrollment college algebra classes. Journal of Computers in Mathematics and Science Teaching, 24(3), 229-253.‏

Johnson, W.L., Rickel, J.W., & Lester, J.C. (2000). Animated pedagogical agents: Face-to-face interaction in interactive learning environments. International Journal of Artificial Intelligence in Education, 11(1), 47-78.

Kali, Y. (2006). Collaborative knowledge building using the Design Principles Database. Computer-Supported Collaborative Learning, 1, 187-201. http://dx.doi.org/10.1007/s11412-006-8993-x

Katsamani, M., Retalis, S., & Boloudakis, M. (2012). Designing a Moodle course with the CADMOS learning design tool. Educational Media International, 49(4), 317-331.‏ http://dx.doi.org/10.1080/09523987.2012.745771

Kortemeyer, G. (2009). Gender differences in the use of an online homework system in an introductory physics course. Physical Review Special Topics-Physics Education Research, 5(1), 010107.‏ http://dx.doi.org/10.1103/PhysRevSTPER.5.010107

Kortemeyer, G., Kashy, E., Benenson, W., & Bauer, W. (2008). Experiences using the open-source learning content management and assessment system LON-CAPA in introductory physics courses. American Journal of Physics, 76(4), 438-444.‏ http://dx.doi.org/10.1119/1.2835046

Lehman, R. (2007). Learning objects repositories. New Directions for Adult and Continuing Education, 113, 57–66. http://dx.doi.org/10.1002/ace.247

Liberatore, M.W. (2011). Improved student achievement using personalized online homework for a course in material and energy balances. Chemical Engineering Education, 45(3), 184-190.‏

Liu, S.H. (2011). Factors related to pedagogical beliefs of teachers and technology integration. Computers & Education, 56(4), 1012-1022.‏ http://dx.doi.org/10.1016/j.compedu.2010.12.001

Martin-Blas, T., & Serrano-Fernandez, A. (2009). The role of new technologies in the learning process: Moodle as a teaching tool in Physics. Computers & Education, 52, 35-44. http://dx.doi.org/10.1016/j.compedu.2008.06.005

McGuire, L. (2005). Assessment using new technology. Innovations in Education and Teaching International, 42(3), 265-276. http://dx.doi.org/10.1080/01587910500168025

Mestre, J., Hart, D.M., Rath, K.A., & Dufresne, R. (2002). The effect of web-based homework on test performance in large enrollment introductory physics courses. Journal of Computers in Mathematics and Science Teaching, 21(3), 229-251.‏

Mor, Y., & Winters, N. (2007). Design approaches in technology-enhanced learning. Interactive Learning Environments, 15(1), 61-75. http://dx.doi.org/10.1080/10494820601044236

Nicol, D. (2009). Assessment for learner selt-regulation: Enhancing achievement in the first year using learning technologies. Assessment & Evaluation in Higher Education, 34(3), 335-352. http://dx.doi.org/10.1080/02602930802255139

Nirmalakhandan, N. (2007). Computerized adaptive tutorials to improve and assess problem-solving skills. Computers & Education, 49(4), 1321-1329.        http://dx.doi.org/10.1016/j.compedu.2006.02.007

Novak, G.M., Patterson, G.T., & Gavrin, A.D. (2000). Just in time teaching: Blending active learning with web technology. Upper Saddle River, NJ: Prentice Hall‏.

Pachler, N., Daly, C., Mor, Y., & Mellar, H. (2010). Formative e-assessment: Practitioner cases. Computers & Education, 54(3), 715-721.‏ http://dx.doi.org/10.1016/j.compedu.2009.09.032

Pintrich, P.R., & De Groot, E.V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33-40.‏ http://dx.doi.org/10.1037/0022-0663.82.1.33

Psycharis, S., Chalatzoglidis, G., & Kalogiannakis, M. (2013). Moodle as a learning environment in promoting conceptual understanding for secondary school students. Eurasia Journal of Mathematics, Science & Technology Education, 9(1), 11-21.‏

Pundak, D., Herscovitz, O., Shacham, M., & Wiser-Biton, R. (2009). Instructors' attitudes toward active learning. Interdisciplinary Journal of E-Learning and Learning Objects, 5, 215-232.

Pundak, D., Maharshak, A., & Rozner, S. (2004). Successful pedagogy with web assignments checker. Journal of Educational Technology Systems, 33(1), 67-80. http://dx.doi.org/10.2190/RQXE-RFB4-V254-F8MP

Pundak, D., & Rozner, S. (2008). Empowering engineering college staff to adopt active learning methods. Journal of Science Education and Technology, 17(2), 152-163.‏ http://dx.doi.org/10.1007/s10956-007-9057-3

Romero, C., Ventura, S., & García, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368-384.‏ http://dx.doi.org/10.1016/j.compedu.2007.05.016

Sanders, D.W., & Morrison-Shetlar, A.I. (2001). Student attitudes toward Web-enhanced instruction in an introductory biology course. Journal of Research on Computing in Education, 33(3), 251-262. http://dx.doi.org/10.1080/08886504.2001.10782313

Shea, P.J., Pickett, A.M., & Pelz, W.E. (2003). A follow-up investigation of “teaching presence” in the SUNY Learning Network. Journal of Asynchronous Learning Networks, 7(2), 61-80.

Shepherd, C. (2006). Objects of Interest. Available online at:         http://olc.gre.ac.uk/ET/ELD/KNTI/etutres.NSF/76cf225430685dbc8025651a00759c95/fa145009572989a580256c470053b195/$FILE/lrngobjects.pdf. (Last access date: February 5th, 2015).

Shemla, A., & Nachmias, R. (2006). How do lecturers integrate the Web in their courses? Web-supported courses at Tel-Aviv University. World Conference on Educational Multimedia, Hypermedia and Telecommunications, 1, 347-354.‏

Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46(5), 30-32.‏

Simon, D.P., & Simon, H.A. (1978). Individual differences in solving physics problems. In R. Siegler (Ed.), Children's thinking: What develops? (pp. 325-348). Hillsdale, NJ: Lawrence Erlbaum Associates.

Stödberg, U. (2012). A research review of e-assessment. Assessment & Evaluation in Higher Education, 37(5), 591-604.‏ http://dx.doi.org/10.1080/02602938.2011.557496

Treagust, D.F., Jacobowitz, B., Gallagher, J.L., & Parker, J. (2001). Using assessment as a guide in teaching for understanding: A case study of a middle school science class learning about sound. Science Education, 85, 137-157.                                                http://dx.doi.org/10.1002/1098-237X(200103)85:2<137::AID-SCE30>3.0.CO;2-B

Trow, M. (1999). Lifelong learning through the new information technologies. Higher Education Policy, 12(2), 201-217.

Wang, F., & Hannafin, M.J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23.        http://dx.doi.org/10.1007/BF02504682

Wiley, D.A. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D. A. Wiley (Ed.), The Instructional Use of Learning Objects. Available online at: http://wesrac.usc.edu/wired/bldg-7_file/wiley.pdf. (Last access date: February 5th, 2015).

Yen, J.C., & Lee, C.Y. (2011). Exploring problem solving patterns and their impact on learning achievement in a blended learning environment. Computers & Education, 56(1), 138-145. http://dx.doi.org/10.1016/j.compedu.2010.08.012

---