Journal of Technology and Science Education

• Home
• About
• Login
• Archives
• Submissions
• Publication Fee
• Indexing & Statistics

Home > Vol 11, No 2 (2021) > Divayana

DEVELOPMENT OF MATERIAL CONTENTS AND ONLINE ASSESSMENT BASED ON THE SEVIMA EDLINK PLATFORM FOR ONLINE LEARNING OF PROGRAM EVALUATION SUBJECT DURING COVID-19 PANDEMIC IN INDONESIA

1Department of IT Education, Universitas Pendidikan Ganesha (Indonesia)
2Department of Electrical Education, Universitas Pendidikan Ganesha (Indonesia)

Received February 2021

Accepted July 2021

Abstract

One of the free platforms made by IT companies in the education sector in Indonesia can be used to facilitate online learning at home during the Covid-19 pandemic. The platform is called the SEVIMA EdLink. This platform needs to be known by academics and the wider community of education in the world. This platform provides facilities to make it easier for users to input material content and online assessment forms. The purpose of this research was to demonstrate the development of material contents of the ‘Program Evaluation’ subject and its assessment form that was embedded into the SEVIMA EdLink platform. The approach used in this research was a development based on the 4D model (Define, Design, Develop, and Disseminate). Subjects who were involved in testing the material contents were 29 students. The measuring instruments used in testing the material contents were questionnaires. Subjects who were involved in the content validity test for multiple-choice test questions were six experts, and two experts were involved in the content validity test for essay test questions. The tools used in testing the content validity of multiple-choice test questions or essay test questions were checklist documents. The analysis technique of the test results of the material contents was descriptive quantitative. The analysis technique for multiple-choice test questions and essay test questions was the comparison of content validity test results with the standard scores of content validity based on Guilford. This research results showed the quality percentage of material contents in the subject of ‘Program Evaluation’ was a good category. The content validity results of the multiple-choice test questions and essay test questions were both in the excellent category.

Keywords – Material contents, Online assessment, SEVIMA EdLink, Program evaluation, Covid-19 pandemic.

To cite this article:

Divayana, D.G.H., Adiarta, A., & Suyasa, P.W.A. (2021). Development of material contents and online assessment based on the SEVIMA EdLink platform for online learning of program evaluation subject during Covid-19 pandemic in Indonesia. Journal of Technology and Science Education, 11(2), 498‑512. https://doi.org/10.3926/jotse.1243

----------

1. Introduction

The Covid-19 pandemic makes the learning and assessment process can’t be carried out directly in class. One of the efforts that can be made to carry out the learning and assessment process optimally is to prepare material contents and assessments based online. Many platforms provide material contents and assessments based online that can be easily accessed via the internet. Some of those platforms included: Quipper School, Schoology, Moodle, Kelase, Zenius, Google Classroom, SEVIMA EdLink, etc (Jannah, Sobandi, & Suwatno, 2020; Santoso, Desprianto, Nurrohmah, Nursalamah & Putra, 2019).

Even though many free platforms have sprung up on the internet, not all of those online platforms can be used optimally in preparing the material contents and assessments based online. This depends on the policies of each educational institution to choose the right platform to use in carrying out the learning and assessment process. The Ministry of Education and Culture of the Republic of Indonesia specifically provides recommendations for several online platforms that can be used in the preparation of material contents and assessment, included: Rumah Belajar, Google G Suite for Education, Kelas Pintar, Microsoft Office 365, Quipper School, Ruang Guru, Sekolahmu, and Zenius. This is reinforced by the statement of Mailizar, Almanthari, Maulina and Bruce (2020) which stated that the Ministry of Education and Culture invites several partners who provide online learning content to support the learning process from home. Those partners, included: Kelas Pintar, Rumah Belajar, Zenius, SEVIMA EdLink, etc.

If viewed from those several recommended platforms, one of them is a platform made by an IT company in Indonesia. That platform is SEVIMA EdLink. SEVIMA EdLink is a free platform created by a consulting company and IT developer in Indonesia that focuses on education. The name of the company that developed the SEVIMA EdLink is PT. Sentra Vidya Utama (SEVIMA).

The SEVIMA EdLink platform can be used to facilitate the provision of material contents and the online assessment process (Rachbini, 2018; Roesminingsih, Nugroho, Nusantara & Eka, 2019). The SEVIMA EdLink platform can be accessed via the following URL: https://edlink.id/. Efforts that can be made to broadly introduce this platform are developing the material contents and an online assessment model based on SEVIMA EdLink. Based on that situation, this research question is how to develop the material contents and assessment form based online using the SEVIMA EdLink platform?

This research is inspired by several previous studies related to several online learning platforms. Damayanti, Solin and Eviyanti’s research (2020) showed that learning content was based on various activities to improve student’s literacy skills. That learning content was able to be used as a medium to support the learning process. However, Damayanti et al.’s research had not shown the platform used to accommodate the learning content so that it was easily accessed online by students whenever and wherever they are. Lubis, Idrus and Rashid’s research (2020) introduced a platform that supports Massive Open Online Courses, namely Zenius. It can be used to create learning classes so easy to accommodate the online material contents. However, Lubis, Idrus, & Rashid’s research only explained the general description of the Zenius platform, and it had not shown details about how to make online learning classes using that platform. Abidah, Hidaayatullaah, Simamora, Fehabutar and Mutakinati’s research (2020) showed several online platforms were able to be used as platforms for providing support facilities to create learning classes that accommodate online learning content during the social distancing. The limitations of Abidah et al.’s research were that it had not shown the operation of online platforms in detail. Ramadiani, Azainil, Hidayanto, Khairina and Jundillah’s research (2020) showed several platforms were able to be used to organize e-learning and mobile learning in accommodating material contents and assessment based online. However, Ramadiani et al.’s research had not shown the stages of developing online platforms into e-learning and mobile learning in realizing learning classes that accommodate material contents and assessment based online.

2. Method

This research used a development approach (Hamid, Lee, Taha, Rahim & Sharif, 2021; Supriyati, Iriyadi, & Falani, 2021). The development model refers to the 4D model. The 4D model consists of four stages, included: Define, Design, Develop, and Disseminate (Hibra, Hakim & Sudarwanto, 2019; Wardani, Degeng & Cholid, 2019). At the Define stage, activities were carried out to analyzing problems in material contents, analyzing the readiness of students, analyzing the tasks of lecturers and students, analyzing the structure of the subject materials. At the Design stage, activities were carried out to prepare material contents, made test questions, and entered material contents and test questions into the SEVIMA EdLink platform. At the Develop stage, tests were carried out on the material contents that had been made at the design stage. The Develop stage also carried out the content validity of multiple-choice test questions and essay test questions. At the Dissemination stage, there was the socialization of the existence of material contents to students and lecturers on a broader scale.

Generally, almost all universities in Indonesia were not able to carry out face-to-face learning in class during the Covid-19 pandemic. One of the state universities in Indonesia (especially in the Bali area), namely Universitas Pendidikan Ganesha also experienced that situation. The total students’ population of Universitas Pendidikan Ganesha in the even academic year of 2020/2021 was 13,658 students (Those complete data on the number of students can be seen at the following URL: https://pddikti.kemdikbud.go.id/data_pt/). All of those students were unable to carry out learning on campus during the Covid-19 pandemic. They were able to carry out online learning only from home.

The samples involved in this research came from subjects who really had the same vision and goal and were directly involved in the online learning process in one subject at Universitas Pendidikan Ganesha using an online learning platform. Therefore, the sample selection was conducted using a purposive sampling technique.

Referring to that purposive sampling technique, the research location was carried out at Educational Evaluation Master Study Program, Universitas Pendidikan Ganesha. The samples involved were 29 students. It was based on the fact that all students at this study program had the same vision and goal in using an online learning platform. That platform was SEVIMA EdLink.

SEVIMA EdLink can present material contents and an online assessment system that is incorporated in a learning facility specifically used for the ‘Program Evaluation’ subject at the Educational Evaluation Master Study Program, Universitas Pendidikan Ganesha. The implementation of online learning during the Covid-19 pandemic received support from all samples (29 students). Students who followed the ‘Program Evaluation’ subject for the even academic year of 2020/2021 at the Educational Evaluation Master Study Program, Universitas Pendidikan Ganesha. It can be seen from the readiness of students to prepare stable internet access at home and their readiness toward the necessary technology in using online learning platforms to support the learning process.

Based on those samples, the numbers of students who accessed and used the SEVIMA EdLink platform in online learning during the Covid-19 pandemic for the ‘Program Evaluation’ subject at the Educational Evaluation Master Study Program, Universitas Pendidikan Ganesha were 29 students. The trials toward material contents were conducted by 29 students. The tools used for trials were questionnaires. The questionnaires consist of 16 questions. Each question was assessed by all students using five types of assessment scores that follow a Likert scale. A score of 5 is mean excellent, a score of 4 is mean good, a score of 3 is mean moderate, a score of 2 is mean less, and a score of 1 is mean poor (Joshi, Kale, Chandel & Pal, 2015; Louangrath & Sutanapong, 2018; Mahayukti, Dantes, Candiasa, Marhaeni, Gita & Divayana, 2018). The content validity tests of multiple-choice questions were conducted by three experts in the field of educational evaluation and three experts in the field of informatics. The content validity tests of the essay test questions were conducted by two educational evaluation experts. The formula was used for content validity tests of the multiple-choice questions was CVR (Content Validity Ratio) formula, while the essay test questions were used the Gregory formula. The form of the Gregory formula can be seen in equation (1) (Ismanto, 2016; Mahendra, Jayantika,  Sumandya, Suarni, Ariawati, Sugiharni, et al., 2020), while the CVR formula can be seen in equation (2) (Fitriyanto, Widjanarko, & Khumaedi, 2019; Shrotryia & Dhanda, 2019).

CV = D/(A + B + C + D)

(1)

Notes:

CV= content validity

A= cells that show disagreement between the two experts

B and C= cells that show the difference in views between the two experts

D= cells that show valid agreement between the two experts

CVR = (Ne - (N/2)) / (N/2)

(2)

Notes:

CVR= content validity ratio

Ne= number of experts who stated an item of the important’s value

N= the total number of experts

The analysis technique of multiple-choice test questions and essay test questions was carried out by comparing the results of the content validity tests with the content validity score standard which refers to the Guilford category. The categorization of content validity referring to Guilford (Suyasa, Kurniawan, Ariawan, Sugandini, Adnyawati, Budhyani, et al., 2018), included: validity score of 0.80 to 1.00 is mean very high validity, 0.60 to 0.80 is mean high validity, 0.40 to 0.60 is mean sufficient validity, 0.20 to 0.40 is mean low validity, 0.00 to 0.20 is mean very low validity, a score validity ≤ 0.00 is mean invalid.

The analysis technique was used in analyzing the results of the material content trials was descriptive quantitative. This technique was done by comparing the percentage of trial results with the standard percentage of the quality level of material contents. The formula was used to calculate the percentage of trial results follows equation (3) (Dalimunte & Salmiah, 2019; Ginting, 2018; Sari & Rezeki, 2019; Sutirna, 2019), while the standard percentage of the quality level of material contents following the range of the score of percentage quality. The percentage score in the range of 0 to 54 is mean poor, the percentage score in the range of 55 to 64 is mean less, the percentage score in the range of 65 to 79 is mean moderate, the percentage score in the range of 80 to 89 is mean good, and the percentage score in the range of 90 to 100 is mean excellent (Mantasiah, Yusri & Jufri, 2020; Nawawi, Nizkon & Azhari, 2020; Sitorus, 2017; Sugiharni, 2018; Yulina, Permanasari, Hernani & Setiawan, 2019).

PTR =(f/N)*100%

(3)

Notes:

PTR= percentage of trial results

f= the total score obtained

N= the maximum total score

3. Results and Discussion

At the Define stage, several problems were found related to the material contents of the ‘Program Evaluation’ subject which were used in the online learning process at the Educational Evaluation Master Study Program, Universitas Pendidikan Ganesha. Problems related to the material contents of the ‘Program Evaluation’ subject, included: 1) The material contents did not match with the level of student understanding; 2) Unstructured presentation of material contents; 3) The form of test questions was not able to be used to measure students’ critical thinking skills.

Analysis of the students’ readiness at the Define stage was able to be seen from several things related to supporting facilities for accessing material contents, and the students’ ability to operate material content features. The results of this research indicated that all students of the Educational Evaluation Master Study Program at Universitas Pendidikan Ganesha already have laptops and relatively fast internet access to use in accessing and operating material contents of the ‘Program Evaluation’. In addition, students also have good skills in operating material content features of the ‘Program Evaluation’.

Analysis of lecturers’ assignments at the Define stage was able to be seen from the lecturers’ readiness in preparing the material contents and test questions about the ‘Program Evaluation’ subject. Analysis of students’ assignments at the Define stage was able to be seen from the students’ readiness to access material contents of ‘Program Evaluation’ subject through the SEVIMA EdLink platform and answer the tests provided in that platform.

The structural analysis of the subject at the Define stage was able to be seen from the structure of each topic in the ‘Program Evaluation’ subject. The material structure was arranged in stages starting from the easy level to the complicated level. The material structure of ‘Program Evaluation’ in stages can be shown as follows: chapter 1 about the basic concept of evaluation, chapter 2 about evaluation models, chapter 3 about types of evaluation instruments, chapter 4 about the validity of evaluation instruments, chapter 4 about the reliability of evaluation instruments, chapter 6 about modification of evaluation models, and chapter 7 about the implementation of evaluation models.

The material contents of the ‘Program Evaluation’ subject had been successfully prepared at the Design stage based on the reference of the material structure described previously at the Define stage. The material contents were made in the form of a .pdf format document and the form of a video sourced from youtube. At the Design stage, test questions were made about the material of ‘Program Evaluation’. The test questions were made in the form of multiple-choice and essays. Multiple choice test questions can be seen in Table 1, while the essay test questions can be seen in Table 2.

Items	Questions
1	The activities of data collection, data processing, and data analysis to obtain a recommendation are called… a. Test b. Assessment c. Evaluation d. Exam e. Measurement
2	The evaluation model developed by Malcolm Provus that emphasizing the existence of gaps in program implementation is called… a. Discrepancy b. CSE-UCLA c. CIPP d. Countenance e. Formative-Summative
3	An evaluation model that focuses on evaluation activities at the description and judgment stages is called... a. CSE-UCLA b. Formative-Summative c. Discrepancy d. Countenance e. CIPP
4	The evaluator does not need to attend to the program objectives. That is the characteristic of the evaluation model…. a. Formative-Summative b. Countenance c. Discrepancy d. Goal Free Evaluation Model e. CSE-UCLA
5	The formula used by two experts to validate the content of evaluation instruments is called... a. Alkin b. Gregory c. Cronbach Alpha d. Provus e. Tyler
6	The formula suitable used by 50 experts to validate the content of evaluation instruments is called... a. Gregory b. Cronbach Alpha c. Alkin d. Lawshe’s CVR e. Provus
7	The formula used to test the reliability of the multiple-choice test instruments is... a. Cronbach Alpha b. KR-20 c. Lawshe’s CVR d. Alkin e. Gregory
8	The formula used to test the reliability of the evaluation instruments in the form of questionnaires is... a. Lawshe’s CVR b. Gregory c. KR-20 d. Cronbach Alpha e. Alkin
9	The stage of the CSE-UCLA evaluation model used to socialize the existence of the program when implementing it in the field is... a. System Assessment b. Program Planning c. Program Implementation d. Program Improvement e. Program Certification
10	The thing that is not needed in the process of modifying the evaluation model is... a. The initial condition of model b. The objective of the object being evaluated c. The state of the object being evaluated d. The form of evaluation model e. The stage of evaluation model
11	The matrix in the Countenance evaluation model used to determine a decision based on the interpretation results between the difference of field observations and standard reference is... a. Description matrix b. Judgment matrix c. Explanation matrix d. Decision matrix e. Consideration matrix

Table 1. Multiple-choice Test Questions Related to the ‘Program Evaluation’ Subject

Items	Questions
1	Explain the difference between assessment and evaluation!
2	Describe the types of program evaluation models!
3	Describe the stages of the CSE-UCLA evaluation model!
4	Explain the steps for validating the content of the evaluation instruments in the form of questionnaires!
5	Explain the steps for making modifications to an evaluation model!
6	Does the evaluation activity require an assessment activity?
7	Explain the steps in conducting a reliability test on evaluation instruments in the form of interview guidelines!

Table 2. Essay Test Questions Related to the ‘Program Evaluation’ Subject

After successfully preparing the material content of the ‘Program Evaluation’ subject, multiple-choice test questions, and essay test questions, the next activity at the Design stage was to enter the material contents and test questions into the SEVIMA EdLink platform. The display of material contents entered into the SEVIMA EdLink platform can be seen in Figure 1. The display of multiple-choice test questions can be seen in Figure 2 and the essay test questions can be seen in Figure 3.

Figure 1. Display of Material Contents Entered into the SEVIMA EdLink Platform

Figure 1 above shows the process of entering material contents into the SEVIMA EdLink Platform. There are several steps to entering material contents into that platform, included: 1) clicking the “Materi (Material)” icon; 2) making the title of the material; 3) select the material to be uploaded; 4) upload of material; 5) clicking the save button.

Figure 2. Display of Multiple-choice Test Questions Entered into the SEVIMA EdLink Platform

Figure 2 above shows the process of entering multiple-choice test questions into the SEVIMA EdLink Platform. There are several steps in entering multiple-choice test questions into that platform, included: 1) click the “Quiz” icon; 2) create a quiz title; 3) create multiple-choice questions; 4) determine the answer options; 5) clicking the save button.

Figure 3. Display of Essay Test Questions Entered into the SEVIMA EdLink Platform

Figure 3 above shows the process of entering essay test questions into the SEVIMA EdLink Platform. There are several steps in entering essay test questions into that platform, included: 1) clicking the “Tugas (Assignment)” icon; 2) create assignment title; 3) making essay questions; 4) clicking the save button.

The material content trials of the ‘Program Evaluation’ subject were carried out by 29 students who followed the ‘Program Evaluation’ subject for the even academic year of 2020/2021 at the Educational Evaluation Master Study Program, Universitas Pendidikan Ganesha. The results of those trials can be seen in Table 3.

No	Respondents	Items-																∑	Quality Percentage (%)
		1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
1	Student-1	4	5	4	5	4	5	4	5	4	4	4	5	4	5	4	5	71	88.75
2	Student-2	5	4	5	5	4	4	4	4	5	4	4	5	4	4	5	4	70	87.50
3	Student-3	4	5	4	4	4	4	4	5	4	4	5	4	5	5	5	4	70	87.50
4	Student-4	4	5	4	4	4	4	5	4	4	4	4	4	4	5	5	5	69	86.25
5	Student-5	5	4	4	5	4	4	4	5	5	4	4	5	4	4	4	5	70	87.50
6	Student-6	4	5	5	4	5	4	4	4	4	4	5	4	5	4	5	4	70	87.50
7	Student-7	5	4	5	4	4	5	5	4	4	4	4	4	4	4	4	4	68	85.00
8	Student-8	4	5	4	4	4	4	4	5	4	5	4	4	4	4	5	4	68	85.00
9	Student-9	4	4	4	5	4	5	4	4	5	5	4	5	4	4	4	5	70	87.50
10	Student-10	5	5	5	5	5	4	5	5	4	4	5	4	5	5	4	4	74	92.50
11	Student-11	4	5	5	4	4	4	4	4	4	4	4	5	4	4	5	4	68	85.00
12	Student-12	5	5	4	5	4	5	4	4	4	5	4	4	4	4	4	4	69	86.25
13	Student-13	4	5	4	4	5	4	4	5	5	4	5	4	5	5	4	5	72	90.00
14	Student-14	5	4	5	5	4	5	5	4	4	5	4	5	4	4	4	4	71	88.75
15	Student-15	4	5	4	4	5	4	4	5	5	4	5	4	4	4	5	4	70	87.50
16	Student-16	5	5	4	5	4	5	5	4	4	4	4	4	5	5	4	4	71	88.75
17	Student-17	4	4	4	4	4	4	4	4	5	4	5	5	4	4	5	4	68	85.00
18	Student-18	5	5	5	5	5	4	4	5	4	5	4	4	4	4	4	5	72	90.00
19	Student-19	5	4	4	4	4	4	4	4	4	4	4	4	5	5	4	4	67	83.75
20	Student-20	4	4	5	5	5	5	5	4	4	4	5	4	4	4	4	4	70	87.50
21	Student-21	5	5	4	4	5	4	4	5	4	5	4	4	4	4	5	4	70	87.50
22	Student-22	4	5	4	5	5	5	5	4	5	4	4	5	4	4	4	5	72	90.00
23	Student-23	5	4	4	4	4	4	5	5	5	5	4	4	4	5	4	4	70	87.50
24	Student-24	5	4	4	5	4	5	4	4	4	4	5	4	5	4	4	4	69	86.25
25	Student-25	5	5	5	4	4	4	4	5	4	4	4	4	4	4	5	4	69	86.25
26	Student-26	4	4	4	4	4	5	5	4	4	5	4	5	4	4	4	5	69	86.25
27	Student-27	5	4	4	4	5	5	4	5	4	4	4	4	4	5	4	4	69	86.25
28	Student-28	5	5	5	4	5	4	4	4	4	5	4	4	4	4	4	4	69	86.25
29	Student-29	4	4	5	5	4	4	5	4	5	4	4	5	4	5	4	5	71	88.75
Average																			87.33

Table 3. Trial Results toward the Material Contents of the ‘Program Evaluation’ Subject Entered into the SEVIMA EdLink Platform

Notes::

Item-1: The material contents are following the syllabus of the ‘Program Evaluation’ subject

Item-2: The material contents are structured ranging from easy to most difficult.

Item-3: The material contents are packaged in a .pdf format document that can easily be entered into the SEVIMA EdLink platform

Item-4: The material contents are packaged in the form of video files that can be accessed through the SEVIMA EdLink platform.

Item-5: The duration of the video file is not too long so that students are not bored

Item-6: Video files contain material that is interesting and easy to understand

Item-7: Each chapter in the material contains examples of cases to improve student understanding

Item-8: The case examples in each chapter are following the material being taught and the facts that have occurred in the field

Item-9: Each chapter in the material contains practice questions

Item-10: The practice questions in each chapter are following the material being taught

Item-11: Each chapter on the material contents contains an assignment

Item-12: The tasks that appear in each chapter are following the material being taught

Item-13: Each chapter in the material content contains obvious reference sources from journal articles, proceeding articles, and other relevant sourcebooks

Item-14: Lecturers can easily update the material contents

Item-15: The material contents are easily accessible anytime and anywhere

Item-16: The security of material contents stored in the platform is strictly maintained

The content validity tests of the multiple-choice test questions were conducted by the six experts (three educational evaluation experts and three informatics experts). The results of the content validity tests toward the multiple-choice test questions completely can be explained as follows.

Experts	Items
	1	2	3	4	5	6	7	8	9	10	11
Educational Evaluation Expert-1	I	I	I	I	I	I	I	I	I	NI	I
Educational Evaluation Expert-2	I	I	I	I	I	I	I	I	I	I	I
Educational Evaluation Expert-3	I	I	I	I	I	I	I	I	I	NI	I
Informatics Expert-1	I	I	I	I	I	I	I	I	I	NI	I
Informatics Expert-2	I	I	I	I	I	I	I	I	I	NI	I
Informatics Expert-3	I	I	I	I	I	I	I	I	I	NI	I
Notes: I: Important, NI: Not Important

Table 4. Results of Content Validity Tests toward Multiple-choice Test Questions by Six Experts

Total of Experts’ Assessment	Items
	1	2	3	4	5	6	7	8	9	10	11
Not Important	0	0	0	0	0	0	0	0	0	5	0
Important	6	6	6	6	6	6	6	6	6	1	6

Table 5. Data Tabulation of Content Validity Test Results toward Multiple-choice Test Questions by six Experts

Referring to Table 5, the calculation process of the content validity of multiple-choice test questions can be done using Lawshe’s CVR formula. The results of the content validity calculations of the multiple‑choice test questions can be seen in Table 6.

Items	Ne	N	N/2	Ne - (N/2)	CVR
1	6	6	3	3	1.00
2	6	6	3	3	1.00
3	6	6	3	3	1.00
4	6	6	3	3	1.00
5	6	6	3	3	1.00
6	6	6	3	3	1.00
7	6	6	3	3	1.00
8	6	6	3	3	1.00
9	6	6	3	3	1.00
10	1	6	3	-2	-0.67
11	6	6	3	3	1.00
Average					0.85

Table 6. Results of the Content Validity Calculations of the Multiple-choice Test Questions

The content validity tests of the essay test questions were conducted by two educational evaluation experts. The results of the content validity test for essay test questions completely can be explained as follows.

Items	Experts
	Educational Evaluation Expert-1				Educational Evaluation Expert-2
	Irrelevant		Relevant		Irrelevant		Relevant
	1	2	3	4	1	2	3	4
1	-	-	-	Ö	-	-	Ö	-
2	-	-	Ö	-	-	-	Ö	-
3	-	-	Ö	-	-	-	Ö	-
4	-	-	Ö	-	-	-	-	Ö
5	-	-	-	Ö	-	-	Ö	-
6	-	Ö	-	-	Ö	-	-	-
7	-	-	-	Ö	-	-	-	Ö

Table 7. Results of the Content Validity Tests toward Essay Test Questions by Two Educational Evaluation Experts

The cross-tabulation process can be done, if we have compiled the data from the content validity test results of the essay test questions. Based on the data shown in Table 7, it is possible to compilation the data from the content validity of the essay test questions. The compilation results intended can be seen in Table 8. The results of the cross-tabulation process can be seen completely in Table 9.

Educational Evaluation Expert-1		Educational Evaluation Expert-2
Irrelevant (Score: 1 - 2)	Relevant (Score: 3 - 4)	Irrelevant (Score: 1 - 2)	Relevant (Score: 3 - 4)
6	1, 2, 3, 4, 5, 7	6	1, 2, 3, 4, 5, 7

Table 8. Compilation of Data from Results of Content Validity Tests that had conducted by Two Educational Evaluation Experts

		Educational Evaluation Expert-1
		Irrelevant (Score: 1 - 2)	Relevant (Score: 3 - 4)
Educational Evaluation Expert-2	Irrelevant (Score: 1 - 2)	A 6 (1)	B - (0)
	Relevant (Score: 3 - 4)	C - (0)	D 1, 2, 3, 4, 5, 7 (6)

Table 9. Cross Tabulation Process from Results Data of the Content Validity Tests of the Essay Test Questions

Based on the cross-tabulation results shown in Table 9, then the content validity of the essay test questions can be calculated using the Gregory Formula. The calculation process can be seen as follows.

CV= D / (A + B + C + D)

= 6 / (1 + 0 + 0 + 6)

= 6 / 7

= 0.86

The socialization of material contents existence for the ‘Program Evaluation’ subject at Disseminate stage conducted by lecturers to students with providing an access code to become a member in the online class through the SEVIMA EdLink platform. Socialization of the existence of assessment system the learning outcome at the Dissemination stage is carried out by distributing assignment questions, quizzes, middle tests, and final tests to all students who have joined as members in an online class through SEVIMA EdLink platform.

If seen from the quality percentage of the material contents of the ‘Program Evaluation’ subject that had been shown earlier in Table 3, in general, the quality of the material contents was classified as good. This was because the quality percentage of the material contents was 87.33%. It was in the range of 80% to 89% in the view of the standard percentage of the quality level of material contents.

If seen from content validity results of the multiple-choice test questions, it was able to be stated that the multiple-choice test questions were categorized as very high validity. This was because the validity score was 0.85. It was in the range of 0.80 to 1.00 in viewed from Guilford’s categorization. Based on the results shown in Table 6, the multiple-choice test question that needs to be discarded was question-10 because its CVR value was negative (–), while the other multiple-choice test questions still were used. If seen from content validity results of the essay test questions, it was able to be stated that the essay test questions were categorized as very high validity. This was because the validity score was 0.86. It was in the range of 0.80 to 1.00 in viewed from Guilford’s categorization. Based on the results shown in Table 9, the essay test question that needs to be discarded was question-6 because assessment results from experts showed an irrelevant category, while the other essay test questions still were used.

This research has succeeded in providing alternative answers to the limitations of several previous studies. The limitation of Damayanti et al.’s research (2020) has been answered by the presence of this research which features the SEVIMA EdLink platform as a facility to accommodate material contents that can be accessed online by students anytime and anywhere. Limitations of Lubis et al.’s research (2020), Abidah et al.’s research (2020), and Ramadiani et al.’s research (2020) have also been answered through this research by the visualization of the SEVIMA EdLink platform used to show material contents and an online assessment system.

In principle, this research is similar to Ahmadi and Ilmiani’s research (2020) which shows free platforms that can be used as support facilities for online learning during the Covid-19 pandemic. In principle, Wahyudi’s research (2020) strengthens the position of this research by showing the use of the SEVIMA EdLink platform which can be used as a facility that accommodates learning content and an online assessment system.

4. Conclusion

The material contents for the ‘Program Evaluation’ subject are entered into the SEVIMA EdLink platform in the form of .pdf documents and videos. The assessment form of learning outcomes for the ‘Program Evaluation’ subject entered into the SEVIMA EdLink platform is in the form of multiple‑choice test questions and essay test questions. The novelty in this research is there is a free platform created by IT development companies in the education sector in Indonesia to accommodate learning content and online assessment models. The impact of this research is the ease of carrying out the online learning process for all study programs in tertiary institutions. Therefore, this platform is suitable and free to be used by each university in supporting the online learning process during the Covid-19 pandemic and post-pandemic. Future work that needs to be done to overcome research limitations, included: 1) Next research is conducting reliability tests on multiple-choice test questions and essay test questions used in assessing learning outcomes of the ‘Program Evaluation’ subject; 2) Next research is making learning videos independently about the ‘Program Evaluation’ subject.

Acknowledgement

The authors express their gratitude to the Chancellor and Chair of the Research and Community Service Institute, Universitas Pendidikan Ganesha, who had given permission and the opportunity for carrying out this research.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This research was supported by the Research and Community Service Institute, Universitas Pendidikan Ganesha through Research Grant based on contract number: 291/UN48.16/LT/2021.

References

Abidah, A., Hidaayatullaah, H.N., Simamora, R.M., Fehabutar, D., & Mutakinati, L. (2020). The impact of Covid-19 to Indonesian education and its relation to the philosophy of “merdeka belajar”. Studies in Philosophy of Science and Education (SiPoSE), 1(1), 38-49. https://doi.org/10.46627/sipose.v1i1.9

Ahmadi, A. & Ilmiani, A.M. (2020). The use of teaching media in arabic language teaching during covid‑19 pandemic. Dinamika Ilmu, 20(2), 307-322. https://doi.org/10.21093/di.v20i2.2515

Dalimunte, M., & Salmiah, M. (2019). Students’ ability at changing direct into indirect speech and indirect into direct speech. Budapest International Research and Critics Institute-Journal, 2(2), 178-185. https://doi.org/10.33258/birci.v2i2.249

Damayanti, I., Solin, M., & Eviyanti, E. (2020). The development of activity literacy teaching material based on multimodal of literacy for the grade 4th at SDS Amir Hamzah. Budapest International Research and Critics in Linguistics and Education (BirLE) Journal, 3(3), 1435-1444. https://doi.org/10.33258/birle.v3i3.1209

Fitriyanto, J.N., Widjanarko, D., & Khumaedi, M. (2019). Validity and reliability test of assessment instrument of the suitability of electric power steering media. Journal of Vocational Career Education, 4(1), 61-69. https://doi.org/10.15294/jvce.v4i1.22186

Ginting, F.Y. (2018). An analysis of students’ ability in using punctuation marks in descriptive paragraph writing. Budapest International Research and Critics Institute-Journal, 1(3), 338-344. https://doi.org/10.33258/birci.v1i3.57

Hamid, S.N.M., Lee, T.T., Taha, H., Rahim, N.A, & Sharif, A.M. (2021). E-content module for chemistry massive open online course (MOOC): development and students’ perceptions. Journal of Technology and Science Education, 11(1), 67-92. https://doi.org/10.3926/jotse.1074

Hibra, B.A., Hakim, L., & Sudarwanto, T. (2019). Development of vlog learning media (video tutorial) on student materials tax at SMK PGRI 1Jombang. International Journal of Educational Research Review, 4(3), 435-438. https://doi.org/10.24331/ijere.573945

Ismanto, I. (2016). Aspect of language on a qualitative analysis of student’s evaluation instrument. Qudus International Journal of Islamic Studies, 4(2), 218-232. https://doi.org/10.21043/qijis.v4i2.1788

Jannah, S.N., Sobandi, A., & Suwatno, S. (2020). The measurement of usability using use questionnaire on the google classroom application as e-learning media (a case study: SMK Negeri 1 Bandung). Teknodika, 18(2), 94-105. https://doi.org/10.20961/teknodika.v18i2.42486

Joshi, A., Kale, S., Chandel, S., & Pal, D.K. (2015). Likert scale: explored and explained. British Journal of Applied Science & Technology, 7(4), 396-403. https://doi.org/10.9734/BJAST/2015/14975

Louangrath, P.I., & Sutanapong, C. (2018). Validity and reliability of survey scales. International Journal of Research & Methodology in Social Science, 4(3), 99-114. https://doi.org/10.5281/zenodo.2545038

Lubis, A.H., Idrus, S.Z.S., & Rashid, S.A. (2020). The exposure of MOOC usage in Indonesia. International Journal of Scientific & Technology Research, 9(2), 2716-2720.

Mahayukti, G.A., Dantes, N., Candiasa, I.M., Marhaeni, A.A.I.N., Gita, I.N., & Divayana, D.G.H. (2018). Computer-based portfolio assessment to enhance students’ self-regulated learning. Journal of Theoretical and Applied Information Technology, 96(8), 2351-2360.

Mahendra, I.W.E., Jayantika, I.G.N.A.T., Sumandya, I.W., Suarni, N.M, Ariawati, N.W., Sugiharni, G.A.D. et al. (2020). Design of digital test using wondershare in supporting the blended learning with kelase platform. Universal Journal of Educational Research, 8(3), 953-959.
https://doi.org/10.13189/ujer.2020.080330

Mailizar, M., Almanthari, A., Maulina, S., & Bruce, S. (2020). Secondary school mathematics teachers’ views on e-learning implementation barriers during the Covid-19 pandemic: the case of Indonesia. Eurasia Journal of Mathematics, Science and Technology Education, 16(7), 1-9. https://doi.org/10.29333/ejmste/8240

Mantasiah, R., Yusri, & Jufri (2020). Semantic feature analysis model: linguistics approach in foreign language learning material development. International Journal of Instruction, 13(1), 185-196. https://doi.org/10.29333/iji.2020.13112a

Nawawi, S., Nizkon, & Azhari, A.T. (2020). Analysis of the level of critical thinking skills of students in biological materials at Muhammadiyah high school in Palembang city. Universal Journal of Educational Research, 8(3D), 47-53. https://doi.org/10.13189/ujer.2020.081707

Rachbini, W. (2018). The impact of consumer trust, perceived risk, perceived benefit on purchase intention and purchase decision. International Journal of Advanced Research, 16(1), 1036-1044. https://doi.org/10.21474/IJAR01/6317

Ramadiani, R., Azainil, A Hidayanto, A.N., Khairina, D.M., & Jundillah, M.L. (2020). Teacher and student readiness using e-learning and m-learning. Bulletin of Electrical Engineering and Informatics, 9(3), 1176-1182. https://doi.org/10.11591/eei.v9i3.2006

Roesminingsih, M.V., Nugroho, R., Nusantara, W., & Eka P,I.G.L.P. (2019). The change of educational tutoring paradigm as a non-formal education unit. International Journal of Innovation, Creativity and Change, 5(5), 611-622.

Santoso, H.B., Desprianto, D.A., Nurrohmah, I., Nursalamah, R.K., & Putra, P.O.H. (2019). Customer journey construction of the Indonesian open-education resources platform. International Journal of Emerging Technologies in Learning, 14(24), 18-30. https://doi.org/10.3991/ijet.v14i24.12045

Sari, S.A., & Rezeki, Y.S. (2019). The development of an ingenious circuit based on chemo-edutainment learning. International Journal of Educational Research Review, 4(1), 15-25. https://doi.org/10.24331/ijere.467078  

Shrotryia, V.K., & Dhanda, U. (2019). Content validity of assessment instrument for employee engagement. Sage Open, January-March 2019, 1-7. https://doi.org/10.1177/2158244018821751

Sitorus, J. (2017). Inhibiting factor of primary school teacher competence in North Sumatra province, Indonesia: pedagogical and professional. Educational Research Applications, 1, 1-10. https://doi.org/10.29011/2575-7032/100004

Sugiharni, G.A.D. (2018). The development of interactive instructional media oriented to creative problem solving model on function graphic subject. Journal of Educational Research and Evaluation, 2(4), 183-189. https://doi.org/10.23887/jere.v2i4.16694

Supriyati, Y., Iriyadi, D., & Falani, I. (2021). The development of equating application for computer based test in physics hots category. Journal of Technology and Science Education, 11(1), 117-128. https://doi.org/10.3926/jotse.1135

Sutirna, S. (2019). Subject teachers’ perceptions of academic mentoring and counseling services. COUNS‑EDU: The International Journal of Counseling and Education, 4(4), 129-133. https://doi.org/10.23916/0020190423040

Suyasa, P.W.A, Kurniawan, P.S., Ariawan, I.P.W., Sugandini, W., Adnyawati, N.D.M.S., Budhyani, I.D.A.M. et al. (2018). Empowerment of CSE-UCLA model based on glickman quadrant aided by visual application to evaluate the blended learning program on SMA Negeri 1 Ubud. Journal of Theoretical and Applied Information Technology, 96(18), 6203-6219.

Wahyudi, A. (2020). SEVIMA EdLink social learning network for nursing science students at STIK Bina Husada Palembang. Language and Education Journal, 5(1), 28-37.

Wardani, D.L., Degeng, I.N.S., & Cholid, A. (2019). Developing interactive multimedia model 4D for teaching natural science subject. International Journal of Education and Research, 7(1), 63-72.

Yulina, I.K., Permanasari, A., Hernani, H., & Setiawan, W. (2019). Analytical thinking skill profile and perception of pre service chemistry teachers in analytical chemistry learning. Journal of Physics: Conference Series, 1157(4), 1-7. https://doi.org/10.1088/1742-6596/1157/4/042046

---