Journal of Technology and Science Education

• Home
• About
• Login
• Archives
• Submissions
• Publication Fee
• Indexing & Statistics

Home > Vol 12, No 1 (2022) > Lilik Hariyanto

A FRAMEWORK FOR MEASURING THE LEVEL
OF ACHIEVEMENT OF VOCATIONAL STUDENTS COMPETENCY
OF ARCHITECTURAL EDUCATION

1Universitas Negeri Yogyakarta (Indonesia)
2Institut Agama Islam Negeri Ponorogo (Indonesia)
3Universitas Sultan Ageng Tirtayasa (Indonesia)

Received December 2020

Accepted January 2022

Abstract

The determination of work competence according to industrial needs that are expected by employers of new graduates in the department of architectural engineering in vocational high schools will help graduates to transit smoothly from the academic education environment to the real conditions of the construction industry in the workplace. This research identifies which group of competencies attributes that graduates should have in explaining appropriate competencies according to construction industry standards in Indonesia. The survey was conducted in 47 building construction companies consisting of 56 respondents using a questionnaire method. Multivariate analyses using the CB-SEM method used to validate the evaluation model. The research analysis used quantitative descriptive and factor analysis with the Explanatory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) methods. The EFA test shows ten factors so that it consists of 30 items of competency measurement. Overall, the CFA test has met the criteria for the goodness of fit statistics. The results of the study concluded that the instrument developed met good validity and reliability and two model frameworks for measuring the level of achievement of student competence in architectural techniques were feasible to use.

 

Keywords – Architectural engineering, Competency achievement, Vocational students, Student’s competency, Explanatory Factor Analysis (EFA), Confirmatory Factor Analysis (CFA).

To cite this article:

Hariyanto, V.L., Daryono, R.W., Hidayat, N., Prayitno, S.H., & Nurtanto, M. (2022). A framework for measuring the level of achievement of vocational students competency of architectural education. Journal of Technology and Science Education, 12(1), 157-171. https://doi.org/10.3926/jotse.1188

 

----------

1. 1. 1. Introduction

The provision of human resources to improve competitiveness according to the Global Talent Competitiveness Index 2020 states that Indonesia ranked 65 of 132 countries (Forum Económico Mundial, 2019). One of the influencing factors is vocational and technical skills that include mid-level skills workforce with secondary education (Olazaran, Albizu, Otero & Lavía, 2019), employability, ease of finding skilled employees (Mingaleva & Vukovic, 2020) and relevance of the education system to the industry (Forum Económico Mundial, 2019). This is influenced by various factors that are synergistically interrelated, including policies, curriculum, education, education personnel, facilities and infrastructure, financing (Supriyadi, Indro, Priyanto & Surwi, 2020), management (Vlasova, Krasnova, Abraukhova & Safontseva, 2018), and transformation of knowledge culture (Karstina, Zechiel, & Machado, 2021).

Education is the key to improving the quality of human resources (HR) so that they can be independent and do not always depend on industry or employment from other parties (Pratomo, Priyambodo & Wiyarsi, 2020; Tsuey‑Fen, 2020). Vocational High School (VHS) is an educational institution that formulates students to be ready to work according to competencies in the industry (Cruz, Saunders-Smits & Groen, 2020; Triyono, Mohib, Kassymova, Pratama, Adinda & Arpentieva, 2020; Tucker & Hughes, 2020), attitudes (Liu, Chen, Yang, Liu, Ma, Craig, G.R. et al., 2020), skills (Wheelahan, 2015; Yudiono, 2018) in their fields of expertise that are by industry needs, meet employer requirements, and are expected to be entrepreneurs. Apart from working in the industry, VHS graduates’ also expected to be able to continue higher education or become entrepreneurs to create job opportunities independently (Daryono, Rochmadi & Hidayat, 2021; Pratomo et al., 2020).

As a formal education that develops educated, skilled, and competent VHS graduates’ (Daryono et al., 2021; Nurtanto, Arifin, Sofyan, Warju & Nurhaji, 2020), VHS is deemed necessary to improve their quality. The current competency of VHS graduates is still not able to bring them to excel (Ayadat, Ahmed, Chowdhury & Asiz, 2020) and become winners in the increasingly competitive world of work competition (Daryono, Yolando, Jaedun & Hidayat, 2020). Motivation and the way of thinking of students who tend to look for work after graduation and do not think of looking for other alternatives such as opening their jobs or doing entrepreneurship still dominate the minds of most VHS graduates. This condition has an impact on the increasing number of unemployed VHS graduates’ (Karstina et al., 2021; Setyadi, Triyono & Daryono, 2021; Vlasova et al., 2018).

The world of work is constantly changing creating new challenges for employers and employees (Berestova, Lazareva & Leontyev, 2020; Khan, Soundararajan & Shoham, 2020). The progress of industry resulted in the advancement of the employment sector and the number of workers in the construction sector (Špaček, Legény & Gregor, 2020). Indonesian employment data explains that the construction sector is in the top four with 18.98% in August 2019. Employment of the population of each business sector shows the ability in the construction sector in the labor absorption rate (Subdirektorat Statistik Konstruksi, 2019).

The data observations conducted at VHS in the architecture department obtained data regarding the recapitulation of the absorption of the number of graduates in the architecture department within three years, from 2016 to 2019 the school year, namely 24.00%, 24.62%, and 41.79%. Absorption data for graduates who have worked for the past three years is 23.53%, 41.54%, and 40.30%. Data on the absorption of graduates for entrepreneurs during the last three years is 0.00%, 3.08%, and 2.99%. While graduate data not detected is 63%, 30%, and 14.93% (Badan Pusat Statistik, 2020). Based on the data absorption of graduates in the category of graduates who work it can be concluded that the graduates have not been able to work optimally.

This research is based on the number of architectural engineering graduates who work not according to their field of expertise. The low readiness of graduates to work is due to the inadequate development and assessment of students’ competence. The material in vocational high schools is not by the needs of industry and developments in construction technology. Therefore, the research focused on the evaluation of competencies implemented in VHS in architectural engineering education, which aligned with the urgent needs currently needed by the construction industry. So that the evaluation model is used to measure the extent, to which student’s mastery of current competencies as architectural engineering graduates to be able to work according to industry demand.

2. Methodology

2.1. Sample Construction

This research uses descriptive research with a quantitative approach using survey methods. The population of this research is construction industry organizations in the field of building construction services as employers for architectural engineering graduates in vocational secondary education level. The sample of respondents was 56 practices from 47 organizations in building construction companies. Data collection was carried out through a questionnaire given to each construction company. The sample profiles in this study are based on the profiles of employers who responded to the questionnaire, the sector-specific civil engineering work performed by the responding organizations, and the categories of organizations. A list of construction company profiles showed in Table 1.

Construction Industry	Total	Percentage (%)
Profile of employer (N=56)
General manager	12	21.43
Project manager	14	25.00
Senior drafter/detailer	17	30.36
Senior engineer	13	23.21
Specific Sectors (N=47)
Buildings	20	42.55
Roads	12	25.53
Bridges	8	17.02
Factory/Workshop	7	14.89
Form of Construction Services Business (N=47)
Limited liability company	19	40.43
Commanditaire vennootschap	21	44.68
Individual	7	14.89
Type of Construction Service Business (N=47)
Planning services	20	42.55
Implementing services	15	31.91
Supervisory services	12	25.53

Table 1. Profiles of construction companies and research respondents

2.2. Research Instruments

The majority of employers are high-ranking officials in their organizations. Most of the respondents came from the building, roads, and bridges sector, but the responses from sectors related to workshop/factory work were also relatively significant. The majority of organizations that participated in the questionnaire survey were non-governmental organizations. A questionnaire designed with 40 attributes considered important at this time. The questionnaire has designed for employers for graduates in architectural engineering majors based on Indonesian National Work Competency Standards and the architectural engineering education curriculum in Indonesia. The questionnaire is structured using a four-point Likert scale where 1 indicates “not important”; 2 “quite important”, 3 “important”, 4 “very important”. Table 2 shows a list of 40 competency items in ten competency aspects.

Competency aspects	Construct	Item of competencies
General Competencies	KU1	Independence and responsibility at work
	KU2	Time discipline and hard work in doing work
	KU3	Complete the work according to the specified criteria
	KU4	Work together to solve problems
Technical Drawing Competencies	GT1	Drawing sketches
	GT2	Presenting the types of engineering drawing equipment
	GT3	Drawing symbol rules on engineering drawings
	GT4	Drawing notation rules on engineering drawings
Statically Structures Competencies	MT1	Presenting the factors that influence the building structure
	MT2	Analyzing the forces on the building structure
	MT3	Calculating the balance of forces in a simple block
	MT4	Calculating the stresses that occur in the beam
Basic Building Construction Competencies	DKB1	Carrying out concrete construction work
	DKB2	Carrying out steel construction work
	DKB3	Carrying out wood construction work
	DKB4	Carrying out land and stone construction work
Land Measurement Engineering Competencies	TPT1	Performing measurement techniques in building construction
	TPT2	Performing staking out of buildings
	TPT3	Analyzing and report measurement data
	TPT4	Evaluating measurement results for construction work
Software Application and Building Interior Design Competencies	APLPIG1	Creating interior pictures
	APLPIG2	Creating 3D images using the material editor function
	APLPIG3	Creating a room acoustic design drawing
	APLPIG4	Creating interior designs and accessories in every room
Road and Bridge Construction Competencies	KJJ1	Presenting the results of road and bridge construction drawings
	KJJ2	Presenting the requirements for road and bridge construction
	KJJ3	Drawing of road and bridge construction
	KJJ4	Drawing road construction details
Construction Cost Estimation Competencies	EBK1	Presenting the technical specifications of the job
	EBK2	Presenting materials specifications for construction work
	EBK3	Calculating the estimated cost of construction work
	EBK4	Creating reports on construction work
Building Construction and Utility Competencies	KUG1	Creating building construction drawings
	KUG2	Creating a utility building image
	KUG3	Making a building mock-up
	KUG4	Creating construction reports and building utilities
Creativity and Entrepreneurship Products Competencies	PKK1	Making mass production planning
	PKK2	Making indicators of the success of the production stages
	PKK3	Doing marketing
	PKK4	Making financial reporting

Table 2. Architectural engineering competencies in the Indonesian context

2.3. Research Methods

The validation process is carried out by testing the validity of the construct. Multivariate data analysis used Structural Equation Model (SEM) with Covariance‐based Structural Equation Modeling (CB-SEM) method used to assess evaluation models (Baber, 2021; Kyriakides & Charalambous, 2020; Prasojo, Habibi, Mukminin, Sofyan, Indrayana & Anwar, 2020; Saleem, Kamarudin, Shoaib & Nasar, 2021; ShayesteFar, 2020). Testing of construct validity was carried out using factor analysis using Explanatory Factor Analysis (EFA) (Broadbent, Sharman, Panadero & Fuller-Tyszkiewicz, 2021; Gok, 2021; Hidayat, Zamri & Zulnaidi, 2018; Saifurrahman, Sudira & Daryono, 2021; ShayesteFar, 2020) and testing the suitability of the model using Confirmatory Factor Analysis (CFA) (Hair, Black, Babin & Anderson, 2010; Hidayat et al., 2018; Liu et al., 2020; Rodríguez-Santero, Torres-Gordillo & Gil-Flores, 2020; Suwanroj, Leekitchwatana & Pimdee, 2019). EFA aims to test whether the statement items or indicators used can confirm a factor or variable (Mateus & Hernández-Breña, 2019; Rodríguez-Santero et al., 2020; Suwanroj et al., 2019). CFA is used to test the suitability of the theoretical model with empirical data (Gok, 2021; Isac, Palmerio & van der Werf, 2019), both the measurement model and the evaluation model are based on four indicators (Alpaslan, 2019; Boonk, Gijselaers, Ritzen & Brand-Gruwel, 2020; Rodgers, Reed, Houchins & Aloe, 2020). Reliability tests were obtained based on the Cronbach Alpha value, component factor analysis using the EFA methods intended to ensure the validity and confirmation of construction.

2.4. Data Analysis

Descriptive statistical analysis, multicollinearity, normality, and data reduction before applied in factor analysis, using SPSS 25.0 software and Amos 22.0. Multicollinearity analysis can conclude the inter-change matrix with a value of ≤0.90 (Kline, 2005). The reliability test used Cronbach’s Alpha coefficients, composite reliability, and average variance extracted values. The CR should be more than 0.60 and AVE should be over 0.50 (Hidayat et al., 2018; Liou, 2021). After that, all items are included in the criteria for the factor analysis test.

EFA was carried out to determine the factors of the instrument measuring the achievement of student competencies. The results of the analysis based on KMO (>0.5) values, Bartlett test values (p <0.05), MSA (>0.5), communalities values (>0.5), eigenvalues (>1.0), factor loading (>0.4) (Harlan & Van Haneghan, 2020; Kang, Hense, Scheersoi & Keinonen, 2019; Pala & Erdem, 2020). The analysis of the measurement model using CFA aims to test the suitability of the theorized model with empirical data. The main criteria for the fit of the model with field data are if at least three conditions are met, namely Chi‑square (p-value >0.05), RMSEA (<0.08), CFI (≥0.90), and TLI (>0.90) (Boonk et al., 2020; Coetzer, Susomrith & Ampofo, 2020; Creed, Hood & Hu, 2020; Liou, 2021; Syed, 2018; Ye, Strietholt & Blömeke, 2021).

3. Results

3.1. Preliminary Analysis

After the first stage in preliminary analysis related to multicollinearity and normality tests, as well as the EFA test in the first stage, 30 out of 40 competency items passed the recommended value requirements for further analysis. The other 10 MSA competency score points and loading factors are below the recommended value, and then the next analysis uses 30 competency items that have met the conformity value standard. The 10 competency points that failed were KU3, GT2, MT4, DKB2, TPT2, APLPIG3, KJJ1, EBK4, KUG2, and PKK1.

Descriptive statistical results of all competency items, the level of urgency of competence in architectural engineering education that is needed by the construction industry, namely the competence of calculating the estimated cost of construction work (EBK3) with an average of 3.679, then competence from evaluating and improving measurement results in the form of working drawings for construction work (APLPIG4), and creating interior designs with elements, materials, models, and accessories in every room (TPT4) with an average of 3.661. Overall, 30 items of competence in the category of urgently with a mean acquisition of 3.508 out of 4.00. Simultaneously, all competency items reached univariate normality (skewness and kurtosis values between -1.783 to 1.95) (Zare & Nastiezaie, 2019). In the case of multicollinearity, the correlation between the ten competency items analyzed in the construction value ranges from 0.248 to 0.737.

From the results of the analysis of the results of the inter-item correlation matrix, all Pearson Correlation values in each competency aspect range from 0.405 to 0.703. This shows that every aspect of competence has an adequate level of validity because the value is more value of the r-table is 0.264 and the inter‑correlation matrix value is ≤0.90 (Chai, Jong, Yin & Chen, 2019). Likewise for the Sig. (2-tailed), all aspects of the reviewed competency score are met with a result of 0.000 (<0.005). So that the forty competencies required for the construction industry are sufficient.

3.2. Reliability Instruments

Reliability is the stability and suitability of each score found. According to Ekolu and Quainoo (2019), if the CA value ranges from 0.90<x≤1.0 then the item is categorized very high, 0.70 <x≤ 0.90 is categorized high, 0.30<x≤ 0.70 is categorized as moderate, and 0.00 ≤x≤ 0.30 is categorized low (Ekolu & Quainoo, 2019). The CR should be more than 0.60 and AVE should be over 0.50 (Hidayat et al., 2018; Suwanroj et al., 2019). The results of the reliability of the instrument showed in Table 3.

No	Competency aspects	CA	Overall CA	CR ≥ 0.6	AVE ≥0.5
1	General Competencies	0.614	0.914	0.882	0.737
2	Technical Drawing Competencies	0.694		0.936	0.833
3	Statically Structures Competencies	0.860		0.900	0.774
4	Basic Building Construction Competencies	0.917		0.899	0.748
5	Land Measurement Engineering Competencies	0.847		0.858	0.676
6	Software Application and Building Interior Design Competencies	0.613		0.931	0.818
7	Road and Bridge Construction Competencies	0.776		0.971	0.921
8	Construction Cost Estimation Competencies	0.797		0.967	0.907
9	Building Construction and Utility Competencies	0.697		0.932	0.821
10	Creativity and Entrepreneurship Products competencies	0.741		0.936	0.830

Table 3. Reliability analysis of competency items

3.3. Exploratory Factor Analysis (EFA)

The EFA test considers each item of competence that has passed the multicollinearity test and normality and reliability for each item. All 40-competency items stipulated in the past questionnaire were included in the 10 competency aspects. The EFA test criteria are based on the value of the KMO Index, Bartlett’s Test, Measure of Sampling Adequacy (MSA), communalities, factor loading, eigenvalues, and plot scree (Chai et al., 2019; Mateus & Hernández-Breña, 2019). The results of the calculation of the KMO obtained a value of 0.801 that has exceeded the set standard value, namely 0.50 so that the scope of each work competency according to employer standards in the building construction industry is satisfactory. Bartlett’s Test of Sphericity Approx. Chi-Square scored 999.849; df = 435; Sig. = 0.000. The next output is the total variance explained table. Total variance explained is the percentage of the measured construct variance explained by several factors that are formed. From the initial eigenvalues column in the cumulative sub-column, it can seem that reducing 30 items reducing it to 10 factors can explain 72.039%.

The next step is to identify the MSA values, communalities, and factor loading. MSA is useful for knowing and determining which variables are suitable for use in factor analysis. Requirements that must be met in the factor analysis are above 0.50. This Communalities table shows the value of the variable under study whether it can explain the factor or not. The variable is considered capable of explaining the factor if the Extraction value is >0.50. The next output is the matrix value on the rotating component. This value shows the loading factor for each factor. The principle of exploratory factor analysis is that each item correlated with all factors, but a good item only has a high factor loading on the factor it measures. Table 4 shows the analysis results of MSA values, communalities, factor loading, and total variance explained.

The principle of exploratory factor analysis is that each item can be correlated with all factors, from the results of the analysis, the factor loading value is between 0.458 to 0.828, and this value has met the standard because it is more than 0.40 (Harlan & Van Haneghan, 2020; Hidayat et al., 2018). The results of the analysis state that each competency item correlated with other factors.

Construct	MSA	Communalities	FL	Construct	MSA	Communalities	FL
KU1	0.707	0.756	0.561	APLPIG1	0.884	0.756	0.642
KU2	0.719	0.725	0.490	APLPIG2	0.863	0.554	0.458
KU4	0.744	0.731	0.667	APLPIG3	0.528	0.739	0.802
GT1	0.565	0.781	0.828	KJJ2	0.807	0.662	0.528
GT3	0.724	0.699	0.660	KJJ3	0.794	0.721	0.784
GT4	0.723	0.839	0.585	KJJ4	0.803	0.761	0.740
MT1	0.856	0.698	0.489	EBK1	0.854	0.640	0.491
MT2	0.869	0.782	0.657	EBK2	0.882	0.761	0.688
MT3	0.811	0.737	0.731	EBK3	0.792	0.562	0.535
DKB1	0.827	0.763	0.728	KUG1	0.751	0.638	0.707
DKB3	0.847	0.813	0.793	KUG3	0.719	0.781	0.608
DKB4	0.885	0.748	0.720	KUG4	0.666	0.649	0.743
TPT1	0.852	0.734	0.526	PKK2	0.890	0.746	0.656
TPT3	0.758	0.646	0.476	PKK3	0.781	0.763	0.794
TPT4	0.834	0.754	0.620	PKK4	0.794	0.676	0.583

Table 4. The output of the EFA analysis

3.4. Measurement Model for Student Competency Achievement Level

3.4.1. Confirmatory Factor Analysis: First-Order Factor Testing

This study examines two measurement models to measure the level of competency attainment of students in architectural engineering education majors in the Indonesian context. The CFA test is intended to verify the factorial validity of the measurement model. CFA can prove the suitability of the suggested model with the identified factor structure through the EFA test. Therefore, the CFA model presented in Figure 1 is a first-order measurement model that shows the measurement structure for the achievement of student competencies in architectural engineering majors in Indonesia.

Figure 1. First-order model for measuring student competency achievement levels

The model is compared with the criteria of goodness of fit statistics (GOF) (Hair et al., 2010; Hidayat et al., 2018; Huang & Hwang, 2019). The model specifications from the CFA output for factor testing in the first order are shown in Table 5. The results of the analysis show that in the testing, all 30 items of competency are declared valid. The P-value (0.129) greater than 0.05 indicates a match between the measurement model concept and field data. The value of RMSEA = 0.040 (≤0.08), then the value of the Incremental Fit Measures which includes IFI = 0.967, TLI = 0.954, CFI = 0.963 (≥0.90) qualifies as a fit model. The first-order CFA test results for the ten-factor model are hypothesized to be very good based on GoF criteria and the model is acceptable.

3.4.2. Confirmatory Factor Analysis: Second-Order Factor Testing

The factor structure of this study is hypothesized and examined. Based on the results of the second-order CFA in the construct of the measurement model for the achievement of student competencies in the architectural engineering department in Indonesia, a path diagram is obtained which is shown in Figure 2.

Results of the model suitability test in the second-order, 12 GOF measures indicate a good model fit for measuring the level of achievement of student competencies in architectural engineering in Indonesia. Thus, it can be concluded that the construct measurement model has met the goodness of fit statistical requirements. The path coefficient for the target competency aspect on the measurement of student competency achievement varies between sub-constructs: KU (0.685), GTk (0.612), MT (0.799), DKB (0.658), TPT (0.818), APLPIG (0.805), KJJ (0.721), EBK (0.801), KUG (0.660), and PKK (0.916). The second-order measurement model for achievement goals also shows acceptable model suitability, P‑value = 0.251, RMSEA = 0.029 (≤0.08), IFI = 0.979, TLI = 0.974, CFI = 0.978 (≥0.90) qualifies as a fit model. The second-order CFA test results for the ten-factor model are also hypothesized to be very good based on GOF criteria and the model is acceptable.

Figure 2. Second-order model for measuring student competency achievement levels

Table 5 lists model fit statistics comparing first and second-order measurement models. Because all the requirements as a fit model are met in the first and second orders, the measurement model for the level of competency attainment of architectural engineering students in Indonesia can be used as a suitable measurement model for collecting data.

Model fit criteria	Parameter fit	Output		Fit	Decision
		First-order	Second-order
Result (Default model)	P-value	0.129	0.251	≥0.05	Good Fit
	Chi-square	376.958	397.058	small	–
	DoF	347	379	small	–
	χ2/df	1.086	1.048	≤5.00	Good Fit
Absolute Fit Measures	GFI	0.724	0.718	≥0.70	Marginal Fit
	RMSEA	0.040	0.029	≤0.08	Close Fit
	RMR	0.026	0.037	≤0.10	Close Fit
Incremental Fit Measures	IFI	0.967	0.979	≥0.90	Good Fit
	TLI	0.954	0.974	≥0.95	Good Fit
	CFI	0.963	0.978	≥0.95	Good Fit
	RFI	0.620	0.634	≥0.05	Good Fit
Parsimonious Fit Measures	PGFI	0.540	0.586	>0.50	Good Fit
	PNFI	0.556	0.593	>0.50	Good Fit
	PCFI	0.768	0.852	>0.50	Good Fit

Table 5. Results of the Construct Model for Measuring Student Competency Achievement

4. Discussion

This study proves the validity and estimates the level of reliability of the instrument to measure the level of competency achievement of students in Indonesia in the architectural engineering education program. The EFA test results show that the respondent data from construction industry participants involves the structure of ten factors, namely: general competencies, technical drawings, construction techniques, the basics of building construction, land measurement techniques, software applications and building interior design, road and bridge construction, construction cost estimation, construction and building utilities, and creativity and entrepreneurship products competencies. These ten factors are hereinafter referred to as aspects of architectural technical competence. The structure is also by the original ten-factor structure of determinants, the measurement of student competency achievement. Although all 40 competency items offered cannot be further analyzed, namely competency items with codes KU3, GT2, MT4, DKB2, TPT2, APLPIG3, KJJ1, EBK4, KUG2, PKK1 because they do not meet the value criteria in the construct validity test. So that 30 items can be accepted for construct validity analysis.

Based on the curriculum currently applied in Indonesia in the department of architectural engineering for vocational high schools, General Competencies consist of faith and piety, nationality and love for the homeland, personal and social character, physical and spiritual health, literacy, and creativity. Furthermore, in the Technical Drawing subject, students learn about how students can present and demonstrate the types and functions of technical drawing equipment, then draw various plane shapes, 2D and 3D projections, and the ability to draw sketches and draw projections with BIM software. In the Statically Structures subject, vocational students are taught material about building structural elements, calculating various forces on building structures, and stresses on a simple beam. Application of SAP 2000 software for engineering mechanics calculations in structural planning.

The subject of Basic Building Construction is a science that learns about the types of building construction, how to carry out a job on a building construction which includes wood, concrete, steel, and soil construction. Introduction and application of Tekla Structure to create and plan building structure models and steel structure models. Furthermore, the Land Measurement Engineering subject provides knowledge so that students understand and can carry out work on surveys and mapping. The use tools for survey measurements are leveling and theodolite, in addition to measuring an area and

The subject of Construction Cost Estimation competencies provides material on how students can calculate the cost budget plan and make progress from the planned schedule of a building construction work. Applying the time schedule with Ms. Projects. Make technical analysis as a reference for determining unit prices. In the Building Construction and Utility subject, students are given the material on how to draw floor plans and details on architectural, structural, and electrical mechanical drawings in building construction. Furthermore, the subject of Creativity and Entrepreneurship Products competencies provides students with materials on how to make, calculate, test prototypes and packaging of a product, do product marketing and make financial reports. In addition, conducting product inspections in accordance with the eligibility criteria and compiling a descriptive description of the products that have been developed.

The CFA approach also confirms the suitability of the two models for measuring competency achievement. As a result, the model can be used to measure the achievement of student competencies in architectural engineering education programs in Indonesia. The reliability of achieving the objectives for the Indonesian sample is largely acceptable. The findings of this study are consistent with previous studies (Chai et al., 2019; Hidayat et al., 2018; Mateus & Hernández-Breña, 2019). The results of the analysis of convergent validity and discriminant validity have fulfilled the multivariate analysis requirements.

The urgency level of competence based on the average assessment results of participants from the construction industry obtained a category that is needed for current job competencies. All competencies consisting of 30 statement items obtained an average of 3.508 out of 4.00. Of the ten competency aspects, the largest mean acquisition was in the competency aspect of land measurement techniques (3.649), then estimation construction costs (3.583), and software applications and building interior design (3.565). This shows that the construction industry places more emphasis on these three competency aspects to serve as job competencies in construction services, especially architectural engineering in Indonesia today. Overall, every competency aspect has met the work competency requirements that are currently required. Meanwhile, of the 30 items of competence that are needed by the construction industry based on obtaining data, namely competence in calculating the estimated cost of construction work, evaluating and improving measurement results in the form of working drawings for construction work, and creating interior designs with elements, materials, models, and accessories in every room.

5. Conclusions

This study aims to investigate the competency aspects needed and must be mastered by students to evaluate the competencies that a graduate has in architectural engineering education in Indonesia. To test this model, an empirical study was conducted. The contribution of this research is multifaceted and provides the following theoretical and practical contributions.

5.1. Theoretical Implications

The first contribution of this research is to reveal competency aspects that can measure the level of competency attainment of students in architectural engineering education in Indonesia. Proving the validity and estimating the reliability of the instrument is acceptable. The results of the EFA analysis found that the structure of measuring the level of achievement of student competencies in architectural engineering education in Indonesia has ten factors. These ten factors are competency aspects that must be mastered by a graduate in architectural engineering education in Indonesia. The analysis confirms that these ten aspects can be a useful scale for measuring the achievement of student competencies for graduates of an architectural engineering graduate in vocational education in the Indonesian context.

The contribution of these two studies revolves around developing a multi-dimensional model to reveal how many competency items are used to measure the level of competency attainment of students in architectural engineering education in Indonesia based on 10 competency aspects. This model was developed based on an intensive literature review and analysis using construct validity testing. The test for competency items that passed the construct validity was 30 out of 40 items offered. This model has substantially explained 72.039% of the variation in the competencies that must be mastered by a graduate of architectural engineering education.

Finally, this research presents an important theoretical contribution to the field of vocational engineering education to architectural engineering education in Indonesia. In addition, this study confirms the validity of the two models developed to measure the level of competency attainment of students in architectural engineering in Indonesia. This is a recommendation to evaluate the competencies possessed by architectural engineering graduates in Indonesia.

5.2. Practical Implications

Because it is very important for vocational-technical education institutions in Indonesia that provide architectural engineering to plan the competencies that must be applied in educational programs as well as in the learning curriculum. On the other hand, the study results and recommendations must be considered to improve the competence of an architectural engineering graduate according to the needs and demands of the construction industry as well as efforts to reduce unemployment in architectural engineering majors in Indonesia. This study provides practitioners with the following practical contributions.

1. 1.The results of the formulation and classification of competencies according to the needs and demands of the construction service industry can be used as material for information, suggestions, and input related to planning, implementation, and supervision of school programs, especially in implementing school curriculum programs, especially architectural engineering education in Indonesia. 

2. 2.As monitoring and evaluation as well as developing cooperation with the construction service industry such as intern ships and practices in industry and compilation of competency curricula in architectural engineering education. 

3. 3.Establishment of cooperation with the construction service industry so that the competence of graduates is by their fields and expectations 

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest concerning the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Alpaslan, M.M. (2019). Examining relations between physics-related personal epistemology and motivation in terms of gender. The Journal of Educational Research, 112(3), 397-410. https://doi.org/10.1080/00220671.2018.1540966

Ayadat, T., Ahmed, D., Chowdhury, S., & Asiz, A. (2020). Measurable performance indicators of student learning outcomes: A case study. Global Journal of Engineering Education, 22(1), 40-50. Available at: http://www.wiete.com.au/journals/GJEE/Publish/vol22no1/06-Ayadat-T.pdf

Baber, H. (2021). Modelling the acceptance of e-learning during the pandemic of COVID-19-A study of South Korea. The International Journal of Management Education, 19(2), 1-15. https://doi.org/10.1016/j.ijme.2021.100503

Badan Pusat Statistik (2020). Keadaan Ketenagakerjaan Indonesia Agustus 2020. Badan Pusat Statistik.

Berestova, A.V., Lazareva, A.V., & Leontyev, V.V. (2020). New Tendencies in Studies within Vocational Education in Russia. International Journal of Instruction, 13(1), 886-900. https://doi.org/10.29333/iji.2020.13157a

Boonk, L.M., Gijselaers, H.J.M., Ritzen, H., & Brand-Gruwel, S. (2020). Student-perceived parental involvement as a predictor for academic motivation in vocational education and training (VET). Journal of Vocational Education & Training, 1-23. https://doi.org/10.1080/13636820.2020.1745260

Broadbent, J., Sharman, S., Panadero, E., & Fuller-Tyszkiewicz, M. (2021). How does self-regulated learning influence formative assessment and summative grade? Comparing online and blended learners. The Internet and Higher Education, 50, 1-8. https://doi.org/10.1016/j.iheduc.2021.100805

Chai, C.S., Jong, M.S.Y., Yin, H., & Chen, M. (2019). Validating and Modelling Teachers’ Technological Pedagogical Content Knowledge for Integrative Science, Technology, Engineering and Mathematics Education. Educational Technology & Society, 22(3), 61-73. Available at: https://www.jstor.org/stable/26896710

Coetzer, A., Susomrith, P., & Ampofo, E.T. (2020). Opportunities to participate in formal and informal vocational learning activities and work-related outcomes in small professional services businesses. Journal of Vocational Education & Training, 72(1), 88-114. https://doi.org/10.1080/13636820.2019.1584637

Creed, P.A., Hood, M., & Hu, S. (2020). Job crafting by students who work and study. International Journal for Educational and Vocational Guidance, 20(2), 331-349. https://doi.org/10.1007/s10775-019-09406-2

Cruz, M.L., Saunders-Smits, G.N., & Groen, P. (2020). Evaluation of competency methods in engineering education: A systematic review. European Journal of Engineering Education, 45(5), 729-757. https://doi.org/10.1080/03043797.2019.1671810

Daryono, R.W., Rochmadi, S., & Hidayat, N. (2021). Development and validation of video-based learning media to increase competency achievement in civil engineering education. Journal of Physics: Conference Series, 1833(1-10), 012022. https://doi.org/10.1088/1742-6596/1833/1/012022

Daryono, R.W., Yolando, A.P., Jaedun, A., & Hidayat, N. (2020). Competency of vocational schools required by construction industry in consultants’ supervisor. Journal of Physics: Conference Series, 1456(1‑10), 012057. https://doi.org/10.1088/1742-6596/1456/1/012057

Ekolu, S.O., & Quainoo, H. (2019). Reliability of assessments in engineering education using Cronbach’s alpha, KR and split-half methods. Global Journal of Engineering Education, 21(1), 24-29. http://www.wiete.com.au/journals/GJEE/Publish/vol21no1/03-Ekolu-S.pdf

Forum Económico Mundial (2019). The Global Talent Competitiveness Index 2020: Global talent in the age of artifcial intelligence. World Economic Forum. Available at: http://www3.weforum.org/docs/WEF_TheGlobalCompetitivenessReport2019.pdf

Gok, T. (2021). The Development of the STEM (Science, Technology, Engineering, and Mathematics) Attitude and Motivation Survey Towards Secondary School Students. International Journal of Cognitive Research in Science Engineering and Education, 9(1), 105-119.
https://doi.org/10.23947/2334-8496-2021-9-1-105-119

Hair, J.F., Black, W.C., Babin, B.C., & Anderson, R.E. (2010). Multivariate data analysis (7th ed.). Englewood Cliffs, NJ.

Harlan, J., & Van Haneghan, J. (2020). Examining the Factor Structure of a Middle School STEM Occupational Values Scale. Journal of Pre-College Engineering Education Research (J-PEER), 10(2), 1-11. https://doi.org/10.7771/2157-9288.1141

Hidayat, R., Zamri, S.N.A.S., & Zulnaidi, H. (2018). Exploratory and Confirmatory Factor Analysis of Achievement Goals for Indonesian Students in Mathematics Education Programmes. EURASIA Journal of Mathematics, Science and Technology Education, 14(12), 1-12. https://doi.org/10.29333/ejmste/99173

Huang, H., & Hwang, G.J. (2019). Facilitating Inpatients’ Family Members to Learn: A Learning Engagement Promoting Model to Develop Interactive E-Book Systems for Patient Education. Educational Technology & Society, 22(3), 74-87. Available at: https://www.jstor.org/stable/26896711

Isac, M.M., Palmerio, L., & van der Werf, M.P.C. (2019). Indicators of (in)tolerance toward immigrants among European youth: An assessment of measurement invariance in ICCS 2016. Large‑Scale Assessments in Education, 7(6), 1-21. https://doi.org/10.1186/s40536-019-0074-5

Kang, J., Hense, J., Scheersoi, A., & Keinonen, T. (2019). Gender study on the relationships between science interest and future career perspectives. International Journal of Science Education, 41(1), 80-101. https://doi.org/10.1080/09500693.2018.1534021

Karstina, S.G., Zechiel, O.N., & Machado, C. (2021). Role of the Kazakhstan-German Cooperation in Improving Scientific Tools for Evaluation of Vocational Education Programs. Vysshee Obrazovanie v Rossii = Higher Education in Russia, 30(1), 132-143. https://doi.org/10.31992/0869-3617-2021-30-1-132-143

Khan, Z., Soundararajan, V., & Shoham, A. (2020). Global post-merger agility, transactive memory systems and human resource management practices. Human Resource Management Review, 30(1), 1-10. https://doi.org/10.1016/j.hrmr.2019.100697

Kline, R.B. (2005). Principles and practice of structural equation modeling. The Guilford Press.

Kyriakides, L., & Charalambous, E. (2020). Testing the theoretical framework of whole school interventions aiming to promote student learning outcomes: The contribution of multilevel structural equation modelling. International Journal of Research & Method in Education, 43(4), 395-412. https://doi.org/10.1080/1743727X.2020.1795114

Liou, P. (2021). Students’ attitudes toward science and science achievement: An analysis of the differential effects of science instructional practices. Journal of Research in Science Teaching, 58(3), 310-334. https://doi.org/10.1002/tea.21643

Liu, X., Chen, Y., Yang, Y., Liu, B., Ma, C., Craig, G.R. et al. (2020). Understanding vocational accounting students’ attitudes towards sustainable development. Journal of Vocational Education & Training, 1-21. https://doi.org/10.1080/13636820.2020.1760333

Mateus, J.C., & Hernández-Breña, W. (2019). Design, Validation, and Application of a Questionnaire on Media Education for Teachers in Training. Journal of New Approaches in Educational Research, 8(1), 34-41. https://doi.org/10.7821/naer.2019.1.329

Mingaleva, Z., & Vukovic, N. (2020). Development of Engineering Students Competencies Based on Cognitive Technologies in Conditions of Industry 4.0. International Journal of Cognitive Research in Science, Engineering and Education, 8(Special issue), 93-101. https://doi.org/10.23947/2334-8496-2020-8-SI-93-101

Nurtanto, M., Arifin, Z., Sofyan, H., Warju, W., & Nurhaji, S. (2020). Development of Model for Professional Competency Assessment (PCA) in Vocational Education: Study of the Engine Tune-Up Injection System Assessment Scheme. Journal of Technical Education and Training, 12(2), 34-45. https://doi.org/10.30880/jtet.2020.12.02.004

Olazaran, M., Albizu, E., Otero, B., & Lavía, C. (2019). Vocational education-industry linkages: Intensity of relationships and firms’ assessment. Studies in Higher Education, 44(12), 2333-2345. https://doi.org/10.1080/03075079.2018.1496411

Pala, F.K., & Erdem, M. (2020). Development of a participation style scale for online instructional discussions. Educational Technology Research and Development, 68(6), 3213-3233. https://doi.org/10.1007/s11423-020-09817-x

Prasojo, L.D., Habibi, A., Mukminin, A., Sofyan, S., Indrayana, B., & Anwar, K. (2020). Factors Influencing Intention to Use Web 2.0 in Indonesian Vocational High Schools. International Journal of Emerging Technologies in Learning (IJET), 15(05), 100-118. https://doi.org/10.3991/ijet.v15i05.10605

Pratomo, H., Priyambodo, E., & Wiyarsi, A. (2020). Vocational High School Students Chemical Literacy on Context-Based Learning: A Case of Petroleum Topic. Turkish Journal of Science Education, 17(1), 147‑161. https://doi.org/10.36681/tused.2020.18

Rodgers, D.B., Reed, D.K., Houchins, D.E., & Aloe, A.M. (2020). The writing abilities of juvenile justice youths: A confirmatory factor analysis. The Journal of Educational Research, 113(6), 438-451. https://doi.org/10.1080/00220671.2020.1854160

Rodríguez-Santero, J., Torres-Gordillo, J.J., & Gil-Flores, J. (2020). Confirmatory Factor Analysis of a Questionnaire for Evaluating Online Training in the Workplace. Sustainability, 12(11), 4629. https://doi.org/10.3390/su12114629

Saifurrahman, M., Sudira, P., & Daryono, R.W. (2021). The Determinant Factor of the Principal Leadership Solutions in Facing the 21st-Century Learning. Jurnal Pendidikan dan Pengajaran, 54(2), 14. https://doi.org/10.23887/jpp.v54i2.34102

Saleem, M., Kamarudin, S., Shoaib, H.M., & Nasar, A. (2021). Influence of augmented reality app on intention towards e-learning amidst COVID-19 pandemic. Interactive Learning Environments, 29, 1-15. https://doi.org/10.1080/10494820.2021.1919147

ShayesteFar, P. (2020). A model of interplay between student English achievement and the joint affective factors in a high-stakes test change context: Model construction and validity. Educational Assessment, Evaluation and Accountability, 32(3), 335-371. https://doi.org/10.1007/s11092-020-09326-8

Špaček, R., Legény, J., & Gregor, P. (2020). Challenge and response at all levels in sustainable architecture education. World Transactions on Engineering and Technology Education, 18(1), 18-23. Available at: http://www.wiete.com.au/journals/WTE&TE/Pages/Vol.18,%20No.1%20(2020)/03-Spacek-R.pdf

Setyadi, M.R.A., Triyono, M.B., & Daryono, R.W. (2021). The influence of industrial work practices and workshop infrastructure facilities on work readiness of students. Journal of Physics: Conference Series, 1833(1-8), 012029. https://doi.org/10.1088/1742-6596/1833/1/012029

Subdirektorat Statistik Konstruksi (2019). Direktori Perusaan Konstruksi: Buku II Pulau Jawa, bali, Nusa Tenggara, dan Kepulauan Maluku.

Supriyadi, E., Indro, H.Y., Priyanto, E., & Surwi, F. (2020). Students’ Evaluation on Teaching in Vocational and Technical Schools. International Journal of Instruction, 13(2), 621-636. https://doi.org/10.29333/iji.2020.13242a

Suwanroj, T., Leekitchwatana, P., & Pimdee, P. (2019). Confirmatory factor analysis of the essential digital competencies for undergraduate students in Thai higher education institutions. Journal of Technology and Science Education, 9(3), 340. https://doi.org/10.3926/jotse.645

Syed, H.S.S. (2018). Measuring attitude towards learning science in Malaysian secondary school context: Implications for teaching. International Journal of Science Education, 40(16), 2044-2059. https://doi.org/10.1080/09500693.2018.1518614

Triyono, B.M., Mohib, N., Kassymova, G.K., Pratama, G.N.I.P., Adinda, D., & Arpentieva, M.R. (2020). The Profile Improvement of Vocational School Teachers’ Competencies. Vysshee Obrazovanie v
Rossii = Higher Education in Russia, 29(2), 151-158. https://doi.org/10.31992/0869-3617-2020-29-2-151-158

Tsuey-Fen, L. (2020). The Relationship between Vocational High School Students Studying in their Field of Specialization, and their Development after Graduation. International Journal of Innovation, Creativity and Change, 11(11), 56-66. Available at: https://www.ijicc.net/images/Vol11Iss11/111108_Fen_2020_E_R.pdf

Tucker, S.L., & Hughes, A.J. (2020). Endorsement of Career and Technical Education: Phenomena Influencing Core-Subject Teacher Perceptions. Journal of Technology Education, 31(2), 40-55. https://doi.org/10.21061/jte.v31i2.a.3

Vlasova, T., Krasnova, E., Abraukhova, V., & Safontseva, N. (2018). Dynamics of russian employers’ perceptions on the training quality of specialists in the system of secondary vocational education. Journal of Social Studies Education Research, 9(1), 76-88. https://doi.org/10.17499/jsser.46598

Wheelahan, L. (2015). Not just skills: What a focus on knowledge means for vocational education. Journal of Curriculum Studies, 47(6), 750-762. https://doi.org/10.1080/00220272.2015.1089942

Ye, W., Strietholt, R., & Blömeke, S. (2021). Academic resilience: Underlying norms and validity of definitions. Educational Assessment, Evaluation and Accountability, 33(1), 169-202. https://doi.org/10.1007/s11092-020-09351-7

Yudiono, H. (2018). An industrial competency-based curriculum alignment model. World Transactions on Engineering and Technology Education, 16(1), 18-225. http://www.wiete.com.au/journals/WTE&TE/Pages/Vol.16,%20No.1%20(2018)/03-Yudiono-H.pdf

Zare, M., & Nastiezaie, N. (2019). The relationship between distributed leadership and work self‑efficacywith the mediating role of academic optimism of the teacher. New Educational Review, 58(4), 169-178.Available at: https://tner.polsl.pl/e58/a13.pdf

---