Structural variable validation of an Online Learning Response Behavior (OLRB) instrument: A comparison analysis of three extraction methods of Exploratory Factor Analysis

Abstract

Purpose

Measuring internal response of online learning is seen as fundamental to absorptive capacity which stimulates knowledge assimilation. However, the evaluation of practice and research of validated instruments that could effectively measure online learning response behavior is limited. Thus, in this study, a new instrument was designed based on literature to determine the structural variables that exist in the online learning response behavior.

Design/methodology/approach

A structured survey was designed and distributed to 410 Malaysian students enrolled in higher-education institutions. The questionnaire has 38 items, all of which were scored using a seven-point likert scale. To begin with, exploratory factor analysis with three types of extraction methods (i.e. principal component, principal axis factoring and maximum likelihood) was used as the method for comparing the outcomes of each extraction method's grouping variables by constantly using a varimax rotation method. In the second phase, reliability analysis was performed to determine the reliability level of the grouping variables, and finally, correlation analysis was performed to determine the discriminant nomological validity of the grouping variables.

Findings

The findings revealed that nine grouping variables were retrieved, with all items having a good value of factor loading and communalities, as well as an adequate degree of reliability. These extracted variables have good discriminant and nomological validity, as evidenced by correlation analysis, which confirmed that the directions of relationships among the extracted dimensions follow the expected theory (i.e. positive direction) and the correlation coefficient is less than 0.70.

Research limitations/implications

This study proposes a comprehensive set of questionnaires that measure the student's online learning response behavior. These questionnaires have been developed on the basis of an extensive literature review and have undergone a rigorous process of validity and reliability for the purpose of enhancing students' online learning response behavior.

Originality/value

This study's findings will aid academic practitioners in assessing the online learning response behavior of students, as well as enhancing the questionnaire's boost factor when administered in an online learning environment.

Citation

Azman Ong, M.H., Mohd Yasin, N. and Ibrahim, N.S. (2022), "Structural variable validation of an Online Learning Response Behavior (OLRB) instrument: A comparison analysis of three extraction methods of Exploratory Factor Analysis", Asian Association of Open Universities Journal, Vol. 17 No. 2, pp. 134-146. https://doi.org/10.1108/AAOUJ-04-2022-0054

1. Introduction

In a typical setting, universities and colleges are urged to incorporate blended learning for all academic programs. Blended learning involves a blend of traditional face-to-face training with digital learning. In blended learning classes, the learning materials were primarily provided through face-to-face interaction, but they were also accessible through a robust learning management system (LMS) to support and increase instructor–student and student–student connection after class. However, as of March 2020, all higher-education institutions in Malaysia have been shuttered owing to the COVID-19 epidemic. During the implementation of this measure, students will still be in the beginning of the semester and will be required to transition to an online learning mode. This has resulted in a rapid shift to online instructional delivery. To assist in limiting the spread of the virus that causes COVID-19, some schools have elected to cancel all face-to-face sessions, including laboratories and other learning opportunities, and have demanded that teachers transfer their courses online. Globally, the majority of colleges and institutions outside of Malaysia have used the same approach as one of the restraints to minimize the COVID-19 viral transmission process.

Due to this, digitalization has contributed to a surge in interest among educational scholars worldwide, particularly in developing nations and the Asian region. Responses to technology adoption are characterized by extremely disparate attitudes, digital skills and degrees of readiness (Yasin et al., 2020). The phenomenon has also led to an emphasis on behavioral intention in the context of individual components such as cognition and motivation (Costley and Lange, 2017; Oluwajana et al., 2019), in addition to learning tools (Wammes et al., 2019). In the context of this study, behavioral engagement is demonstrated by students' participation such as idea generation, participation in the classroom and extracurricular activities, concentration, assignment submission and adherence to the instructor's directions (Reschly and Christenson, 2012). All these variables are associated with a higher number of behavioral interventions and educational tools for students to increase their involvement in online classrooms for successful learning (Hughes et al., 2020). Consequently, it is vital to research how novice digital learners react to learning.

Although there are similarities in the research issues on behavioral response, the approaches used to investigate students' perspectives on learning behavior have not been consistently verified. Therefore, educational researchers utilizing digital platforms must have access to instruments or measures that have been cross-validated across varied groups (Guo and Liu, 2018; Hsiao et al., 2019). This would increase the reliability, generalizability and practicality of online learning studies.

Still, validated instruments are used rather infrequently in research on the response behavior of students to online learning (Yasin et al., 2020). As a result, the purpose of this study is to investigate the procedures that may be used to determine the validity and reliability of the instrument. Evaluation of students' abilities to comprehend, produce and evaluate information offered within the context of a Malaysian online educational setting is carried out. The same method is advantageous for designing evaluation tools to measure other aspects of a student's readiness, such as their perspectives and preferred learning style. Consequently, we established two aims for this study: (1) to identify significant factors for assessing the online learning response behavior in higher-education institutions and (2) to assess the structural variable validity and reliability of the online learning response behavior questionnaire.

2. Methodology

2.1 Study design

In this study, both a cross-sectional applied survey and a combination of quantitative methodologies were utilized. According to Creswell (2014), the purpose of this study is to validate and investigate the structural aspects of the targeted structures, and hence, these two research methodologies were chosen. Moreover, the phrase of the operational items in the Likert scale that were adopted from the previous study was amended in accordance with the study context. Therefore, it is vital for the researchers to revalidate this instrument using a suitable quantitative technique (Hair et al., 2010).

This study is particularly interested in Malaysian public university students having at least one year of experience using an online learning platform during the Movement Control Order (MCO). In the lack of a sample frame for selecting respondents, this study used a combination of convenience and judgmental sampling methods, both of which are regarded appropriate (Creswell, 2014).

There were 410 Malaysian public university students who volunteered to participate in this study by responding to a social media–distributed questionnaire. The sample size for this study is sufficient since the minimal sample size for a large population (1m or more) is 384 (Krejcie and Morgan, 1970). In addition, Hair et al. (2010) and Tabachnick and Fidell (2007) recommend evaluating at least 350 exploratory factor analysis (EFA) samples. Consequently, a sample size of 410 respondents may be considered adequate for this study.

2.2 Process of selecting the sources of a questionnaire using systematic review

The process of constructing a questionnaire begins with the collection of accessible definitions, concepts and practicalities of the key notion of students' learning response behavior from the previous academic literature. This procedure begins with a search of the most cited sources pertaining to the assessment of learning response behavior in the Scopus database, which contains only peer-reviewed scientific publications, in an effort to exclude duplicate sources.

Using the Boolean search operator “TITLE ABS-KEY” (“learning response behavior”), a total of 167 relevant sites were retrieved (survey OR scale OR questionnaire OR model). From these, 108 relevant sources remained because they met the selection criteria for the inclusion of the studies, which are as follows:

1. The research is published in a scientific publication.

2. The study consists of empirical research with survey-based learning response behavior measures.

These 108 relevant sources were retrieved via the digital library service provided by the Universiti Teknologi MARA or via the open access option, with the material regarding survey-based assessment of learning response behavior being the only topic of interest. From a total of 108 suitable sources, 101 were eliminated from this procedure for the following reasons:

1. Title and abstract irrelevance (15 papers);

2. Conceptual papers (23 papers);

3. Papers that employ a qualitative technique (43 papers);

4. The articles' content did not address the optimal usage of questionnaire methodology (13 papers);

5. The papers were not authored in English (7 papers).

Following the screening procedure, seven papers that met the aforementioned selection criteria were included in the systematic review. Figure 1 depicts the flow diagram of the preferred reporting items for systematic reviews and meta-analyses (PRISMA)-based search procedure used in this research. We integrate two major types of methodological criteria in the process of framework or item assessment: evaluation of a survey-based operationalization measurement scale (Table 1) and evaluation of measurement quality (Table 2). These stages, according to Carmines and Zeller (1979), are required but not sufficient for obtaining a trustworthy and valid structural variable instrument. Based on these two methodological criteria, all seven of the selected studies fit the requirements, with 100% of the criteria met (Table 3).

2.3 Questionnaire items

To evaluate the learning response behavior of students, the researcher developed instruments based on a systematic review analysis that matched the PRISMA criteria and two major methodological criteria, as shown in Table 3. A 38-item survey was modified using a seven-point Likert scale, where 1 corresponds to strongly disagree and 7 corresponds to strongly agree, and was randomly distributed in the instrument setup. Adapted from Yasin et al. (2020), three elements of the student's learning readiness were used (i.e. Q1, Q5, Q6). Based on Gao et al. (2020), the student's learning engagement is defined as the reflection of cognitive and emotional feelings representing their behavior in response to an online learning environment consisting of seven modified items with two dimensions: cognitive engagement (i.e. Q2, Q7, Q9) and emotional engagement (i.e. Q2, Q7, Q9) (i.e. Q3, Q4, Q8, Q10). This instrument consists of eight items to assess the learning motivation construct referred to as the beliefs, intentions and feelings that motivate students to learn, modified from the study by Liu (2020): four items for intrinsic motivation (i.e. Q12, Q13, Q16 and Q19) and four items for extrinsic motivation (i.e. Q11, Q14, Q15 and Q17).

Alternative assessment, on the other hand, is the variable used to evaluate students' learning and support their cognitive development by regularly examining and providing feedback on their work. Thus, eight questions (i.e. Q21, Q24, Q25, Q27, Q30, Q31, Q33 and Q35) were adopted from the study by Leeuwenkamp et al. (2019). This study also modified the Awidi and Paynter (2019) student's learning experience, which is defined as the process of knowledge construction and converging ideas developed through social interactions, with four items (i.e. Q22, Q23, Q29 and Q32), whereas the remaining four indicators were adapted from the study by Utriainen et al. (2018) to evaluate the critical thinking learning experience (i.e. Q20, Q26, Q28 and Q37). This instrument also incorporated students' response behavior toward online learning, adapted from the research of Bui et al. (2020), with four items (i.e. Q34, Q36, Q38 and Q18). The questionnaire is supplied in Appendix. Importantly, the premise of utilizing this questionnaire as an elicitation tool is that the unit of analysis must have online learning experience as the purpose of this questionnaire is to measure the respondent's online learning response behavior. In addition, the responder must be an adult learner in order to provide an accurate response to each questionnaire item as the items included in this questionnaire tend to be psychological measures that require exact evaluation.

2.4 Statistical analysis

The IBM SPSS version 21.0 statistical software was used for data input and analysis methods. The EFA and correlation analysis were used because the primary purpose of this study was to validate and investigate the structural variable validity and reliability of the instrument after undergoing the process of sentence modification and Likert-scale use by the original authors (Hair et al., 2010; Tabachnick and Fidell, 2007). As recommended by Hair et al. (2010) and William et al. (2010), this form of study enables the researcher to validate and modify the employed indicators (2010). In this work, three EFA extraction methods were utilized to compare the instrument's valid structure. The first extraction method that was used is a combination of principal component (PC) extraction method with a varimax rotation (PC + varimax), the second and third extraction methods were principal axis factoring (PAF) extraction method with a varimax rotation (PAF + varimax) and the final extraction method was maximum likelihood extraction method with a varimax rotation (ML + varimax).

Using a Kaiser–Meyer–Olkin (KMO) index, the existence of a nonidentical matrix and the sufficiency of the covariance matrix among the indicators must be determined at the initial phase of the study. As proposed by Field (2009) and Pallant (2010), this index value must exceed 0.60. As a support for this conclusion, Bartlett's test of sphericity must be significant (Pallant, 2010); hence, it confirms the existence of a nonidentical matrix and that the covariance matrix among the indicators is adequate for the EFA. Thompson and Daniel (1996) suggested that, in the second stage of this study, a researcher should use several criterion techniques to identify the number of dimensions that should be retrieved from the constructs and to check and validate the number of dimensions that exist in the constructs. Therefore, only dimensions with eigenvalues more than 1.00 are retrieved from the analysis, and the total percentage of variance extracted should be greater than 60%, as recommended by Hair et al. (2010), Field (2009), Tabachnick and Fidell (2007) and Thompson and Daniels (1996). In addition, Watkins (2000) recommends using the Monte Carlo PCA parallel analysis simulation eigenvalues to calculate the number of extracted variables. In this study, only extracted variables having an eigenvalue larger than the eigenvalue of the Monte Carlo simulation are considered extracted.

Examining the value of factor loading and communalities from each extraction technique (such as PC + varimax, PAF + varimax and ML + varimax) is the next step in validating the indicators under each dimension extract. According to Field (2009), for an exploratory study on refining and validating indicators using the EFA, the values of factor loading and communalities must be more than 0.40 and the sample size must exceed 350. This is a crucial step as it ensures that the retrieved dimensions are different, meaningful and valid with a sufficient degree of confidence. In the last step of the EFA methods, a Cronbach's alpha reliability test was used to evaluate the internal consistency of the indication group. Nunnally and Bernstein (1994) indicated that a threshold of 0.70 or above can be used to show the reliability of grouped elements.

As proposed by Creswell (2014), Hair et al. (2010) and Kaptein (2008), discriminant validity and nomological validity were assessed following the EFA and reliability analysis. Using correlation analysis, they describe nomological validity as the act of examining the logical and meaningful link between dimensions that follows the predicted direction of association (i.e. positive relationship or negative relationship). In addition, Hair et al. (2010) and Kaptein (2008) used the correlation coefficient to assess discriminant validity using correlation analysis. Observation suggests that when the correlation coefficient is less than 0.70, the extracted dimensions have discriminant validity.

3. Data analysis and results

3.1 Convergence validity from the EFA and reliability analysis

The findings of multiple criterions for determining the number of variables to be retrieved from this instrument's 38 total items are summarized in Table 4. The study shown in Table 4 revealed that the EFA analysis for these three extraction techniques should extract nine variables from a total of 38 items since the first nine eigenvalues under Kaiser's criteria (e.g. range: 1.059 to 5.762) were greater than 1.00. Moreover, when comparing these Kaiser's eigenvalues with simulation eigenvalues from the Monte Carlo analysis, the same result occurs: only the first nine Kaiser's eigenvalues are greater than the first nine simulation eigenvalues from the Monte Carlo analysis, confirming that nine variables should be extracted from a total of 38 items. The cumulative percentage of variation explained for these nine extracted variables is 83.83%, indicating that the extraction of these nine variables from a total of 38 indicators is legitimate. Bartlett's test of sphericity for the items was likewise highly significant (Χ² = 20419.75, p < 0.01). On the other hand, the KMO index for these three extraction techniques of the EFA was 0.974. Therefore, the covariance matrix for these 38 items was not a matrix of identity. Consequently, all elements in the instruments can be used for the EFA if all three extraction techniques are included.

Table 5 displays the factor loading and communality value for each item based on the three extraction methods used in the EFA. As with the first EFA extraction approach (i.e. PCA + varimax), the analysis revealed that all items had loading (range: 0.504 to 0.903) and communality (range: 0.709 to 0.963) values greater than the minimal threshold value of 0.40. Alternatively, the second extraction technique of the EFA (i.e. PAF + varimax) yields the same result, with all 38 items having loading (range: 0.439 to 0.874) and communality (range: 0.508 to 0.915) values greater than 0.40. The loading (range: 0.387 to 0.948) and communality (range: 0.373 to 0.919) values were just below 0.40 for the third extraction technique of the EFA, ML + varimax. Since the sample size for this research (n = 410) is regarded to be substantial, the ML + varimax technique's findings can be considered reliable. Therefore, it is possible to infer that all 38 items were valid and could be utilized to measure the intended components in this study.

Based on the grouping pattern of indicators, these three EFA extraction techniques exhibit a similar pattern. Student's learning readiness, student's perception of alternative assessment provided and student's response behavior toward online learning factors were discovered to have maintained their one-dimensional structure. Based on these three extraction analysis approaches, all analyses demonstrate that the dimensions of the student's learning engagement, student's learning motivation and student's learning experience variables, each of which has a two-dimensional structure, were preserved. Cronbach's alpha scores for the grouped indicators ranged from 0.902 to 0.965, suggesting that all nine grouped indicators meet the minimal threshold value of 0.70. Therefore, it can be stated that all grouped indicators were legitimate and measured the variables consistently.

3.2 Discriminant and nomological validity from the correlation analysis

The nomological and discriminant validities of the extracted variables were examined using correlation analysis, and the results are presented in Table 6. At the 5% level of significance, the analysis revealed a substantial positive association between the extracted variables. All extracted variables may thus be confirmed to have good nomological validity since all bivariate correlations correspond to the predicted bivariate connection, which is a positive relationship. In addition, correlation coefficients for the analysis ranged from 0.410 to 0.667% when examined using the correlation coefficient. Since all correlation coefficients are less than 0.70, it can be concluded that the extracted variables provide a satisfactory level of discrimination between the variables.

4. Conclusion

This study illustrated the statistical processes for verifying, investigating and comparing the variable structures in the survey instrument utilizing three extraction techniques of the EFA and correlation analysis for the online learning response behavior instrument. Based on these three extraction techniques of the EFA, the validity of all items utilized to measure the targeted variables is satisfactory. This is due to the fact that all three different extraction methods for the EFA (i.e. PC + varimax, PAF + varimax and ML + varimax) show a similar pattern of item grouping after undergoing the process of item sentence modification and Likert-scale weighting at the start of instrument development. In addition, the examination of reliability revealed that all extracted variables had sufficient and satisfactory internal consistency, as all Cronbach's alpha values were greater than 0.70. The correlation analysis also confirmed that all extracted variables from the EFA have good nomological and discriminant validity as the bivariate relationship between the extracted variables was significant and followed the expected direction of the relationship, and all correlation coefficients were less than 0.70.

On the other hand, based on the grouped elements from the EFA, student's learning readiness may be viewed as a continuous process that shows the students' mental preparation to accept the diverse method of technology-based learning (Yasin et al., 2020). According to Gao et al. (2020), student's learning engagement may be characterized as a technique that guides students in cultivating the instructor-acquired information and can be classified as cognitive engagement and emotional engagement. The fourth and fifth grouped factors, internal and extrinsic motivation, respectively, can be considered the total learning motivation of a learner. In this study, extrinsic motivation is defined as motivating students to attain their objective by providing an external incentive throughout the learning process on an online platform in the hope that students will acquire a strong interest in learning via the online platform (Liu, 2020). In contrast to extrinsic motivation, the intrinsic motivation variable was measured by the internal needs of the students during the process of learning on an online platform, where internal incentives such as creating an enjoyable environment during the learning process are anticipated to bolster intrinsic motivation among the students (Liu, 2020).

On the basis of the quality of alternative assessments offered by instructors, students' perceptions of alternative assessments may be interpreted as their perceptions of their learning progress and information about their desired learning goals (Leeuwenkamp et al., 2019). Experience with competencies and critical thinking are two criteria that may be used to assess a student's learning experience. This variable represents the entire learning experience encountered by the learner during the learning process. According to Utriainen et al. (2018) and Awidi and Paynter (2019), a student's learning experience can also be viewed as the construction of knowledge by the student through interaction with others or as the development of competencies and the capacity to apply knowledge to other contexts, which is critical thinking experience. Finally, student's reaction behavior toward online learning variable may be described as the behavior of students continuing to use the online learning platform as the learning medium in the future and introducing the online learning platform to their social circle as the optimal means of learning.

5. Discussion

Based on the findings of the EFA and correlation analysis, this online learning response behavior instrument building procedure is suitable for measuring the intended constructs. Due to the present COVID-19 epidemic, this study method may also be categorized as significant processes as this instrument can serve as an alternative beneficial tool for monitoring the entire online learning response behavior of students. This research also stressed the usefulness of the EFA and correlation analysis for verifying the survey instruments after the items have undergone a process of alteration from the perspectives of convergent validity, discriminant validity and nomological validity.

In addition, the outcomes of this study suggested that this questionnaire has the potential to be utilized internationally, not just in the Malaysian higher-education institutions setting, because most higher-education institutions worldwide have shifted to an online learning platform. In addition, all the factors used to quantify the online learning response behavior occur in the majority of higher-education institutions. This questionnaire is simple to administer because it consists of 38 items that may be categorized as small. Consequently, the likelihood of administrative data inaccuracy is likewise minimal. Since the structural themes of all variables revealed in this study lean toward the positive theme, the interpretation of the variables should likewise be positive. Using the average score approach, the score of each variable's structure must be high for a positive interpretation.

6. Limitations

In reference to this research, one of the drawbacks of this study is the lack of a causal analysis to identify the global questionnaire fit indices. This study thus recommends a causal analysis to further explore the global fit indices of the instrument's validity by employing a covariance-based confirmatory factor analysis or partial least square consistency factor analysis. In addition, for the purpose of calculating an overall global fit index, this instrument may be administered to secondary students and private higher-education students in both urban and rural regions. Moreover, our systematic review approach is restricted to terms that assess just the learning response behavior, and only seven papers are best accessible for reviews. Therefore, it is advised that, for future study, a more comprehensive word, such as students' response behavior, and a greater number of publications should be chosen for examination.

Figures

Figure 1

Flow diagram of the searching procedure using PRISMA procedure

Table 1

Evaluation of a survey-based operationalization assessment scale

Category	Criteria	Code
Conceptual background of item scale	Report the definition of concepts	C1
	Report the definition of dimensions	C2
	Report the source of items	C3
Sample information	Sample size of the study is higher than 300 samples	C4
	Report the sample characteristics	C5
Statistical analysis	Report the descriptive statistics of the items (mean, standard deviation, etc.)	C6
	Report the correlation/covariance matrix or coefficients	C7
Measurement model fit	Report the result of exploratory factor analysis or confirmatory factor analysis	C8

Table 2

Evaluation of measurement quality

Category	Criteria	Code
Construct validity	Report the convergent validity of the measurement criteria (loading > 0.7 or weight > 0.4 and AVE > 0.5)	C9
	At least one discriminant validity criterion was reported (Fornell–Larcker, HTMT or cross-loading discriminant tests)	C10
	Report the criteria validity by indicating that the correlation between interconstructs is greater than 0.5	C11
Internal consistency	The stated Cronbach's alpha must be greater than 0.7	C12

Note(s): AVE = Average variance explanation, HTMT = Heterotriat-Monotrait ratio of correlations

Table 3

Evaluation of the 7 selected studies

Study	Criteria code
	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12
(1)	/	/	/	/	/	/	/	/	/	/	/	/
(2)	/	/	/	/	/	/	/	/	/	/	/	/
(3)	/	/	/	/	/	/	/	/	/	/	/	/
(4)	/	/	/	/	/	/	/	/	/	/	/	/
(5)	/	/	/	/	/	/	/	/	/	/	/	/
(6)	/	/	/	/	/	/	/	/	/	/	/	/
(7)	/	/	/	/	/	/	/	/	/	/	/	/

Note(s): Criteria code refers to Table 1 and Table 3. (1) = Yasin et al. (2020), (2) = Gao et al. (2020), (3) = Liu (2020), (4) = Leeuwenkamp et al. (2019), (5) = Awidi and Paynter (2019), (6) = Utriainen et al. (2018), (7) = Bui et al. (2020)

Table 4

Multiple criteria for factors to be extracted

Component number	Initial eigenvalue	Monte Carlo simulation eigenvalue	Cumulative % variance explained	Decision
1	5.762	1.554	43.78	Accept to extract
2	2.879	1.407	50.26	Accept to extract
3	1.802	1.331	55.68	Accept to extract
4	1.315	1.293	63.52	Accept to extract
5	1.269	1.202	68.52	Accept to extract
6	1.187	1.157	72.13	Accept to extract
7	1.124	1.097	76.97	Accept to extract
8	1.099	1.071	80.81	Accept to extract
9	1.059	1.043	83.83	Accept to extract
10	0.991	1.036	–	Reject to extract
11	0.788	1.011	–	Reject to extract

Note(s): Only the initial 11 out of 38 components were reported

Table 5

Comparative results of three extraction methods of the EFA

Components and items included	PC + varimax		PAF + varimax		ML + varimax
	λ	δ	λ	δ	λ	δ
Student's learning readiness
Q1	0.889	0.875	0.653	0.527	0.718	0.503
Q5	0.817	0.853	0.613	0.508	0.602	0.434
Q6	0.903	0.963	0.874	0.717	0.948	0.634
Eigenvalue = 2.879, % variance explained = 18.24%. Cronbach's alpha = 0.902
Cognitive engagement
Q2	0.688	0.850	0.559	0.803	0.466	0.790
Q7	0.685	0.896	0.578	0.891	0.520	0.919
Q9	0.757	0.842	0.667	0.796	0.618	0.799
Eigenvalue = 1.802, % variance explained = 16.72%. Cronbach's alpha = 0.929
Emotional engagement
Q3	0.734	0.883	0.727	0.889	0.713	0.889
Q4	0.700	0.851	0.705	0.865	0.719	0.888
Q8	0.697	0.709	0.612	0.609	0.599	0.602
Q10	0.690	0.855	0.688	0.861	0.680	0.863
Eigenvalue = 1.315, % variance explained = 7.21%. Cronbach's alpha = 0.938
Intrinsic motivation
Q12	0.594	0.890	0.628	0.915	0.474	0.601
Q13	0.574	0.877	0.613	0.899	0.446	0.568
Q16	0.531	0.837	0.468	0.810	0.391	0.406
Q19	0.504	0.799	0.473	0.701	0.387	0.373
Eigenvalue = 1.099, % variance explained = 3.24%. Cronbach's alpha = 0.918
Extrinsic motivation
Q11	0.508	0.836	−0.633	0.825	0.519	0.617
Q14	0.519	0.819	−0.588	0.822	0.396	0.413
Q15	0.557	0.838	−0.659	0.845	0.487	0.509
Q17	0.728	0.822	−0.538	0.602	0.606	0.698
Eigenvalue = 1.059, % variance explained = 2.56%. Cronbach's alpha = 0.925
Student's perception toward alternative assessment
Q21	0.666	0.819	0.621	0.773	0.619	0.765
Q24	0.673	0.804	0.648	0.771	0.649	0.769
Q25	0.749	0.854	0.733	0.839	0.734	0.837
Q27	0.700	0.863	0.701	0.868	0.701	0.866
Q30	0.677	0.812	0.629	0.750	0.629	0.744
Q31	0.722	0.826	0.678	0.787	0.676	0.785
Q33	0.668	0.821	0.649	0.801	0.651	0.798
Q35	0.638	0.838	0.622	0.815	0.619	0.816
Eigenvalue = 5.762, % variance explained = 19.34%. Cronbach's alpha = 0.965
Competency learning experience
Q22	0.694	0.775	0.555	0.589	0.546	0.569
Q23	0.692	0.851	0.673	0.869	0.627	0.829
Q29	0.683	0.852	0.674	0.858	0.668	0.869
Q32	0.665	0.850	0.658	0.872	0.649	0.922
Eigenvalue = 1.187, % variance explained = 5.30%. Cronbach's alpha = 0.934
Critical thinking learning experience
Q20	0.703	0.829	−0.688	0.806	0.702	0.823
Q26	0.645	0.841	−0.665	0.840	0.659	0.831
Q28	0.620	0.776	−0.639	0.792	0.640	0.755
Q37	0.677	0.792	−0.663	0.772	0.689	0.787
Eigenvalue = 1.269, % variance explained = 5.95%. Cronbach's alpha = 0.914
Student's response behavior toward online learning
Q34	0.613	0.866	0.587	0.846	0.684	0.871
Q36	0.592	0.854	0.469	0.819	0.537	0.744
Q38	0.509	0.821	0.439	0.759	0.519	0.718
Q18	0.545	0.813	0.506	0.739	0.619	0.842
Eigenvalue = 1.124, % variance explained = 5.27%. Cronbach's alpha = 0.929

Note(s): The negative signs in the loading values indicate that these items were located at the negative axis of the rotation method; all three extraction methods of the EFA produce the same value of eigenvalues and hence produce the same variance explained from the extracted components which is 83.83%; PC = principal component; PAF = principal axis factoring; ML = maximum likelihood; λ = loading value; δ = communality value; refer to Appendix for code item definition

Table 6

Correlation analysis among extracted dimensions from the EFA

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
(1)	1.000
(2)	0.523*	1.000
(3)	0.591*	0.657*	1.000
(4)	0.554*	0.609*	0.626*	1.000
(5)	0.567*	0.613*	0.603*	0.667*	1.000
(6)	0.621*	0.628*	0.631*	0.619*	0.598*	1.000
(7)	0.410*	0.507*	0.483*	0.492*	0.489*	0.623*	1.000
(8)	0.422*	0.531*	0.507*	0.476*	0.463*	0.619*	0.661*	1.000
(9)	0.532*	0.498*	0.452*	0.468*	0.509*	0.637*	0.610*	0.598*	1.000

Note(s): (1) = student's learning readiness; (2) = cognitive engagement; (3) = emotional engagement; (4) = intrinsic motivation; (5) = extrinsic motivation; (6) = student's perception toward alternative assessment given; (7) = competency experience; (8) = critical thinking experience; (9) = student's response behavior toward online learning; n = 410; *p < 0.05

Table A1

Question wording and code

Code	Item description
Q1	I have a private place in my home or at work that I can use for extended periods
Q2	Using the online platform prompts me actively devote myself to study
Q3	I enjoy the learning because using the online platform
Q4	The course is interesting because using the online platform
Q5	I have adequate time that will be uninterrupted which I can work on my online course
Q6	I value flexibility which I can work on my online course
Q7	Using the online platform keeps me active in learning
Q8	Using the online platform reduces the amount of time I spend doing something else in the class
Q9	Using the online platform can keep my intention focused on learning
Q10	I feel exciting when learning by using this online platform
Q11	I feel that learning via online platform is helpful for my academic future
Q12	I enjoy the fun of learning interactively via online platform
Q13	I feel that engaging with and learning interactively via online platform is fun
Q14	I feel that learning via online platform attract me to participate in the learning system
Q15	I think the experience learning via online platform will be helpful for studying into the next level
Q16	I feel that learning via online platform is interesting
Q17	I think it will be beneficial to my academic future If I achieve good result when I learn using online platform
Q18	I will use online platform on a regular basis in the future
Q19	I feel learning via online platform can challenge me to learn extensively
Q20	My learning experience in this online platform has make me able to analyze and organize information for my subject
Q21	In general, at this moment I perceive that, testing and assessment in online platform have a positive effect on my learning
Q22	As a learner I see myself as highly competent in using online platform technologies
Q23	My learning experience in this online platform has made me feel confident as a learner
Q24	In general, at this moment I perceive that, testing and assessment in online platform add value to the time I have spent on the work done
Q25	In general, at this moment I perceive that, testing and assessment in online platform are valuable instances of learning in their own right
Q26	My learning experience in this online platform has make me able to evaluate issues critically for my subject
Q27	In general, at this moment I perceive that, testing and assessment in online platform me to continue learning
Q28	My learning experience in this online platform has make me able to apply theoretical knowledge to practice
Q29	My learning experience in this online platform has motivated me to learn more
Q30	In general, at this moment I perceive that, testing and assessment in online platform help me to navigate my own learning process
Q31	In general, at this moment I perceive that, testing and assessment in online platform are geared towards the retention of my competencies in the longer period
Q32	My learning experience in this online platform has fully engaged me as a learner
Q33	In general, at this moment I perceive that, testing and assessment in online platform prepare me well for future learning activities
Q34	I will make use the online platform regularly in the forthcoming time
Q35	In general, at this moment I perceive that, testing and assessment in online platform give me confidence to continue learning
Q36	I intend to make use the functions of online platform for providing assistance to my academic activities
Q37	My learning experience in this online platform has make me able to develop new ideas
Q38	I will give out my recommendation to others to use the online platform