A system for equity: enhancing STEM education during a pandemic

Abstract

Purpose

The purpose of this paper is to report the change in students' STEM perceptions in two different informal learning environments: an online STEM camp and a face-to-face (FTF) STEM camp.

Design/methodology/approach

For this quasi-experimental study, 26 students participated in an online STEM summer camp and another 26 students participated in the FTF STEM camp. Students from each group took the same pre- and post-STEM Semantics Survey documenting their perceptions of the individual STEM fields and of STEM careers. Wilcoxon Signed-Rank tests, Mann–Whitney U tests and corresponding effect sizes were used to compare the pre- and post-scores within and between the camps.

Findings

Results indicate that both camps produce similar outcomes regarding STEM field and career perceptions. However, analysis of all statistical values indicates that the online STEM camp can produce a larger positive influence on STEM field perceptions and the FTF camp can produce a larger positive influence on STEM career perceptions.

Research limitations/implications

This suggests that STEM camps, both online and in-person, can improve students' perceptions of the STEM fields and of STEM careers. Implications from this study indicate that modifications of informal learning environments should be based on the type of learning environment.

Originality/value

This manuscript discusses the development and impact of an online STEM camp to accommodate for the sudden onset of the COVID-19 pandemic and the inability to hold an in-person STEM camp. These results may influence the curriculum and organization of future online and FTF STEM camps.

Citation

Baucum, M.N. and Capraro, R.M. (2021), "A system for equity: enhancing STEM education during a pandemic", Journal of Research in Innovative Teaching & Learning, Vol. 14 No. 3, pp. 365-377. https://doi.org/10.1108/JRIT-12-2020-0087

The COVID-19 pandemic precipitated new complications for both formal and informal educational practices. Chief among these were the challenges and affordances students and instructors face when all learning must be remote, technically delivered and Internet dependent, a situation for which no precedents exist. In the spring of 2020, educators were forced to revert all educational experience to a virtual format within a matter of days. The immediate need for a different kind of instructional environment left schools desperately searching for the best learning management and communication delivery systems. This abrupt disturbance in K–12 education also extended to summer activities and forced informal learning experiences to amend curriculum and delivery of programming. For this paper, we examined the struggles and accomplishments of a science, technology, engineering and mathematics (STEM) summer camp redesigned to be delivered through two separate, yet synchronous formats: a traditional face-to-face (FTF) STEM camp and an online STEM camp. We also compared the impact the online STEM camp had on STEM field and career perceptions compared to the FTF STEM camp. Although STEM camps and online informal learning have been studied individually, the influence of online STEM camps has not yet been published. Therefore, the purpose of this paper is to report the change in students' STEM perceptions in two different informal learning environments: an online STEM camp and an FTF STEM camp.

Background

Informal learning settings provide students with an alternative educational environment that allows for academic growth and further development of interest in STEM content. Additionally, informal learning environments typically provide high-impact learning experiences delivered in less stressful and more collegially centric settings without the constraints of state or national learning standards or other accountability requirements common in formal learning environments. Translocating an informal learning environment into the home and remotely interacting with students creates a unique set of affordances and challenges.

Online informal learning environments

Online informal learning environments provide a flexible atmosphere for students to increase their knowledge and interest in academic content. Specifically speaking of STEM education, online informal learning affords students the opportunity to work through the scientific process by making observations, recording data, developing research questions, conducting background research, proposing hypotheses and drawing conclusions (Marty et al., 2013). Additionally, the relaxed learning contexts provided by both FTF and online informal learning encourage students to take charge of their own learning (Hall, 2009; Meyers et al., 2013). In the case of online learning settings that occur in students' individual homes, independent learning is encouraged and strengthened because students have full control over the materials and have to make informed decisions on when to use them (Hall, 2009). Online learning, when synchronous, also provides a group atmosphere and constant access to support from the teacher, albeit remotely. Furthermore, students like using their skills in a personalized environment that is flexible to their learning speed and that can adjust to their personal learning interests (Chakowa, 2018). Although online learning may present its own set of challenges, the independent environments created by virtual learning may encourage students to take ownership of their own learning, which could then heighten their interest in the learning content.

Beyond the academic developments of an informal online learning environment are its social aspects. Although FTF informal learning environments create a more flexible atmosphere for social interactions, hosting informal learning experiences online merges people, content and technology. This dynamic complements settings already encouraged by, but not necessarily implemented in, most schools and workplaces (Meyers et al., 2013). In general, students find value in the interactions between people and content that happen in both FTF and online informal learning environments (Hall, 2009; Tan, 2013). However, students are able to share their experiences and engage with their instructors in a more relaxed way in online informal learning environments because the students see the instructor as more of a participant than in a traditional FTF setting (Sackey et al., 2015). These characteristics mean that online informal learning creates a more accessible and friendly educational environment than FTF settings, which encourages students to proactively expand their academic knowledge and interests.

STEM camps

STEM camps are an excellent example of an informal learning environment that fosters an affinity to STEM in general. First, participation in STEM camps increases students' STEM content knowledge (Hirsch et al., 2017), interests toward the individual STEM fields (Mohr-Schroeder et al., 2014) and perceptions of creativity and problem solving in STEM fields (Bicer et al., 2017). Second, STEM camps provide students with an interaction of STEM materials, professionals and hands-on experiences that has been found to influence their perceptions of STEM fields and careers (Christensen and Knezek, 2017; Kwon et al., 2019; Mohr-Schroeder et al., 2014; Vela et al., 2020). For example, camp tasks where students solved problems that helped people they cared about had a significant impact on their interest in STEM as a career choice (Maiorca et al., 2020). Additionally, students showed an increased interest in STEM careers after participating in a STEM camp because through hands-on, inquiry-based activities, they had a better understanding of what STEM professionals actually do (Asiabanpour et al., 2010; Hirsh et al., 2017). The increased student interest in STEM that results from participation in a STEM camp is further enhanced when STEM professionals provide information and insight into their jobs and daily activities (Maiorca et al., 2020; Vela et al., 2020). Students' perceptions, attitudes and interest in STEM careers may improve as a result of participating in a STEM camp because of their exposure to the opportunities and activities associated with STEM careers.

Theoretical framework

The basis for this study is situated on the individual as an engaged participant within the broader context of the learning environment. We ascribe to a learning model based on the autonomous engaged learner within a quasi-group setting. This theory situates the learner as both the focal point of learning and the centroid for intrinsic, self-directed and collaborative learning in an online learning environment. An online, informal setting maximizes self-directed learning from a student as they experience distanced interpersonal interactions along with the ability to negotiate and build content expertise. The keystone feature is the student's intrinsic motivation to persist in tasks and to engage with peers and teachers in a virtual learning space while each person is personally and privately engaged with the construction of their product or learning outcome.

Two major social cognitive theories were foundational for this study. Social cognitive theory indicates an interaction between a student's behavior and their environment, specifically in the areas of self-efficacy, outcome expectations and goals (Bandura, 1986, 2001). Social cognitive career theory (SCCT) extends this idea and argues that a student's interest in a career is influenced by a number of factors, including predispositions, background, gender, race/ethnicity, learning experiences, self-efficacy and outcome expectations (Lent et al., 1994). The interaction between social cognitive theory and SCCT is the nexus of setting and the personality characteristics of the individual. In this study, where the learning content is identical, the instructors are similar and the duration is equal, the focus is on the individual and the way they engage with differing environments. Such a framework allows researchers to understand how these theories may yield interesting findings and important implications for online informal STEM learning.

Method

This quasi-experimental, nonequivalent-group comparative study included two groups, one that participated in a STEM camp virtually (N = 26) and one that participated in a STEM camp that was FTF (N = 26). The following research questions framed the study:

1. How does participation in a one-week online STEM camp influence students' perceptions of the individual STEM fields compared to a one-week face-to-face camp?

2. How do perceptions of STEM careers vary between students who participated in a face-to-face camp compared to those who participated in an online camp?

Setting and participants

All participants were enrolled in a one-week summer camp designed to engage students with and promote the STEM fields and encourage interest in STEM careers through project-based learning and interactions with STEM professionals. Participants were self-selected and resided in multiple parts of the world. Importantly, participants attended either an FTF version of the camp or a synchronous online version. The FTF camp occurred at a large university located in the southeastern United States. Students engaged in STEM classes such as engineering design, coding, physics and standardized test prep. Teachers instructed classes using problem-based learning, in which the lesson were focused around an end product and the students were responsible for working together to build that end product or solve the predetermined problem. In the afternoon, participants toured STEM-related laboratories located on the campus of the university and listened to panel discussions promoting STEM career opportunities. The participants also attended a chemistry show and a physics show in person presented by university faculty who worked to ensure visual and visceral impact. Evening social activities, such as game nights and visits to local attractions, were offered, and students stayed overnight in campus housing.

The virtual camp took place entirely online through Zoom. Any required materials were either mailed to students ahead of time or the students were told they were responsible for collecting them around their homes. Each day, students participated in classes such as coding (which utilized a free, online program), engineering design, physics and standardized test prep. These took place in the morning. The afternoon session of the online camp consisted of an additional STEM activity, such as building a stethoscope for animals using funnels and plastic tubing, viewing a video of the aforementioned chemistry show or attending virtual panel discussions with STEM professionals. Online participants also had the opportunity each day to participate in a one-hour evening social activity, such as virtual escape rooms and online games such as Jeopardy. All of these activities were designed to enhance students' STEM career interests and perceptions through an online platform.

Both camps were organized by the same department, and the online camp was designed to mimic the FTF camp as much as possible, including encouraging student-to-student interactions and group work in online breakout rooms. Because there were some limitations to what materials students attending the online camp could access, the online camp activities may not have been identical to the ones taught in the FTF camp. However, the major ideas, products the students were expected to build and the collaboration among classmates were similar. All lessons delivered in both the online and FTF camps were designed by the instructors as STEM problem-based learning activities. Additionally, the afternoon panel sessions were of the same duration, although the online camp did not include the laboratory tours. The major difference between the two camps, however, was the delivery of instruction, with the virtual camp being delivered entirely through an online platform and the FTF camp being entirely in person.

Twenty-six students were enrolled in the online version of the camp, and 30 students participated in the FTF version of the camp. Participants ranged from 9th to 12th graders. Propensity score matching based on gender and pre-camp scores was used to create similar groups with equal sample sizes at the start of the current study (N = 26 for both groups) in each of the perception areas (Rosenbaum and Rubin, 1983). There were 12 female students and 14 male students who attended the online camp. For the FTF camp, the number of female students and male students varied for each of the perception fields. The majority of the students were from the same state as where the camp took place. Table 1 presents the demographics for the participants for both camps.

Instrument

All students completed the STEM Semantics Survey (Christensen et al., 2014), which measures students' dispositions toward the individual STEM fields of science, mathematics and engineering as well as STEM careers, before the camp began and after the camp concluded. Each field had five questions with adjective pairs and was scored using a Likert-type scale (1–7). For two of the five questions in each section, a score of 1 represented a negative adjective and a score of 7 represented a positive adjective. Therefore, three of the questions in each section were reverse coded for analysis. Each factor had a range of 7–35. A modified version of the instrument can be found in Table 2. The adjective pairs were consistent among all four factors.

Data analysis

Data were analyzed using Microsoft Excel 16 and STATA 16 (see Barroso et al., 2019, 2020 for dataset information). First, Cronbach's alpha for each subscale was calculated to determine the reliability for the data in hand. Then, nearest-neighbor propensity score matching with replacement was conducted on the two data sets using the total score of each category. Only data points that were matched were used in further analysis (Rosenbaum and Rubin, 1983). Means, medians and standard deviations for both groups' pre- and posttest scores are reported. The data were not univariate normal, so nonparametric tests were used for data analysis. A Wilcoxon signed-rank test for each camp and each of the factors was used for within-group analysis (Kruskal and Wallis, 1952). A Mann–Whitney U test was conducted to estimate the differences for the between-group analysis (Grissom and Kim, 2012; Mann and Whitney, 1947; Nachar, 2008). The a priori alpha level was set at α = 0.05. To perform the aforementioned tests, rank correlations were determined. The rank-biserial r is reported for each of the fields for the between-group analysis, and the matched-pairs rank-biserial r is reported for the within-group analysis.

For both the within-group and between-group analyses, researchers used the simple distance formula suggested by Kerby (2014), which considers the simple difference between the proportion of favorable outcomes and unfavorable outcomes for the online camp. The nonparametric effect size method allows researchers to analyze the effect size in terms of the number of favorable outcomes rather than by an average increase in scores (Kerby, 2014). For this study, the online camp functioned as the experimental group, so the between-group effect sizes are calculated in terms of the online scores relative to the FTF scores. The rank-biserial r and matched-pairs rank-biserial r were then squared to allow comparison. The results were interpreted through standards that honor both p values and measures of effect within nonparametric analyses. Finally, a post-hoc power analysis was conducted using g*Power 3.1. Results of all analyses were discussed together to draw a more accurate conclusion about the effects of an online and FTF STEM camp on students' STEM perceptions.

Results

Reliability is a psychometric property of scores where higher values are considered better. A cutoff of 0.60 functions as a lower recommended limit (George and Mallery, 2003). Reliability is important to consider because it has a direct attenuation of the obtained effect size (Thompson, 1994, 2002). Cronbach's alpha reliability, a measure of internal consistency, was calculated for each subscale (see Table 3). The alpha values were satisfactory for all subscales except for the science pretest score (Bland and Altman, 1997). The alpha level for the science pretest along with the measures of center and spread indicate that students came into camp with similar perceptions of science. The change in alpha level for the science posttest occurred because the scores were spread out more from the mean, indicating the effects caused scores to both increase and decrease from the pretest. Because the posttest reliability was within the recommended limit, there is no need to correct the obtained results.

Means, medians and standard deviations for pre- and posttests were calculated before conducting statistical tests. Descriptive statistics of group by subscale are contained in Table 4. Because propensity score matching was used to make groups similar on all measured background variables, the starting mean scores were evidence that the propensity score matching was suitable. Because data were non-normal, median scores were the descriptive of interest. The median pretest scores were higher in all four categories for the FTF camp than for the online camp. The median posttest scores favored the online camp for science, mathematics and career perceptions and favored the FTF camp for engineering perceptions.

Research question 1

To answer research question 1 (How does participation in a one-week online STEM camp influence students' perceptions of the individual STEM fields compared to a one-week face-to-face camp?), researchers used within-group and between-group analyses. To determine how each camp influenced students' perceptions of the STEM fields, researchers computed within-group p values and effect sizes (matched-pairs rank-biserial r) to measure the differences in the pre and posttest scores of each camp type (see Table 5). The Wilcoxon signed-rank test (i.e., for the within-group analysis) did not reveal statistically significant effects for the three subscales (science, mathematics and engineering) for both camps. This indicates that neither the online camp nor the FTF camp showed statistically significant improvements of the students' perceptions of the STEM fields. However, the within-group effect sizes of the FTF camp's scores indicated a negative effect on students' science, mathematics and engineering perceptions (r2 = 0.11, r2 = 0.14, r2 = 0.05, respectively). This means that the proportion of unfavorable outcomes (higher scores on the pretest) exceeded the proportion of favorable outcomes (higher scores on the posttest). The online camp produced no effect on science perceptions (r² < 0.001) but favored the posttest for mathematics and engineering perceptions (r2 = 0.03, r2 = 0.03, respectively). This indicates that the ratio of favorable outcomes (higher scores on the posttest) was higher than the ratio of unfavorable outcomes (higher scores on the pretest). From the within-group analysis, we can conclude that although there were no statistically significant differences between the pretests and the posttests, the online camp had a larger positive influence on students' perceptions of the STEM fields.

In order to compare the results of the online and FTF camps, a between-group analysis was conducted to measure the posttest score differences between the two camp types. The Mann–Whitney U test results did not indicate statistically significant results for all three subscales (see Table 6), indicating that there was not a noticeable difference in posttest scores for the perceptions of the STEM fields between the two camps. Posttest score analysis for the between-group effect sizes (rank-biserial r) slightly favored the FTF camp for science, mathematics and engineering perceptions (r2 = 0.001, r2 = 0.04 and r2 = 0.10, respectively). This means that the proportion of scores that favored the FTF camp was higher than the proportion of scores that favored the online camp. Although the between-group analysis favored the FTF camp for STEM field perceptions, the results of the within-group and between-group analyses considered together lead us to a somewhat ambiguous conclusion concerning how each camp influenced students' perceptions of the individual STEM fields.

Research question 2

To answer research question 2 (How do perceptions of STEM careers vary between students who participated in a face-to-face camp compared to those who participated in an online camp?), researchers again used within-group and between-group analyses with p values and effect sizes (rank-biserial r and matched-pairs rank-biserial r). The Wilcoxon signed-rank test did not indicate a statistically significant result for students' STEM career perceptions for the online camp (p = 0.461); however, it did indicate a statistically significant result for the FTF camp (p = 0.030). For the FTF camp, results favored the posttest for students' STEM career perceptions (r2 = 0.27). Conversely, results favored the pretest for the online camp for STEM career perceptions (r2 = 0.03).

The Mann–Whitney U test for between-group comparison revealed that the online camp produced a higher proportion of tests that favored posttest scores than the FTF camp for career perceptions (r2 = 0.14). However, it should be noted that the online camp started with higher median scores for career perceptions than the FTF camp. Similar to our conclusion concerning STEM field perceptions, the results of the effect of the different STEM camps on students' STEM career perceptions are not conclusive, but the FTF camp is favored.

A post-hoc power analysis was conducted after the researchers noticed the statistically nonsignificant results (see Table 7). The low values are likely due to small sample sizes and indicate the inflation of a Type II Error, as 11 of the 12 statistical tests produced statistically nonsignificant results.

The measures of centers and effect sizes, when analyzed individually, provide us with an ambiguous conclusion. To view the statistical measures more holistically, we concurrently evaluated the change in score means, change in score medians, the within-group r2 and the between-group r2 for each category by camp. The visual analysis in Table 8 depicts this parallel, with a shaded box indicating a value favoring that camp. This analysis indicates that both online and FTF STEM camps may produce similar results in regard to the STEM fields and STEM career perceptions when viewing the results more holistically.

Discussion

The focus for this study was to compare the potential change in students' perceptions of the STEM fields and STEM careers when attending two different camps. Prior research has indicated that FTF STEM camps can improve perceptions of the STEM fields and STEM careers (Christensen and Knezek, 2017; Kwon et al., 2019; Mohr-Schroeder et al., 2014; Vela et al., 2020). Additionally, our framework centralizing the learner in an online, quasi-group setting coupled with social cognitive theory (Bandura, 1986, 2001) and social cognitive career theory (Lent et al., 1994) provided a theoretical base for the improvement of STEM field and career perceptions during a one-week STEM camp, both online and in-person.

Results from the between-group and within-group analyses lead to an ambiguous conclusion concerning how each camp affected student' STEM perceptions. Because the p values were statistically nonsignificant for all factors besides the FTF camp pretest/posttest comparison for STEM career perceptions, we cannot conclude that either camp produced statistically significant effects on students' STEM field perceptions. The high β values (due to the low power) lead us to believe that there is a large likelihood of a Type II Error, indicating that although our study does not conclude statistical significance for improved perceptions of the STEM fields and for STEM careers for the online camp, there may exist a statistical significance in the population. However, analysis of results should extend past the reporting of p values in order to examine practical significance (Capraro, 2004; Coe, 2002); thus, effect sizes were calculated.

The results of the within-group analysis revealed that there may not be a large noticeable difference of effect on STEM field perceptions for either of the camps, but the online camp may be slightly favored. As we know from prior work, informal FTF camps tend to have a positive impact on student interest in the individual STEM fields (Mohr-Schroeder et al., 2014) and students' perceptions of the STEM fields (Vela et al., 2020). Although the work of Vela et al. (2020) also reported students' perceptions of the STEM fields were not statistically significant from pretest to posttest, there was a positive effect size reported for all four categories. However, our effect sizes showed that the FTF and online camps impacted students' perceptions differently.

Our within-group results indicated a negative effect on STEM field perceptions for all three categories for the FTF camp but only for science for the online camp. The differences between our FTF camp results and the results of Vela et al. (2020) could be attributed to small sample size or the choice of the effect size measure rather than STEM content or activities. Cohen's d uses the means of the two groups, whereas the matched-pairs rank-biserial r is calculated using the proportion of favorable and unfavorable outcomes. Because the two effect sizes use different measures, it is not surprising that our effect size results differ from those found in Vela et al. (2020). The difference between the results for the online camp and those found by Vela et al. (2020) could be attributed to the lack of hands-on, collaborative learning that the online environment could offer for science classes. The positive effect sizes for the other two categories indicate that the courses improved students' perceptions of the mathematics and engineering fields. Future STEM camp curriculum for online participants, particularly in science, should reflect the same type of collaboration and hands-on activities that the mathematics and engineering courses offered. In general, both online and FTF STEM camps should continue to offer engaging and hands-on experiences to heighten students' interest in the STEM fields.

The between-group effect sizes of students' STEM field perceptions indicate an almost opposite conclusion. When comparing the camps' posttest scores directly, the FTF camp produced higher scores for science, mathematics and engineering than the online camp. Comparatively, viewing the effect sizes of the within-group analysis, the online camp produced a larger proportion of favorable outcomes vs non-favorable outcomes compared to the FTF camp. This means that, overall, the FTF camp produced higher scores on the posttest, but the online camp produced a greater change in students' scores pre- to posttest. This observation indicates that the online camp may have been more successful in improving perceptions of the STEM fields than the FTF camp.

Although the conclusions about students' perceptions of the individual STEM fields may favor the online camp, there is a different result regarding students' perceptions of STEM careers. Similar to the STEM field perceptions, the within-group and between-group analyses produced conflicting results. The within-group effect size favors the FTF camp, whereas the between-group effect size favors the online camp. This means that the FTF camp had a larger impact on students regarding their STEM career perceptions than the online camp. The difference here could be attributed to the features of the camps. Both camps were similar in design regarding classes, but the FTF camp offered more immersive and more frequent supplemental STEM activities outside of class time. Students that attended the FTF camp participated in STEM laboratory tours and panel sessions with STEM professionals daily, whereas the online camp participants were only able to attend four panel sessions with STEM professionals. Additionally, students in the FTF camp were able to work on their hands-on activities in person with other participants, STEM college students and professors. Although the program tried to mimic as much of this as possible for the online camp, there were limitations to what the students could experience in real time. This may be a reason why there was a statistically significant result for the FTF camp regarding STEM career perceptions but not for the online camp.

Although the Mann–Whitney results favored the online camp for STEM career perceptions, the median pretest scores for the FTF camp may have produced a larger ceiling effect than the median pretest scores for the online camp, indicating that the between-group analysis results may not be the best indicator of improvement of students' career perceptions. The magnitude and direction of the within-group effect size for STEM career perceptions for the online camp compared to the results found by Vela et al. (2020) and from the FTF camp from this study for STEM career perceptions indicate that an FTF environment could be more conducive to improving students' perceptions of STEM careers. This notion, that the learning environment has a significant effect on a student's interest in a particular career, is also supported by social cognitive career theory (Lent et al., 1994). The findings from this study are important for theory development, as the use of online instruction is wide-spread and continually developing.

Educational significance

The debate concerning FTF and virtual learning may continue to rage, especially as the pandemic and other factors continue to be present. However, from this study there was no clear-cut evidence that one format was preferable or delivered greater beneficial effects across all dimensions. There are potential lurking variables that could differentiate the outcomes for the camps. For example, it is not clear that the groups themselves might be completely different. For instance, students who participate in a virtual camp might have never participated in a FTF camp because of individual differences, physical impairment or social or behavioral conditions. Although all the students in the current study had originally expressed interest in a FTF camp, there is no guarantee that they would have attended.

The results of this study inform the field about the viability of providing informal STEM learning activities through online distance-learning platforms. The efficacy of online versus FTF education is currently highly debated, and the results of this study can aid in determining which environment can produce better results in terms of students' perceptions. In the case of STEM camps, online and FTF informal learning programs can produce nearly identical changes in students' perceptions of science, mathematics, engineering and STEM careers, although there may be a slight advantage for the online camp to positively influence STEM field perceptions and for the FTF STEM camp to positively influence STEM career perceptions. Research on STEM perceptions should be expanded to include larger sample sizes and to investigate how the perceptions of the STEM fields can influence a student's perception of a STEM career through online and FTF STEM camps. Additionally, instead of viewing online camps as “less” or “more” effective than FTF camps, further research should be conducted to determine how online camps can be restructured to improve students' perceptions of STEM careers.

Table 1

Demographic information of camp participants

Characteristic	Online camp		FTF science match		FTF math match		FTF engineering match		FTF career match
	n	%	n	%	n	%	n	%	n	%
Gender
Female	12	46	17	65	12	46	18	69	11	42
Male	14	54	9	35	14	54	8	31	15	58
Grade
6	0	0	0	0	4	15	0	0	0	0
7	4	15	17	65	9	35	5	83	12	46
8	2	8	4	15	6	23	2	8	7	27
9	1	4	0	0	5	19	9	35	2	8
10	7	27	0	0	1	4	1	4	0	0
11	8	31	2	8	0	0	0	0	2	8
12	4	15	3	12	1	4	9	35	3	12
Race/Ethnicity
Asian	4	15	0	0	4	15	4	15	0	0
Black or African American	2	8	0	0	0	0	0	0	0	0
Hispanic or Latino	8	31	20	77	1	4	10	38	5	19
White	12	46	6	23	17	65	12	46	21	81
Other	0	0	0	0	4	15	0	0	0	0
Residency in Texas	25	96	26	100	26	100	26	100	24	92
Outside of Texas	1	4	0	0	0	0	0	0	2	8

Table 2

Engineering scale from survey

To me, engineering is…
Appealing	1	2	3	4	5	6	7	Unappealing
Fascinating	1	2	3	4	5	6	7	Mundane
Means nothing	1	2	3	4	5	6	7	Means a lot
Exciting	1	2	3	4	5	6	7	Unexciting
Boring	1	2	3	4	5	6	7	Interesting

Table 3

Cronbach's α

	Pre	Post
Science	0.458	0.731
Mathematics	0.895	0.932
Engineering	0.906	0.924
Career	0.761	0.955

Table 4

Descriptive statistics

	Online						FTF
	Pre			Post			Pre			Post
	x‾	Median	SD	x‾	Median	SD	x‾	Median	SD	x‾	Median	SD
Science	31.3	31.5	2.9	29.5	32.0	7.8	31.9	35.0	3.8	30.5	29.0	2.2
Mathematics	27.4	26.0	6.3	28.5	31.0	6.8	27.5	28.0	6.8	27.2	28.0	7.6
Engineering	27.7	28.5	6.9	28.9	30.0	6.1	26.6	32.0	8.4	25.3	32.0	11.4
Career	31.3	32.5	4.4	30.2	35.0	8.2	32.3	33.5	3.0	33.8	34.5	2.0

Table 5

Wilcoxon sign-rank test by category for within-group analysis

	p value	z	r	r2
Science
Online	0.980	0.03	−0.01	<0.00
FTF	0.131	1.51	−0.33	0.11
Mathematics
Online	0.476	−0.73	0.17	0.03
FTF	0.162	1.40	−0.37	0.14
Engineering
Online	0.502	−0.67	0.17	0.03
FTF	0.363	0.91	−0.22	0.05
Career
Online	0.461	−0.74	−0.74	0.33
FTF	0.030	−2.17	0.52	0.27

Table 6

Mann–Whitney U test by category for between-group analysis

	p value	U	r	r2
Science	0.075	243.0	−0.09	0.08
Mathematics	0.669	315.0	−0.19	0.04
Engineering	0.689	316.5	−0.32	0.10
Career	0.434	298.5	0.38	0.14

Table 7

Power analysis results

	Within-subject online	Within-subject FTF	Between-subject
Science	0.37	0.36	0.10
Mathematics	0.25	0.37	0.10
Engineering	0.34	0.25	0.28
Career	0.19	0.98	0.58

Table 8

Comparison of online and FTF statistic