Getting along with others as an educational goal: An implementation study of Sanford Harmony

Abstract

Purpose

The purpose of this paper is to examine how participants in diverse schools newly implement the Sanford Harmony social and emotional learning (SEL) program and perceive its benefits for students and overall school climate.

Design/methodology/approach

The current study employed a convergent parallel mixed-methods design with a sample of five elementary schools in the western USA. Measures included classroom observations, administrator interviews, teacher interviews and focus groups, student focus groups, and a teacher questionnaire.

Findings

Findings indicated expected variation in implementation across schools, although all participants reacted favorably to the program and, importantly, would recommend the program to others. Administrators, teachers and students all saw the value of the program, particularly in terms of student relationship building and improved school climate. Implementation challenges experienced by schools were consistent with research on diffusion of innovations.

Practical implications

The present study demonstrates the importance of effective professional development, continued support, collective decision making and intentional integration of the SEL program throughout a school to support robust implementation and ultimately achieve intended outcomes.

Originality/value

Researchers have yet to examine in-depth implementation of the Sanford Harmony program and how best to support scale-up and more intentional implementation in schools. As implementation fidelity is a key component of a program achieving intended outcomes, the findings from the present study contribute to the knowledge base of supporting SEL program implementation.

Citation

Morrison, J.R., Reilly, J.M. and Ross, S.M. (2019), "Getting along with others as an educational goal: An implementation study of Sanford Harmony", Journal of Research in Innovative Teaching & Learning, Vol. 12 No. 1, pp. 16-34. https://doi.org/10.1108/JRIT-03-2019-0042

In recent years, social and emotional learning (SEL) has become a topic of increased focus for K-12 schools across the USA (Belfield et al., 2015; Brackett and Rivers, 2014; McGraw-Hill and Morning Consult, 2018; Miller et al., 2017; Zins and Elias, 2007). As defined by the Collaborative for Academic, Social, and Emotional Learning (CASEL), SEL teaching seeks to develop students’ personal skills with regard to self-awareness and management, social awareness, relationship building and effective decision making (CASEL, 2018). Not surprisingly, research has found links between the quality of students’ social-emotional development and a variety of school-oriented outcomes including behavior in school (Durlak et al., 2011; Sklad et al., 2012), engagement in school (Valiente et al., 2007; Yang et al., 2018) and academic achievement (Corcoran et al., 2018; Durlak et al., 2011). Given these findings, research that explores how schools can best incorporate SEL-focused teaching as part of their instructional programming is becoming of increased importance. The purpose of the present study is to examine implementation processes and activities with regard to one such approach to developing social-emotional skills in elementary school students: The Sanford Harmony program.

The importance of SEL

Research clearly suggests the importance of children’s social-emotional development, both as a means for improving learning outcomes and also as an important outcome in and of itself (Taylor et al., 2017; Wang et al., 1997; Zins and Elias, 2007). Children’s social-emotional aptitude has been found to be a predictor of the extent in which they engage in prosocial behavior, empathetic behavior and effective conflict resolution (Zins and Elias, 2007; Zins et al., 2003). Social-emotional abilities have also been found to correlate with students’ capacity to self-regulate their behavior (Brackett and Rivers, 2014) and form positive peer relationships (Argyle and Lu, 1990). Both of these skills have been found to drive positive outcomes of their own. Improved regulation of behavior has been linked with enhanced concentration in school (Lane et al., 2003; McClelland et al., 2007) while positive peer relationships have been linked to improved engagement with school (Wentzel et al., 2012) and are thought to improve the effectiveness of collaborative types of learning activities (DeLay et al., 2016; Ladd et al., 2012; Wentzel and Watkins, 2002). Research has also found that students’ overall social-emotional abilities are positively linked with other factors including students’ self-efficacy, educational aspirations, respect for teachers, ability to cope with school stressors, attendance and classroom participation (Zins and Elias, 2007). As summarized by Zins and Elias (2007):

Research shows that SEL has positive effects on academic performance, benefits physical health, improves citizenship, is demanded by employers, is essential for lifelong success, and reduces the risk of maladjustment, failed relationships, interpersonal violence, substance abuse, and unhappiness (p. 3).

Adding to these findings, research has found that students with poor social-emotional competence often struggle in school (McEvoy and Welker, 2000). Specifically, children with poor self-regulation abilities and maladaptive social skills have been found to do worse academically (Ladd et al., 1997; Webster-Stratton and Reid, 2004) and are more likely to struggle forming positive peer relationships (Argyle and Lu, 1990; Shores and Wehby, 1999; Webster-Stratton and Reid, 2004). Further, these same children often are less engaged with school, fail to complete assignments and exhibit less on-task behavior during lessons (Patterson et al., 1990). Taken in combination, these findings clearly demonstrate that SEL can have an important influence on a host of factors that impact how students experience and engage with school.

Research on curricula for SEL

As schools place increasingly more focus on incorporating social-emotional based teaching into their instructional programming (Belfield et al., 2015; Brackett and Rivers, 2014; McGraw-Hill and Morning Consult, 2018; Miller et al., 2017; Zins and Elias, 2007), it is not surprising that growth has also occurred in the number of SEL curricula available for schools to adopt (Miller et al., 2017). Considering the documented influence that improved social-emotional functioning can have on a variety of student outcomes (Taylor et al., 2017; Wang et al., 1997; Zins and Elias, 2007), this trend is understandable. What research has yet to consistently show, however, is how and to what extent these programs function to enhance student achievement. Systemic reviews of the research literature predominantly include studies focusing on behavioral outcomes and student attitudes (Mahoney et al., 2018). While the research synthesized suggests that SEL curricula can have a significant positive influence on student achievement, these analyses have been consistently based on only a small sample of studies (e.g. Durlak et al., 2011; Taylor et al., 2017; Sklad et al., 2012; Wigelsworth et al., 2016). For instance, in Taylor et al.’s. (2017) meta-analysis, which spanned a search period of over 30 years, only 8 of the 82 included studies examined student achievement. In Durlak et al.’s (2011) synthesis, which spanned over 50 years of research and over 210 studies, only 35 examined achievement. In each case, the vast majority of evaluations were conducted during or prior to the early 2000s. Given the noticeable expansion in the number of SEL curricula that have entered the education marketplace in recent years (Miller et al., 2017) along with the growing emphasis on SEL teaching in schools (Belfield et al., 2015; Brackett and Rivers, 2014; McGraw-Hill and Morning Consult, 2018; Miller et al., 2017; Zins and Elias, 2007), there remains much uncertainty about whether student achievement gains, at least in the short run, are a primary impact of SEL interventions. Rather, SEL program frameworks typically demonstrate improvements in student behavior, attitudes and general social-emotional functioning as outcomes that precede academic achievement gains (Mahoney et al., 2018; Weissberg et al., 2015).

In synthesizing the findings of over 70 studies on universal school-based SEL programs, Sklad et al. (2012) reported the following:

The analyzed interventions had a variety of intended outcomes, but the increase in social skills and decrease in antisocial behavior were most often reported. Although considerable differences in efficacy exist, the analysis demonstrated that overall beneficial effects on all seven major categories of outcomes occurred: social skills, antisocial behavior, substance abuse, positive self-image, academic achievement, mental health, and prosocial behavior (p. 892).

Considering that SEL programs vary greatly in their scope and complexity, the “considerable differences in efficacy” noted above are not surprising (see Corcoran et al., 2018; Sklad et al., 2012). Existing SEL programs range from those with scripted, whole-school approaches which necessitate significant instructional time each week (e.g. Promoting Alternative Thinking Strategies; Schonfeld et al., 2015), to those such as the present focus, Sanford Harmony, which are designed to provide schools with a more flexible SEL framework to integrate into existing school schedules and academic curricula (Harmony, 2019).

In light of these variations, research appears particularly favorable for SEL programs that incorporate certain elements. For example, Durlak et al. (2011) found that SEL programs that incorporate a coordinated sequence of lessons, facilitate active forms of learning, have at least one component focused on social skills and explicitly target specific social-emotional skills, typically improve student outcomes more effectively than other SEL programs. Moreover, programs that can be implemented systemically across a school and facilitate opportunities for SEL-based teaching in different subject areas and grade levels also appear to be particularly promising (Elias et al., 2015; Mart et al., 2015). Given the challenges in today’s schools with finding the instructional time to implement SEL programs (Oberle et al., 2016), one can further hypothesize advantages for programs that are highly adaptable, easy to integrate with instruction in other content areas and least planning intensive for teachers. As examined below, research on SEL programs that have greater implementation challenges are likely to produce noticeably worse effects than programs that are easier to implement (Durlak et al., 2011).

Challenges with implementation

As with many other types of educational interventions, implementation fidelity is a critical component in ensuring that SEL programs function how they are intended. As summarized by Wanless and Domitrovich (2015), “Using evidence-based SEL interventions, however, is not enough to ensure positive outcomes. The success of an intervention on children’s social-emotional competence depends on how it is implemented” (p. 1037). Indeed, implementation quality has been found to be a key predictor of program outcomes across a host of different types of large-scale educational interventions (Shapley et al., 2010) including SEL curricula (Durlak and DuPre, 2008; Durlak et al., 2011; Elias, 2006; Greenberg et al., 2005, 2003). Considering that SEL programs may function best when integrated systemically across an entire school or district (Elias et al., 2015; Mart et al., 2015), careful implementation may be especially important (Durlak et al., 2011; Wanless and Domitrovich, 2015). Not surprisingly, contemporary viewpoints advocate conducting implementation research as a prerequisite to research examining program outcomes (Desimone, 2002).

Despite the documented importance of implementation fidelity, prior research on the successful implementation of educational interventions suggests that complete fidelity is often challenging and relatively rare for schools to attain (Shapley et al., 2010). Schools often selectively implement certain program components more extensively than others (Kurki et al., 2005) and often vary considerably in their level of readiness for implementation (Wanless and Domitrovich, 2015). Implementation of SEL programs in particular is further challenged by the demands of curricula for core academic subjects often taking precedence for schools (Oberle et al., 2016). SEL programs, such as Sanford Harmony, that require less planning and instructional time for teachers, therefore may gain an advantage (Davis, 2014; Roman, 2016).

In light of these findings, it is important that implementation research targets a variety of specific factors demonstrated to be predictors of program fidelity. General readiness factors (e.g. staff openness for change), along with program-specific readiness factors (e.g. staff familiarity with social-emotional development) have been found to be particularly important aspects of program implementation for research to explore (Dymnicki et al., 2014; Wanless and Domitrovich, 2015). To improve program implementation, research has documented the importance of professional development for teachers, follow-up support on professional development and support from school leadership (Shapley et al., 2010). Research that explores the integration of these strategies in program delivery is also of clear importance.

The Sanford Harmony program

The focus of this study, the Sanford Harmony program designed to use highly practical methods for reducing relational conflicts in the classroom while increasing student confidence and relationship skills. Developed by Arizona State University in 2008 and currently managed by National University, the program includes lessons spanning six units and a variety of classroom activities and exercises that engage students in working with different classmates. Units focus on diversity and inclusion, empathy and critical thinking and peer relationships (Harmony, 2019).

Unlike most of the SEL curricula currently available for K-12 schools, Sanford Harmony balances the teaching of student-focused social-emotional skills with the development of a positive socio-environmental context (Miller et al., 2017). The skills component provides instruction to students on encoding social cues, generating prosocial problem-solving strategies and resolving divergent viewpoints (DeLay et al., 2016). To support the development of a positive socio-environmental context, the program incorporates many techniques rooted in Allport’s (1954) Intergroup Contact Theory. This theory posits that by providing opportunities for individuals from diverse groups to interact under positive conditions, a sense of connection can be built between the groups (Allport, 1954; Pettigrew et al., 2011). All of the activities used in Sanford Harmony occur in a group or collaborative setting. The program’s two core routines, Meet Up and Buddy Up, involve students in partnering with a different set of classmates each week to complete an activity or discuss a topic of shared interest. Moreover, the program’s lessons incorporate explicit instruction on developing positive relationships with classmates and often include activities designed to promote the collaboration of all students in the class.

Recent quasi-experimental research conducted on Sanford Harmony with fifth-grade students has produced promising findings. Miller et al. (2017) examined outcomes for roughly 370 fifth-grade students across two cohorts from two schools using the program for one school year in a suburban school district. The researchers collected program data in the form of end-of-year grades, student surveys (the School Liking and Avoidance Questionnaire; Ladd and Price, 1987; Psychological Sense of School Membership Scale; Goodenow, 1993) and teacher surveys (the Child Behavior Scale; Ladd et al., 2009; Ladd and Profilet, 1996). Findings indicated that the program had a significant positive impact on treatment students’ engagement in school, classroom identification and overall feelings of inclusion, as compared with students not receiving the program in demographically matched comparison schools. The authors also noted variation in implementation fidelity and stressed the importance of further examining the factors that influence teacher implementation in future research. Treatment and comparison students, however, had statistically significant differences in demographic characteristics. Treatment students tended to come from families with a greater proportion of annual household income, had a greater proportion of parents with masters’ or doctorate degrees and were more likely to identify as White/Caucasian/European American.

These findings acquire particular importance in view of a recent donation, on June 5, 2018, of $100m by philanthropist and program founder T. Dennie Sanford, to National University to scale-up Sanford Harmony nationally, with a goal of reaching 30m children within five years. Understanding how the program is adopted by schools, supported through resources and training and implemented by teachers is essential for not only informing the rollout but increasing the potential of its impacting students both individually and schoolwide. Traditionally, Sanford Harmony has been made available to interested schools at no cost and with minimal requirements regarding teacher preparation or usage (i.e. scope or fidelity). Not surprisingly, as noted, program implementation has been highly variable and frequently limited to the Meet Up and Buddy Up components, which are easiest to employ (Morrison et al., 2017) . For the present study, the program providers at National University attempted a more formal process that imposed minimal implementation and training expectations specified in a memorandum of understanding (MOU) that participating schools were required to sign. To examine and evaluate this approach, we conducted a replicated mixed-methods case studies in five diverse elementary schools. Research questions were:

RQ1.

How are schools implementing the Sanford Harmony program?

RQ2.

What factors facilitate or hinder program implementation?

RQ3.

What are participants’ reactions to different program components and overall?

Method

Design and participants

The current study employed a convergent parallel mixed-methods design as described by Creswell and Plano Clark (2011). Specifically, qualitative and quantitative data collection activities occurred concurrently and had equal importance within the study. The mixed-methods approach employed allowed for the confirmation and triangulation of data, lending increased validity and support for the conclusions offered (Denzin, 1989).

Participants in this study included principals, teachers and students from five participating elementary schools in a large metropolitan area in the western USA. The sampling approach employed was purposeful and voluntary. These schools implemented the Sanford Harmony program over the course of the 2017–2018 school year and received $25,000 from National University for participation as a research partner. Though the schools did not belong to a single school district, they each were situated within adjacent districts.

In combination, these schools were attended by an ethnically and socio-economically diverse population of roughly 3,000 students during the year of program implementation. The student population in the first school, Clear Creek Elementary (pseudonym), is predominantly Hispanic/Latino (94 percent) with the majority identified as socio-economically disadvantaged and over half (58 percent) as English language learners. The second school, Riverdale Elementary (pseudonym), also had a high proportion of socio-economically disadvantaged students (88 percent), and just under half of the students are English language learners, with a predominant ethnicity of white (60 percent) followed by Hispanic/Latino (25 percent). The third school, Sunnyside Elementary, and fourth school, Thorton Ridge (pseudonyms), are similar in that the predominant student ethnicity is White in both schools, followed by Hispanic/Latino. Both have relatively few (<20 percent) English language learners and less than a quarter of students are socio-economically disadvantaged. The fifth school, Hampton Elementary, has a student population comprised predominantly of 45 percent Hispanic/Latino, followed by White students (27 percent). As with Sunnyside and Thorton Ridge, few (<20 percent) of students are English language learners or socio-economically disadvantaged.

Measures

The measures included both qualitative and quantitative data. We conducted site visits at each of the five schools and conducted classroom observations, along with interviews and focus groups. We administered an online teacher survey and a print-based student survey. We also gathered student behavioral data from the schools. Instruments and data sources are discussed in detail below.

Principal/administrator interviews

A semi-structured interview protocol was developed to solicit principal and administrators’ perceptions of the Harmony program including program history, implementation and perceptions of the program including strengths and recommendations for improvement.

Teacher focus groups

Although the semi-structured focus group protocol for teachers followed the same pattern as those for principals, researchers prompted teachers to discuss their perceptions of the program from their positions as first-hand users. For example, teachers were asked to provide more detailed information about program tools and curriculum, levels of use, student outcomes and students’ reactions to the program.

Student focus groups

A semi-structured focus group protocol was also developed for students in Grades 3–5. Questions invited students to share their experiences related to Harmony and their perceptions of its impact and outcomes for themselves and their classmates.

Teacher questionnaire

The teacher questionnaire was adapted from the questionnaire administered in the Morrison et al.s’ (2017) study and included 36 Likert-type and 5 open-ended items addressing such topics as: preparation and support for using Harmony; implementation practices and components used; perceived impacts on student social-emotional development and learning; perceived impacts on school climate overall; and strengths, weaknesses and recommendations. The Likert-type item scales varied based on topic, for example, where 1=“strongly disagree” or “never used” to 5=“strongly agree” or “extensively used.” A teacher demographic section collected data on teacher background, type of professional development provided by National University and number of years implementing Harmony. Internal consistency reliability for the instrument, as measured through Cronbach’s α, was 0.93.

Student climate instrument

The researchers developed a student climate questionnaire for students in Grades 3–5. In contrast with the other measures utilized as part of this project, this questionnaire did not address matters specific to the Harmony program; rather, it was used as a broad gauge of how students perceive the overall climate in their schools. The questionnaire consisted of 20 Likert-type ratings items where 1=“strongly disagree” and 5=“strongly agree.” Questions solicited students’ perceptions on social relationships (n=5), support for learning (n=5), individuality/diversity (n=4), safety/bullying (n=4) and discipline (n=3). Internal consistency reliability for the instrument, as measured through Cronbach’s α, was 0.88.

Classroom observations

Across the five sites, 27 classrooms were observed implementing aspects of the Sanford Harmony program. The observed classrooms included those from every elementary grade level (K-5th). Researchers were prompted to examine the classroom structure and environment including the context of implementation, teacher and student activities and levels of student engagement. The principal at each location identified the classrooms researchers were to observe predominantly based on the classroom schedule for the day.

Behavioral data

We obtained suspension rates from all five schools. Data were obtained for the pre-program and program year where possible in order to descriptively compare changes over time.

Procedure

The MOU between the participating sites and National University outlined expectations for Harmony program implementation. Expectations included the establishment of schoolwide SEL goals and that each teacher within the school would incorporate lessons from at least one unit of Sanford Harmony and implement Buddy Up and Meet Up daily activities.

Harmony unit themes are divided into five different sections: diversity and inclusion, empathy and critical thinking, communication, problem solving and peer relationships. Each unit consists of specific features including goals and objectives, research and relevance behind the lessons, a home school connection to encourage parental involvement, key concepts and vocabulary and lesson plans for teachers to follow. Younger grades (e.g. pre-K through Grade 2) utilize storybooks to teach lessons. Older grades engage in games, discussions and other activities.

Meet Up and Buddy Up are designated by the program developers as daily activities. During Meet Up, students gather as a whole class or in small groups to establish and monitor expectations for treatment of one another, share ideas and experiences and to problem solve. This activity is designed to take between 10 to 20 min and should occur daily. Buddy Up pairs two classmates for the week, creating opportunities for buddies to get to know one another, partner during learning activities and learn communication skills. Buddy Up activities occur four to five times a week and last between 2 and 45 min depending on the activity. The teacher toolkit contains Quick Connection Cards that are brief discussion prompts and activities to be used during Meet Up and Buddy Up.

Prior to data collection, the program developers conducted training at each of the five schools. The training sought to provide teachers with an overview of the program and how to specifically implement the practices within their individual classrooms. The training lasted approximately 1.5 h.

Principals from each of the five participating sites were interviewed by phone in February of 2018, shortly after implementation began, in order to obtain early impressions of the program. The school climate questionnaire was administered by classroom teachers to their students in Grades 3–5 at the five participating schools in April of 2018. In total, 820 students from third, fourth and fifth grade from the five participating schools completed the survey for a 64.2 percent completion rate. The completion rate was fairly consistent across grades (Grade 3=65.5 percent, Grade 4=59.8 percent, and Grade 5=68.1 percent). Table I summarizes the demographic characteristics of the student sample.

We conducted our interviews, focus groups and classroom observations during site visits at the end of May and early June of 2018. Principals were interviewed at each school and an additional interview, using the same protocol, was conducted with one guidance counselor who supported his/her school’s implementation of Harmony. Each interview lasted approximately 45–60 min. A subset of classroom teachers from each of the five implementing schools participated in focus groups or individual interviews, roughly 45 min in length. At four of the schools, focus groups consisted of five teachers, and at the fifth school, individual interviews were conducted with six teachers. Principals identified teachers to participate in the focus groups and interviews. A subset of Grade 3–5 students from each of the five implementing schools participated in focus groups. Students were randomly selected based on those students whose parents agreed to allow their child to participate. The focus groups lasted generally around 45 min in duration and each consisted of approximately three to six students. In order to conduct the focus groups at times that minimized disruption to each school’s daily schedule, some focus groups were conducted with a mix of grade-level students, while others were conducted with each grade separately.

The teacher questionnaire was administered online using the Qualtrics survey platform during a four-week window at the close of the 2017–2018 school year. In total, 106 teachers across the participating schools were invited to take the questionnaire and 77 successfully completed it in the specified window, for a response rate of 72.6 percent.

Data analysis

Quantitative (e.g. questionnaire) data were analyzed descriptively to examine overall trends and potential differences in responses between schools. Members of the research team transcribed all interviews and focus groups within one day of data collection. Completed transcripts were imported immediately to Nvivo (QRS International) for storage and analysis and two members of the research team coded each transcript. The analytic session began with the dyad team reading through the transcript and discussing themes. The team employed a grounded theory methodology (Glaser and Strauss, 1967) whereby codes were developed based on emerging themes in the data. After all interviews and focus groups were completed, each code was reviewed individually for internal consistency and uniqueness. Then, the coding scheme was organized hierarchically, such that broad themes were comprised of sub-codes, which provided increasingly nuanced information.

Results

In the following sections, we first review findings regarding the implementation context and support for implementation. We next review results pertaining to program implementation, student outcomes, reactions and recommendations. In each section, we aggregated findings from the five participating schools to provide a more complete picture of the participants’ consensus across each category. Where applicable, however, we note nuanced differences that arose between schools.

Implementation

This section begins with results of data collected through principal interviews, teacher focus groups and the teacher questionnaire regarding preparation and support for implementation. Then, we present findings pertaining to implementation of the program at the five sites, including data collected during our site visits.

Principals were asked to describe their experiences and impressions of the professional development provided prior to implementation of Harmony. Feedback related to the in-person training was generally unflattering and noted that it was much too limited. The principal from Thorton Ridge was particularly critical of the in-person training (The training we received was a total flop) and implied that attending the training did more harm to implementation than good. Interestingly, the counselor from this school agreed that the up-front training was difficult for teachers to take on because of the timing but felt that the follow-up training provided to teachers by grade level positively impacted teacher practice and motivation to use the program. Throughout the interviews, principals expressed a need and desire for more professional development in general. As summarized by one principal “It was only 1.5 hours. That’s not enough time to sit down and really look through the materials.”

Ultimately, teachers’ opinions concerning the effectiveness of the professional development largely mirrored those of principals. Less than half of teacher survey respondents (45.7 percent) indicated that they felt they were well prepared to implement the Harmony program effectively. There were some important differences between the five sites. Two thirds of the teachers from Hampton, and about half of the teachers from Clear Creek, Riverdale and Sunnyside felt prepared to implement Harmony. By contrast, however, less than 10 percent of the teachers from Thorton Ridge expressed that they felt prepared, clearly a notable departure from the rest of the sample. This finding may relate to the lack of principal leadership spearheading implementation.

Consistent with teacher survey results, teachers at all five schools noted during focus groups that the initial training offered was insufficient. Part of this perception was due to the timing. Training occurred midway through the school year and as teachers regularly noted, the timing was not ideal since routines were already in place and the content was best-suited for the start of the school year. Despite concerns over timing, teachers conveyed that the initial training was brief and not as in-depth as they would have liked. At Clear Creek, a teacher described this initial training as “a Harmony coach presented for 15–30 minutes” and that all this teacher remembers from the training is being told “be sure to use the box of (Quick Connection) cards.”

Within two schools, Riverdale and Hampton, an experienced teacher initially helped to influence peer use of the program. At Riverdale, one second grade teacher was well-versed in the program having used Harmony the previous year. This teacher spoke to her peers about Harmony and also showed others what the program entailed but did not assume an active role in training. At Hampton, the Transitional Kindergarten (TK) teacher showed the program to the other teachers during a staff meeting half-way through the school year. This teacher explained that “I did initial training” and, as a peer teacher observed, “she [the TK teacher] gave out the boxes to everyone and although buy-in was not universal, she is so bubbly and great that you want to do it [Harmony].”

At two of the schools, teachers indicated that more in-depth training was provided to them at the school’s request. This training, which involved a Harmony staff member visiting the school and working directly with the teachers for at least a half day, resulted in much more positive perceptions of the program and increased teachers’ comfort in implementing program components. As a Thorton Ridge teacher commented, “A lot of teachers wanted additional training, so we had somebody come in and meet with grade-level teams. Everyone was really happy after that” while another teacher commented, “the follow-up training was crucial in making us feel comfortable.”

Despite feeling that initial training was lacking, teachers across schools conveyed during the focus groups that they did feel prepared to implement Harmony. This perceptions stood in some contrast with what was ascertained from the teacher survey, however, where teachers were largely split with regard to whether they were well prepared to implement the program “effectively.” During the focus groups, most often, teachers noted that they were able to read through materials and had a general comfort level with implementing the program due to their experience in teaching. One teacher also indicated that they were informed that the present year was a pilot year and they therefore did not experience pressure to implement the program with high fidelity. This freedom allowed teachers to implement components of the program as they saw fit. Teachers also described further professional development that might be helpful. Specifically, teachers at one school thought it would be beneficial to discuss their experiences as a school to identify strengths and opportunities with implementation. At two other schools, teachers expressed a desire to see more modeling of the program, whether through videos or onsite demonstrations, to better understand nuances of implementation.

As discussed in the upcoming section, Harmony implementation appeared to vary between participating sites. While some schools exhibited enthusiastic implementation of the program from the beginning, others demonstrated various levels of resistance or inertia. Program implementation did appear to improve, however, across most sites over the course of the year, and select program components, mainly the Quick Connection Cards, Meet Up and Buddy Up activities were consistently reported by participants as frequently used and effective.

During the interviews, principals were asked to describe how their schools had implemented Harmony over the past school year. Implementation was described differently at each of the five schools. The principal from Riverdale depicted an enthusiastic implementation and explained, “The need was established before this came to us. My teachers were struggling to address student trauma. We were kind of the perfect situation to bring something like Harmony in.” Principals from Clear Creek and Thorton Ridge described a more challenging context for implementation. At Clear Creek, school activities were described as “chaotic” amid the former principal’s resignation. Teachers at this school struggled to meet the needs of the high-poverty, high ELL population of students, and that SEL was considered to be beyond their present capacity. Similarly, teachers at Thorton Ridge were described as resistant and in the principal’s words, “irritated” up front, causing a delay in implementation. Reflecting on her role, the principal added, “It probably would have gone better if I would have given teachers more specifics early on […] We are in the baby steps phase, we are not there yet. I still have a couple of teachers who are defectors but you’re always going to run into that, anytime you have a new initiative.” The counselor from Thorton Ridge explained that by the end of the school year, Meet Up and Buddy Up were being used in all classrooms but that teachers were not required to implement lessons and storybooks. She explained:

As a counselor, I try to start small and then spread out from there, scale-up. It’s going to take a couple of years to be fully implementing all of the components of it. I’m excited about next year because the kids have already had exposure, so we can build on it.

This type of gradual implementation was the most common theme implied across all schools. Even at Riverdale, where enthusiasm for implementation appeared highest overall, implementation occurred within one grade and then spread to others. Such was the case at Sunnyside and Hampton as well. The principal from Sunnyside explained, “Things will be better next year when starting from the beginning. This year we developed culture in classroom, routines, put procedures in place.”

Relative to the principal interviews, items on the teacher questionnaire focused on the implementation of more specific program elements. By a fairly wide margin, the Quick Connection Cards, Meet Up and Buddy Up components were used by teachers most frequently, as roughly 90 percent of teachers reported at least occasional use of each of these three components. Similarly, teachers in all focus groups referenced using Meet Up and teachers in all but one focus group noted the use of Buddy Up and the Quick Connection Cards. This finding also was buttressed by students’ responses during the focus groups. In response to how teachers help students to get to know each other better, students across all five of the schools most frequently shared examples of Meet Up activities, and students across four of the schools shared examples of Buddy Up activities. In scenarios where participants reported never using certain components, overwhelmingly, the most frequently cited reason was lack of time.

In addition to reporting the frequency in which they used certain Harmony components, survey respondents also reported their perceptions of each component’s effectiveness. Not surprisingly, those components used most frequently by participants were also those perceived to be the most effective. Over 85 percent of participants reported that they felt the Quick Connection Cards, Meet Up and Buddy Up components were each effective tools. About three quarters of teachers reported that they felt the Grade Level Lessons and Activities were effective. Just under half of teachers reported the Storybooks as effective.

Consistent with teachers’ reported frequent implementation of Meet Up and Buddy Up, classroom observations revealed some combination of Meet Up activities, Buddy Up activities and Harmony lessons. In most classes, teachers were observed using verbal cues to explain activity instructions, expectations, guidelines for activities and Harmony lesson goals. Teachers also used verbal cues to remind students of rules and review previous lessons before beginning activities. About half of the observed classes included a Meet Up activity (n=14). The most common meet up activity included some form of a whole-class discussion (n=11). Most observed classrooms included a Buddy Up activity (n=22) with the most common activities being a buddy discussion (n=10) or hands-on activity (n=9). The Harmony lesson topics included: communication, conflict resolution, empathy, friendship building, getting to know others, getting along with others and recognizing positive character traits. Over one-third of the observations reported seeing a Harmony lesson (n=12). Of these classes, half included the teacher reading a story while the other half included a teacher-led discussion and/or group activity.

Participant reactions

All three participant groups (principals, teachers and students), shared favorable views concerning the Harmony program, though a few notable variations did appear between schools. Overall, the strong consensus across each of the participant groups was that Harmony has been particularly beneficial in helping students form better relationships with their classmates and has helped foster better overall classroom climate in the participating schools. Participants also consistently reported that they would recommend the program to other teachers and schools. These findings, along with additional trends pertaining to participants’ overall reactions to the program, are discussed in the section below.

During interviews, principals’ overall impression of the program and feedback were overwhelmingly positive. Among the reactions offered were “It’s incredible” and “I’m impressed.” One said, “It’s going well considering the later start. I would definitely recommend it to other schools.” Another said, “I feel good, we are setting the foundation with collective buy-in, building capacity.”

When prompted about teachers’ responses to the program, principals most frequently noted teachers’ appreciation of the structure that Harmony provides for them to integrate SEL skills into the regular school day. Harmony also facilitated dealing with emotionally charged situations in the classroom. For example, one principal described a change in how teachers respond to disruptive students:

You sometimes want something other than what you typically do like sending kids to the office. What we think should be done is not what always changes the behavior. Harmony provides some steps you can do in the classroom that can change that. Even though you think you know it because you’ve done it, when you see examples you learn stuff. Even after 34 years, I realized I was/wasn’t necessarily doing the right thing.

Principals also described some resistance among teachers. Such resistance was a particularly strong theme in Thorton Ridge. As outlined by the school principal:

After initial training, teachers were irritated and didn’t really want to implement. That was part of the reason for delay in implementation. My staff is very discerning. They don’t want things on top of their already full plates they feel would be wasted time and energy.

The counselor at this school explained a more hopeful view: “Teachers that were grumpy at the beginning are starting to take it on. They are coming around. The teachers who were excited about it think it’s so great, and that’s making it gain acceleration throughout.”

In their focus group and survey responses, teachers conveyed generally positive perceptions with regard to the Harmony program overall. Nearly 80 percent agreed that participating in the program was beneficial for their students and over 70 percent indicated that they would recommend the program to other educators.

Not surprisingly, we observed differences in perceptions between sites. Over 80 percent of teachers from Clear Creek and Riverdale agreed that participating in Harmony was beneficial for their students. A slightly smaller proportion of teachers (around 70 percent) from Sunnyside and Thorton Ridge agreed that this was the case, while most teachers from Hampton expressed neutral opinions. Wider differences existed between schools with regard to recommending the program. Roughly 95 percent of the teachers at Riverdale agreed that they would recommend Harmony to other teachers; and around 70 percent from Sunnyside and Hampton agreed. In contrast, only about half of the teachers at both Clear Creek and Thorton Ridge agreed that they would recommend Harmony.

Despite variation between schools in survey responses, teachers from all five schools noted during focus groups they would recommend the program to others. The reasons they offered were consistent with their perceived benefits and impacts of the program. Most often, teachers recognized the importance of teaching their students social-emotional skills and appreciated that Harmony is a full program that they can use without having to create lessons or activities themselves. Teachers also stated they would recommend Harmony to others because it is easy to use by new and experienced teachers, builds a sense of community within the school and develops important skills in students.

Students also shared their overall impressions of Harmony during focus groups and were very positive. The near unanimous consensus from students across the five schools is that they would recommend Harmony to other schools. Exemplary statements are:

I think it is great, it puts a smile on everyone’s face, and that is what it is all about – the inventor did a good thing!

It helps us through our day since we do it first thing in the morning – it gets us prepared to work.

It’s fun and I like it because it teaches different things, (like) ‘not be a bully, but be a buddy!’

Students also shared a variety of examples of ways that they have used what they have learned in Harmony outside of class. Most commonly, they highlighted improved social skills that they have developed from using Harmony, such as improved sportsmanship, better skills with interacting with friends, better overall listening skills, better skills with talking to adults and improved generosity with strangers. In addition, students frequently shared examples of how they applied Harmony teachings in interacting with their siblings and in resolving fights and disagreements. A common theme that also arose in the focus groups from three schools was how students believe the program is teaching them lessons that will be valuable as they progress through school, and eventually, into adulthood. Specifically, multiple students expressed that the program has helped them become better at making friends and interacting with new people, and that this will come in help when they leave elementary school for middle school.

Discussion

We conducted this mixed-methods study to examine how participants in diverse schools newly implement the Sanford Harmony social-emotional program and perceive its benefits for students and overall school climate. More specifically, we focused on the scale-up potential of Sanford Harmony using a combination of traditional implementation practices allowing for school extensive school autonomy with a more formal “MOU-based” usage model. An overall goal was to inform the planned expansion of the program nationwide to serve 30m school children over the next five years. As interpreted below, our findings indicated expected variation in implementation across schools, although all participants reacted favorably to the program and, importantly, would recommend the program to others. Administrators, teachers and students all saw the value of the program, particularly in terms of student relationship building and improved school climate. Common implementation practices and challenges supported recommendations for strengthening implementation support for schools desiring to sustain the program over time and maximally derive its potential benefits.

Implementation

As part of the study, each of the five schools agreed to implement the daily practices of Meet Up and Buddy Up, along with one lesson selected at the teachers’ discretion. Findings from the present study revealed inconsistencies in terms of implementing these components. Specifically, teachers noted they most often utilized the Meet Up and Buddy Up activities but were less likely to report incorporating a unit lessons. Few teachers reported implementing the Sanford Harmony program to the level specified in the project agreement. This finding of implementation variation is consistent with Miller et al.’s (2017) analysis of implementation data in their study of Sanford Harmony.

One constraint on implementation fidelity was the later start for program implementation. Teachers conveyed a strong desire to implement Harmony activities early in the school year to not only aid in building a sense of community, but also to better allow teachers to integrate the program into their teaching routines. Another limitation according to some teachers in the surveys and focus groups was the need for more professional development. While survey results indicated that teachers generally felt prepared to implement the Harmony program, interviews and focus groups revealed that both principals and teachers felt the initial 1.5-hour professional development was too brief. As noted by Garet et al. (2001), shorter professional development is less likely to have an impact on teacher practices as compared with sustained and intensive professional development. Further, training that incorporates such practices as observations, discussion, practice and reflection are important components to ensure effectiveness (Garet et al., 2001; Putnam and Borko, 2000). While the initial training incorporated active learning activities, the time for teachers to truly understand how to incorporate the program fully may have been insufficient. Two schools requested additional, follow-up training and it appeared that this training helped teachers better understand how to incorporate the program activities into their classrooms. Rogers (2003) describes this type of knowledge as how-to knowledge, which is important in helping to inform the user how to implement the innovation, as “when an adequate level of how-to knowledge is not obtained prior to the trial and adoption of an innovation, rejection and discontinuance are likely to result” (p.173). A final limitation based on interviews and focus groups was a lack of strong leadership in two schools to support implementation.

Despite the implementation challenges at some schools, participant reactions to the Harmony program were highly positive. Teacher survey results indicated that of those that used program components, over 85 percent indicated agreement that they were effective tools. The vast majority of teachers also agreed that participating in the Harmony program has been beneficial for their students. This finding reflects general teacher buy-in for program implementation. Principals were also very positive about the program within their schools, particularly regarding the positive impact they were seeing within their building. Last, students appeared to value the program with near unanimous consensus that they would recommend the program to others. They recognized the positive impact the program was having in teaching them problem-solving and peer relationship skills.

Several factors appeared to facilitate the implementation of Sanford Harmony, and these factors are consistent with the model of implementation described by Durlak and DuPre (2008), as well as Rogers (2003) diffusion of innovation theory. Both note the importance of the program characteristics, the individuals choosing to adopt the program and the organizational structure in terms of implementation and outcomes.

Our study revealed that the features of the Harmony program were important to implementation. As described by Rogers (2003), the characteristics of an innovation such as relative advantage, compatibility, complexity, trialability and observability are key predictors of adoption. Within our study, teachers did appear to recognize that the program was a vast improvement over their previous practices of identifying materials to support SEL on an ad hoc basis. This notion of relative advantage has been demonstrated in prior research to affect adoption of an innovation (Chitiyo and May, 2018; Wilson et al., 2008). Second, participants also saw that Harmony was compatible with their school goals and the needs of their students. This factor was most apparent in schools with highly diverse student populations. Third, a notable attribute of Sanford Harmony is its simplicity. Many teachers noted that the program was easy to use by both new and experienced teachers. Last, teachers were able to see a positive impact of the program within their students, which relates to the importance of observability as described by Rogers (2003) when considering adopting an innovation. Given these positive perceptions of the program, it is not surprising that the vast majority of teachers used the program components at least occasionally.

The second factor that we found that affected implementation was how the decision was made to adopt the Harmony program. In two of the participating schools, the principal made the executive decision (i.e. an authority innovation decision) to adopt the program, whereas in three of the schools, teachers were more involved in the decision (i.e. a collective innovation decision). Consistent with Durlak and DuPre’s (2008) finding in their review of implementation research, shared decision making enhances implementation of a program. The authors relate this finding to the principle of empowerment in community psychology. Here, individuals within a community exert influence and control over the decisions that affect their lives (Zimmerman, 2000).

We also found stronger implementation within schools where a teacher had previous experience with the program and influenced adoption by her peers. These teachers helped to serve as champions (Rogers, 2003), supporting adoption through modeling implementation and being an internal expert on the program. Prior research has shown that innovation champions can have a strong influence on the adoption of an innovation within their social system (Durlak and DuPre, 2008; Goodman and Stekler, 1989; Schmidt and Taylor, 2002).

Implications for practice and research

As National University initiates its planned five-year national scale-up of Sanford Harmony, the present research has potentially important implications for increasing success in involving and supporting schools in using the program. Specifically, as shown in the Morrison et al.s’ (2017) study and our findings here, Sanford Harmony offers teachers highly practical and appealing strategies for integrating SEL with daily school activities. These very positive program qualities, however, can also impede full program implementation and concomitantly limit potential benefits for students, when teachers overly rely on less logistically demanding components, such as Buddy Up and Meet Up, while deemphasizing use of lessons, storybooks and other program components. In this regard, as the scale-up initiative unfolds, strengthening implementation support and accountability as described below might be considered. Some of these enhancements are already underway in product design and others are suggested from the case-study findings reported here.

One need expressed by teachers is greater access to training and implementation support. Currently, the Sanford Harmony developers are creating online and web-based resources to provide teachers with on-demand professional development and general information about program components. This online resource will allow for onsite training to focus more on how to integrate Harmony program components across grade levels and in different subject areas. Another refinement is making materials adaptable and downloadable online so that teachers are not required to make photocopies, thus streamlining lesson implementation. Based on the present findings, further recommendations include providing ongoing internal and external support to teachers implementing the program, assisting teachers in adapting daily schedules so that there is time to implement Harmony components regularly and ensuring that each school has a SEL team to support implementation. Such a team might include the principal, guidance counselor and experienced and dedicated teacher that can help peers to problem solve challenges with integration and can model Harmony practices.

Limitations and future research

Obvious limitations of the present study concerned the small sample size of only five schools and reliance on participant and research perceptions of implementation and outcomes. While Sanford Harmony is well liked by teachers, principals and students, and perceived to be beneficial for SEL, rigorous research is needed to demonstrate its efficacy in improving outcomes such as student behavior and school climate. Currently, several such studies are planned for the 2019–2020 school year, which will involve comparing educational and social-emotional outcomes for students in Sanford Harmony schools and matched control schools.

Table I

Student survey sample

	Percentage of sample (n)
Grade level
3rd Grade	32.5% (262)
4th Grade	29.4% (237)
5th Grade	38.1% (307)
Gender
Male	46.2% (435)
Female	53.8% (373)
Race
Asian	12.1% (88)
Black/African–American	9.2% (67)
Latino/Latina	35.0% (255)
White	43.7% (318)