Assessment resistance: using Kubler-Ross to understand and respond

Abstract

Purpose

Faculty participation in the assurance of learning (AoL) is requisite both for the effective operation of the system and for accreditation compliance, but faculty often resist engaging in AoL tasks. The purpose of this paper is to provide specific recommendations to address faculty concerns and to guide AoL systems toward maturity.

Design/methodology/approach

This paper provides a comprehensive model of faculty resistance perspectives aligned to AoL maturity, provides specific responses to faculty resistance and introduces success markers of progress toward maturity.

Findings

Specifically, a three-stage model of AoL system maturity is presented and aligned with five faculty perspectives. For each faculty perspective, responses targeting causal factors are proposed and signs of progress toward the next level of faculty engagement are highlighted.

Practical implications

Faculty and AoL leaders will be able to identify their current stage of AoL system maturity and implement practical solutions to move to the next stage of system maturity.

Social implications

Understanding the motivations for faculty resistance will facilitate more meaningful and effective internal interactions as a school seeks to improve its AoL system. In turn, a more effective AoL system will promote better learning experiences for students; and better learning allows students to become productive in their chosen careers more quickly, thus improving society as a whole.

Originality/value

To the knowledge, no prior paper has organized faculty resistance along a maturity continuum, provided targeted responses based on the level of maturity or included signs that indicate growth toward the next level of maturity.

Citation

Tarnoff, K.A., Bostwick, E.D. and Barnes, K.J. (2021), "Assessment resistance: using Kubler-Ross to understand and respond", Organization Management Journal , Vol. 18 No. 5, pp. 176-186. https://doi.org/10.1108/OMJ-03-2021-1177

Introduction

Faculty members are concerned about what students need to learn to be successful and about how well their students master their course content. However, imparting requisite knowledge and skills involves a number of factors. Setting aside student characteristics and non-academic factors to focus only on academic content, a “traditional college education” consists of a curriculum that purports to coordinate multiple courses into a cogent collection of skills, knowledge and experiences that will culminate in professional success. Courses are often from different disciplines and are taught by different instructors; and course sequences take years to complete. Thus, as productive as one instructor might be, it takes learning sequenced across courses to adequately prepare students for success. However, the importance of these connections is lost when courses are only connected to one another by a prerequisite chain. How, then, can student learning be coordinated meaningfully across time, courses and instructors? One way to facilitate curriculum congruence is via assessment.

Assessment has been defined with respect to accounting education as “making judgments about student learning and development in a systematic way. Information is collected, analyzed and interpreted to make inferences about student learning and development” (Herring & Izard, 1992, p. 2). Since faculty own the curriculum, they should want to participate in the evaluation of its quality and effectiveness. Accreditors also generally require an assessment to ensure academic program quality. For example, the standards of the Association for the Advancement of Collegiate Schools of Business International (AACSB) define assessment as “the systematic processes and assessment plans that collectively demonstrate that learners achieve learning competencies for the programs in which they participate that are within the scope of the school’s accreditation” (AACSB Business Accreditation Standards, 2020, p. 41). Thus, given instructors’ focus on student success and accreditors’ assessment mandates, one would expect faculty participation in assurance of learning (AoL) to be nearly universal. However, AoL deficiencies are cited at AACSB conferences as a common reason for schools receiving a Continuous Improvement Review (CIR) rather than accreditation reaffirmation (Banks & Reid, 2017; Haberline & Romano, 2018). Certainly, faculty resistance to assessment could be one of the key factors contributing to such AoL deficiencies.

This article uses a three-stage model of AoL system maturity (Martell, 2005) and subdivides these stages into five faculty perspectives. For each perspective, specific responses to faculty concerns are proposed to facilitate AoL system maturity.

Levels of assurance of learning system maturity

The development and implementation of an AoL process is a journey requiring both time and effort. Assessment process development can be viewed as evolving through three maturity stages as follows: Awareness, Initial Implementation and Mature Implementation (Figure 1). Each stage is denoted by hallmark characteristics (Martell, 2005).

The Awareness Stage is characterized by a school recognizing the need for program assessment. Schools will likely have identified missions, goals, specific objectives and assessment methods. The school’s mission statement plays a critical role as the primary driver of the school’s academic programs and their assessment in the AACSB approach. Schools in this stage also define mission-aligned program competency goals and operationalize them into specific, measurable learning objectives. The creation of an initial assessment plan and infrastructure is also characteristic of this stage. However, the language of assessment likely remains relatively unknown among faculty, and staff and faculty and staff may not yet be convinced of the inherent value of assessment as a data-driven student improvement process viewing it, instead, as a compliance exercise (Martell, 2005). Faculty resistance, whether direct or passive, can be quite robust at this stage and is driven primarily by three factors a lack of understanding AoL basics, not knowing what is really required and not seeing the process’s value and the benefit of engagement.

The Initial Implementation Stage is marked by the presence of the first assessment plan and the development of some requisite infrastructure. In this stage, processes for collecting, storing, retrieving and reporting data are being implemented. A clear transition to direct measures is evident, and the language of assessment is recognized with discussions about assessment taking place outside of programmed opportunities (e.g. faculty meetings) (Martell, 2005). Faculty resistance in this stage is a function of an incomplete understanding of the process’s benefits. Faculty typically recognize that their engagement is necessary; however, the system is likely inefficient requiring a disproportionate investment of faculty time and effort while not delivering promised improvement. System inefficiencies and implementation difficulties may directly increase the intensity of faculty resistance. However, if the AoL culture is developing well, faculty resistance may be less vocal and more passive.

In the Mature Implementation Stage, assessment is a priority engrained in a school’s culture. Typically, one individual or a small group of faculty (e.g. assessment committee) are responsible for the process with funding budgeted for assessment. Faculty have regular access to assessment data, which is used to improve learning. The hallmark of a mature AoL system is that the “measurement loop” is closed for each objective. This means that each learning objective has been measured twice, once before and once after a data-driven, student-focused outcome improvement; and that the assessment results have been analyzed to determine the effectiveness of improvements with respect to improved student achievement of learning objectives. Resistance can still occur in this stage, but it is not likely to resemble the resistance encountered in earlier maturity stages. Rather, resistance in mature systems is often more active with faculty wanting greater influence over the implementation, operation and adaptation of the system.

Faculty perspectives, possible responses and “small wins”

As the AoL culture is maturing, the reasons for and expressions of faculty resistance may shift necessitating different responses to facilitate faculty engagement. During this maturation process, faculty perspectives on AoL can be delineated into phases mirroring the stages of grief (Kübler-Ross & Myss, 2008), namely, denial, anger, bargaining, depression and acceptance (Figure 1). While Kübler-Ross’s model and work on grieving were not developed in organizations, it has previously been used in the discussion of organizational change (Daugird & Spencer, 1996; Perlman & Takacs, 1990; Schoolfield & Orduna, 1994). For example, Woodward & Buchholz (1987), note that “When a loss occurs, the people who remain have to go through some basic states – denial, anger, bargaining, depression – to finally achieve acceptance.” (p. 66). As schools’ AoL processes evolve, initial negative faculty perspectives shift to more positive productive perspectives. Although individual faculty members may be at different phases of the AoL maturity process, the general attitude of a particular faculty group (e.g. college, department, discipline) can be characterized by the perspectives held by many/most of the members of that group and/or by certain influential faculty members. In fact, it is essential that a few faculty members be at a different phase than other faculty members to have a “faculty champion” or “early adopter” with whom to work to build and enhance AoL processes and to develop a positive culture of AoL. These perspectives align predictably with the three stages of maturity. Denial and anger are common in the awareness stage, whereas bargaining and depression are frequently exhibited in the initial implementation stage. Faculty perspectives shift toward acceptance in the mature implementation stage.

Anticipating and planning for these perspectives proves extremely beneficial to the system’s evolution to facilitate proactive support mechanisms addressing faculty concerns and encouraging faculty participation in developing a highly functional assessment culture. At each step, guided discussions on focused topics can mitigate negative consequences and foster progress in the AoL process. These discussions become increasingly meaningful and effective when specific topics align with probable types of resistance typical for each perspective. Such discussions often result in “small wins,” defined as observed behaviors indicating incremental progress toward AoL system maturity supporting meaningful assessment and driving effective improvements. Each of the following sections focuses on one of these perspectives and provides responses that address emergent faculty concerns and describes the “small wins” that AoL leaders may observe as indicators that the faculty and the system are maturing. Figure 1 aligns system maturity stages with the five faculty perspectives and summarizes possible responses for each phase.

Denial in the awareness stage

Early in maturity, faculty are often in denial regarding assessment and its requirements (ABET, 2017). Faculty reactions can be summarized by the phrase, “I won’t do it!” Faculty express strong resistance to the very notion of assessment (Rubin & Martell, 2009) viewing assessment as being dictated by external forces (Rohrbacher, 2017) with little or no personal or organizational benefit (Demoranville, 2010). Faculty in this phase often view assessment as impinging upon their academic freedom (Demoranville, 2010; Pringle & Michel, 2007). Resistance in the Denial Phase often occurs because faculty have little or no understanding about what assessment truly is. Faculty in this phase may also be resistant due to confusion regarding differences between course and program assessment. For example, faculty who routinely improve their courses each semester may mistakenly conclude that they are already engaged in program assessment. Faculty resistance in this phase may also occur because AoL is viewed as being imposed by accreditors and administrators. Faculty may not yet have internalized that assessment’s true purpose is providing an environment conducive to improving student learning.

Constructive responses to faculty in this phase begin with genuinely listening to faculty concerns and providing reassurances (Martell, 2005; Rubin & Martell, 2009). Much of the effort in this phase focuses on training that emphasizes assessment’s beneficial outcomes for students and enhancing program reputation (Rubin & Martell, 2009). Faculty training should dispel misperceptions about what AoL is and is not, reinforce AoL’s purpose to improve student learning and teach assessment terms and AoL basics (Kelley et al., 2010). Training should reinforce that the purpose of assessment is the measurement of student learning not the evaluation of instructional quality or faculty performance (Demoranville, 2010; Eschenfelder et al., 2014; Martell, 2005). Since most faculty feel responsible for student learning, demonstrating how assessment results align with and enhance faculty’s curriculum management efforts can yield significant benefits. Evidence of AoL-based changes that have improved student learning may be scant initially, but highlighting even small changes that have improved student learning provides powerful evidence to reinforce enhancing student learning as the purpose of assessment. Faculty may focus energetically on academic freedom, and if this issue is raised, it is important to remind participants that academic freedom protects faculty freedom of speech and perspective but does not support refusal to participate in assessment (Williams, 2005). As dialog continues, leaders should ask faculty for input to identify ways to minimize faculty workload while maximizing the impact of assessment. Such conversations allow leaders to better determine what specific support is needed. Identifying and acknowledging constructive criticism is a meaningful way to demonstrate the continuous improvement mindset, to reinforce that AoL is faculty-led and to give credit to faculty supportive of AoL. However, one of the most crucial actions may be simply celebrating the willingness of early adopters to participate.

Small wins in the Denial Phase include the willingness of some faculty to engage in conversations about assessment data, to recommend improvements and to demonstrate a basic understanding of AoL terminology. Given the criticality of training faculty to develop an effective student learning environment, faculty participation in training is also a positive indicator of progress.

Anger in the awareness stage

As faculty learn more about assessment, their next response is often anger. While moving faculty beyond the Denial Phase to the Anger Phase may not seem like a “win,” it truly is. Faculty in the Anger Phase no longer deny the existence of the AoL system and are, in fact, engaging with the system despite having a negative mindset. Faculty in this phase may feel strongly that AoL interferes with their autonomy, professionalism and expertise (Rohrbacher, 2017). Virulent resistance is often driven by the mistaken belief that the data will also be used to evaluate teaching effectiveness (Rexeisen & Garrison, 2013; Kelley et al., 2010; Martell, 2005; Pringle & Michel, 2007; Rohrbacher, 2017). Thus, resistance to AoL from faculty in this phase is often driven by fear of punishment if student performance is inadequate. This fear is often based on a lack of understanding about the difference between a course and program assessment and confusion over the difference between learning and teaching evaluation. Furthermore, faculty may view assessment as a passing fad, and thus a waste of faculty effort that will not likely result in benefit to students, faculty or the organization.

Constructive responses to faculty in this phase include continued communication, training and reinforcement of an emerging AoL culture. Assessment is a service task similar to other more familiar faculty tasks. Leaders should also continue to identify ways to support faculty and encourage their input with the goal being faculty ownership of the process. Continued training in the concepts and language of assessment remains important, but the most important element is likely to be continued reassurance that assessment’s purpose is improving learning not evaluating faculty instruction or performance (ABET, 2017; Garrison & Rexeisen, 2014; Kelley et al., 2010; Rohrbacher, 2017; Rubin & Martell, 2009). Including faculty in the system’s design can provide some reassurance (ABET, 2017) and administrators can further allay these fears by publicly committing to never use AoL data inappropriately for personnel actions and by reinforcing this commitment with appropriate safeguards (e.g. always using aggregate data with all identifying information removed). The goal of improved learning should be perpetually kept at the forefront of the process (Rubin & Martell, 2009), and examples of efficacious AoL-driven interventions and faculty participation should be continuously celebrated.

Small wins in the Anger Phase include continued faculty participation in AoL conversations and training. Faculty should begin to recognize improvements in student learning afforded by data-driven interventions, and non-participating faculty may begin to notice accolades given to participating faculty. At this stage, the most important “win” for continued system maturity is earning the faculty’s trust in the system’s integrity as a tool to measure learning and improve program curricula rather than evaluate faculty performance.

Bargaining in the initial implementation stage

As the maturity of the AoL process advances from Awareness to Initial Implementation, faculty resistance and reactions also shift. Faculty are likely to acknowledge the necessity of assessment, but the faculty are not likely to fully embrace AoL yet. The predominant sentiment in the Bargaining Phase can be captured by the statements, “What if […]” or “If only […].”

While faculty in this phase interact with the system via reciprocal exchanges [for “life to be returned to what it was” (Kübler-Ross & Kessler, 2014, p. 17)], this is still an improvement vis-à-vis the Anger Phase, as emotions cool and conversations become more rational. Faculty express the rationale for their “exchanges” in a number of ways.

Rather than being overtly negative, resistance in this phase focuses on the anticipated time and effort required to implement requisite assessment activities by attempting to recapture the past by harkening back to pre-AoL times. Common faculty complaints include that assessment takes time (Demoranville, 2010; Garrison & Rexeisen, 2014; Kelley et al., 2010; Pringle & Michel, 2007; Rohrbacher, 2017), increases the cost and complexity of duties (Eschenfelder et al., 2014; Garrison & Rexeisen, 2014; Kelley et al., 2010; Pringle & Michel, 2007; Rohrbacher, 2017), lacks adequate resources (Rexeisen & Garrison, 2013; Garrison & Rexeisen, 2014; Rohrbacher, 2017), is not rewarded (Garrison & Rexeisen, 2014) and/or results in an inequitably distributed faculty workload (Garrison & Rexeisen, 2014). Recognizing that their workloads will increase, faculty seek to identify specifically what will be required to determine their personal time and effort commitments. If the perceived workload is too great, faculty will continue to resist efforts to gain their participation.

The most effective responses to faculty in the Bargaining Phase focus on improving process efficiency. For example, before collecting more data, it may be advantageous to determine whether current assessment data could be better used (e.g. using one student artifact to assess multiple objectives). In this stage, it is important to evaluate the distribution of work to ensure that an efficient division of labor is in place. Assigning support staff to assist in the AoL process can be helpful. Continuing to help faculty understand course versus program assessment can also enhance participation. While improvements from course assessment provide a foundation for improved student learning, these effects are multiplied when faculty cooperatively coordinate their improvements across the program based on program assessment. Focusing early efforts on the learning areas most important to the faculty links these conversations to mutually desirable outcomes, demonstrates responsiveness to faculty concerns and invites faculty ownership of the process. Identifying faculty who are “early adopters” fosters continued engagement as does including their input into process improvement. Leaders publicly praising increasing participation and positive contributions is critical to the maturation process.

Small wins in the Bargaining Phase include faculty recognizing improved student performance resulting from AoL-driven improvements, faculty appreciating the recognition received for work in the system and faculty acknowledging system refinements that have reduced and more equitably distributed the AoL workload. As each of these elements becomes increasingly apparent, the assessment culture will continue to mature.

Depression in the initial implementation stage

In the Depression Phase, faculty are sometimes exhausted by the work already completed especially if the AoL process is inefficiently designed. While “depression” may not seem desirable, it is preferable to the Bargaining Phase, as faculty are looking beyond personal time and effort costs and are evaluating the system to make it more efficient, effective and sustainable.

It is common for faculty in this phase to feel that the system is neither functional nor sustainable. Faculty may express despair (depression) about inefficient and ineffective assessment (ABET, 2017; Kelley et al., 2010; Rubin & Martell, 2009). The scope of this reaction may be as broad as a lack of confidence about assessment knowledge (Eschenfelder et al., 2014; Garrison & Rexeisen, 2014; Pringle & Michel, 2007; Rohrbacher, 2017), as specific as frustration over imprecise measures or subjective student outcomes (Rohrbacher, 2017) or as fundamental as not knowing how to implement a meaningful intervention (Garrison & Rexeisen, 2014; Martell, 2007). Faculty may also become aware of difficulties assessing core areas of the curriculum that are not “owned” by anyone (Rohrbacher, 2017). Faculty may also struggle to demonstrate assessment’s value (Rohrbacher, 2017) or may fail to see adequate improvement in learning.

Faculty in the Depression Phase may feel that support to sustain AoL efforts is lacking (Rohrbacher, 2017). Thus, resistance in this phase often reflects the belief that data collection requires too much time and effort. This burden may be compounded by the system’s design if individual faculty members are being tasked with the collection of data, a compilation of results, analysis of data and making improvement recommendations. Perpetuation of a lack of understanding about the differences between course and program assessment, carried over from the Awareness Stage, can cause faculty to continue to be required to perform multiple course tasks, some of which could be more efficiently performed as a centralized administrative function.

The most important response to faculty in the Depression Phase is encouragement. Most faculty will have acknowledged the need to participate in AoL and may just need to give their efforts time to bear fruit. Individual recognition continues to be important in this phase and celebrating groups of faculty working together on AoL-driven improvements is also beneficial (e.g. designing new assessment instruments, clarifying stakeholder expectations of graduates, implementing cross-curricular improvements). Student learning is of paramount importance to faculty, so connecting specific, data-based AoL interventions with improved student performance helps faculty conclude that AoL is worthwhile. Where these connections are not obvious, conversations can focus on questions about system improvements (i.e. mission alignment of goals and objectives, improvements to measures and representativeness of sampling). Continued training for faculty is essential, and leaders might use specific examples to prompt additional questions guiding discussions. Finally, continued effort should be dedicated to eliminating “pain points.” Asking faculty where they need help and incorporating their feedback regarding system improvements can improve morale.

Small wins in the Depression Phase include increasing faculty participation in training and the emergence of collaborative AoL activities among faculty groups (e.g. discipline-specific innovations, cross-disciplinary AoL coordination, task forces or learning communities for specific objectives). Faculty may better understand AoL terminology, although fluency may remain challenging. Improvements in student performance, even if limited, can serve as a “proof of concept” for skeptical faculty.

Acceptance in the mature implementation stage

As faculty move into Mature Implementation, interactions will generally have a more positive tone as faculty begin to see the connection between AoL and improved student learning and acknowledge that AoL is “here to stay.” While it might be tempting to surmise that the Acceptance stage is characterized by a fully functioning, highly efficient and effective AoL process absent of problems or resistance, that may not be the case. Although faculty ownership of the process increases with most faculty fully engaged in the AoL system, some faculty may still lack AoL knowledge and may still be unsure of how to use assessment results (Eschenfelder et al., 2014; Kelley et al., 2010). Overall, faculty are willing to improve the AoL system and even those who are not yet ready to assume ownership of AoL processes are likely to ask, “What do we do next?” (ABET, 2017). Irregular AoL processes may continue as adjustments are made (ABET, 2017; Rohrbacher, 2017) and AoL activities may lack follow-through or an effective feedback loop (Rohrbacher, 2017). As a result, some faculty may believe they are just “going through the motions” and that compliance is still the primary motivation for AoL (Garrison & Rexeisen, 2014). However, this phase can be distinguished from the Depression Phase by faculty involvement in the AoL system as follows: continuously focusing on improved student learning, regularly seeking more efficient and effective AoL processes and increasingly taking ownership of AoL results to inform curriculum decisions (Knott et al., 2018).

Responses to faculty in the Acceptance phase should guide faculty toward greater participation and ownership of processes and outcomes. While emphasizing continued support and advanced training to maintain momentum and to continue the maturation of both faculty and the system, leaders might constructively ask faculty questions to seek input and specific recommendations from assessment results, foster engagement, guide adoption and educate them on the consequences of their decisions. Administrators should also clearly identify where and how assessment activities will be included in faculty evaluations. When concerns and complaints about learning or the system arise, a constructive way to shift conversations positively is to view these issues as a plea for better data to drive improvements. Asking faculty what data is needed to solve their problem and providing the requisite data invites deeper participation with the system and reinforces a data-informed, continuous improvement mindset to enhance student learning. Leaders might also consider sharing the AoL “master plan” with interested faculty to aid their understanding of the design and logic of the system and to help them anticipate engagement expectations. Without question, improvements in learning must remain the AoL process’s focus.

With increasing faculty ownership, leaders should identify a faculty champion to be trained to take on “guided ownership” of the process. However, it is important to remember that the system must remain viable and sustainable, so caution should be exercised when system complexity increases. Guiding faculty to simple approaches is preferable to maintaining complicated approaches, as unnecessary complexity threatens the continued viability of the system. Pilot testing provides a valuable way to evaluate new approaches. Additionally, when data is collected, results should be made available to faculty quickly to allow for timely improvement recommendations and their implementation. Refinement of the process’s structure and improved communication of results (e.g. using high impact graphics) are beneficial in this stage.

What may prove most difficult in this phase is the maintenance of what has been gained. Although regression is possible at any phase, the threat of regression may be greatest in the Acceptance Phase since faculty and AoL leaders do not have a “next level” for which to strive. As a result, AoL may be considered “done,” and administrative and faculty focus might shift to other initiatives. If not addressed, the system’s maturity could be jeopardized by internal and external forces such as preparing and training new faculty, hiring experienced faculty with negative AoL perspectives, turnover in administration, reduction in resources, the distraction of faculty, changes in accreditation standards and/or large-scale disruption to industry expectations of graduates.

Small wins in the Acceptance phase include faculty reinforcing the AoL culture, making suggestions to improve the system, maintaining the gains already achieved and ensuring that the robust AoL culture is not eroded over time. It is important to maintain simplicity and to recognize that 100% faculty participation is not necessary. Faculty in this phase should keep improved student learning at the forefront of discussions, make data-driven curriculum revisions, capture results with high impact graphics, continue ownership of the process, encourage new faculty to participate in the assessment and mentor new faculty as needed. “Closing the loop” will be a regular part of faculty conversations; and system improvement will be guided by questions regarding what did or did not work, where students need help and next steps. Ultimately, faculty will continue gaining confidence in their ownership of the process, their communication of the results and their leadership of AoL discussions.

Conclusion

Once accreditation is achieved, the successful school shifts to maintenance and focusing on the next CIR. Changes and challenges will occur with turnover, standard revisions and perhaps even financial and enrollment windfalls and shortfalls. While the school may achieve AACSB accreditation, the AoL development cycle is an ongoing and evolving process.

The purpose of this article is to identify common phases of faculty resistance to AoL and to provide specific recommendations to help leaders guide AoL systems toward maturity. Given accreditors’ mandates for assessment and the importance faculty place on student success, faculty participation in AoL should be high. However, AoL is a common point of accreditation failure (Banks & Reid, 2017; Haberline & Romano, 2018) which may be due to intense faculty resistance. To assist faculty and AoL leaders in their journey toward enhancing system maturity, a three-stage model of AoL system maturity comprising Awareness, Initial Implementation and Mature Implementation was presented. The stages of maturity were further subdivided into five, more tightly-focused faculty perspectives with specific and practical responses provided for each phase to help faculty advance system maturity. Also, highlighted were the “small wins” that AoL leaders may observe as signals that faculty and the AoL system are, in fact, advancing toward maturity.

Figures

Figure 1.

Alignment of assurance of learning system maturity levels, faculty perspectives and responses to promote system development