Creating a content validation process to advance a campus-wide leadership framework

Abstract

Purpose

A content validation process of an institutional leadership framework is described for leadership educators in higher education. We created this process to further integrate our leadership framework across campus, maintain alignment with advancements in leadership research and ensure it is broadly inclusive and culturally responsive.

Design/methodology/approach

Our approach included seven essential design elements and was informed by a review of leadership frameworks in practice and the literature, validation studies and a comprehensive document review.

Findings

Our approach yielded a validated leadership framework with modifications to its principles, values, competencies and outcomes. Modifications addressed pre-determined criteria and were deemed relevant to leadership research and our institutional context.

Originality/value

The external content validation process of our leadership framework is novel and serves as a valuable guide for those considering opportunities to strengthen their own institutional approaches to leadership education.

Citation

Kaufman, A.L. and Kueppers, M.R. (2024), "Creating a content validation process to advance a campus-wide leadership framework", Journal of Leadership Education, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JOLE-04-2024-0055

Introduction

Leadership frameworks are used widely in institutions of higher education (IHEs) by campus stakeholders to understand and coordinate diverse leadership initiatives (Gigliotti & Ruben, 2017; Kotnour, Hoekstra, Reilly, Knight, & Selter, 2014; Rosch & Stephens, 2017; Soria & Johnson, 2020). Common frameworks for leadership help discover synergies across leadership initiatives and formalize the many, disparate approaches to leadership within higher education (Gigliotti & Ruben, 2017; Ruben, De Lisi, & Gigliotti, 2018). Using frameworks raises two important questions. Are leadership frameworks validated to align with leadership scholarship? How are leadership frameworks developed to meet the needs of those most marginalized?

After developing a campus-wide leadership framework at the University of Wisconsin-Madison (UW-Madison) through an internal collaborative process, organizational leaders designed a process to ensure concepts had relevance to the field of leadership studies while accounting for their context. The decision to engage in a content validation process was motivated by a desire to further integrate the leadership framework across campus, maintain alignment with advancements in leadership research and ensure it is culturally responsive. No other IHEs had yet conducted external content validation processes of their institutional leadership frameworks. This article describes our content validation process, including the findings and recommendations for those considering such an approach.

University case context

The UW-Madison, a public university and top-tier research institution, is the flagship university of the University of Wisconsin System. The University of Wisconsin was founded the year the state of Wisconsin was established. UW-Madison became the state’s land-grant university after Congress adopted the Morrill Act on ceded land of the Ho Chunk nation. UW-Madison’s vision is to serve as a “public resource that works to enhance the quality of life in the state, the nation and the world,” with a mission to remain a comprehensive teaching and research university with a commitment to public service (www.wisc.edu).

UW-Madison has a rich history of developing students, faculty and staff as engaged community members in an ever-changing global society. Leadership is demonstrated in the private sector, non-profit organizations and public service. This is not coincidental but an outgrowth of UW-Madison’s culture, mission and commitment to the Wisconsin Idea, which encourages education to influence people’s lives beyond the boundaries of the classroom.

UW-Madison launched a campus-wide leadership framework in 2014. Leadership development has long been present at our institution, yet there was no shared understanding of leadership on our large, decentralized campus. A concerted effort was undertaken in December 2011 to understand and align campus leadership development opportunities. A staff member within the Center for Leadership and Involvement (CfLI), a department within the Division of Student Life, assessed campus leadership efforts by interviewing more than 100 campus stakeholders. This assessment yielded considerable interest in developing a shared understanding and common language for leadership at UW-Madison. The phases of this initiative moved from framework development (2011–2014) to stewarding its adoption on campus (2014–2019) and conducting a rigorous content validation of the leadership framework (2019–2021).

Leadership framework development

Several campus units sponsored and championed the initiative, including the Office of the Provost, the Division of Student Life, the Office of Human Resource Development (OHRD) and CfLI. In the summer of 2012, OHRD and CfLI approached those with positional authority for institutional support. A planning team was created to ensure broad grassroots participation in the development of a campus-wide leadership framework. The team was comprised of graduate and undergraduate students, administrators, classroom instructors and academic staff from many campus units. These members, representing a cross-section of leadership educators and practitioners, worked collaboratively to provide universal leadership language for the campus community. Among the planning team members, four teams were formed around areas that would be used to develop a campus-wide leadership framework: (1) leadership research and scholarship, (2) leadership programs, (3) institutional values and initiatives and (4) outreach and feedback from campus practitioners.

All stakeholders committed to the view of leadership for positive change and set out to develop a common leadership framework that accounted for our institutional context. In 2013–2014, these teams developed and vetted the proposed framework and then launched it in 2014. The UW-Madison Leadership Framework was designed as a non-prescriptive guide to bridge campus contexts, serve multiple audiences and integrate key concepts from leadership studies and our institutional context.

Leadership framework content

Leadership framework concepts were organized into four categories of content: principles, values, competencies and outcomes. Table 1 offers general descriptions of the four categories and the number of original concepts per category. The leadership framework was grounded by a set of foundational principles and integrated institutional and universal leadership values with leadership competencies, which, when enacted, can result in leadership outcomes.

The leadership framework’s principles, informed by leadership programming and research, included:

• (1)

  Leadership is an action-oriented endeavor not based on position or level of authority;

• (2)

  Context matters – each situation requires unique engagement and

• (3)

  Leadership is understood as the phenomenon of change in an individual, group or communities’ beliefs, values or behaviors.

The leadership values in the framework were developed mostly from our institutional history and context and are an outgrowth of our university’s culture, mission and values. The three original values in this leadership framework were integrity, inclusive engagement and connection and community. These values were respectively grounded in institutional commitments to academic freedom (fearless sifting and winnowing to seek the truth), shared governance (shared responsibility in decision-making) and the Wisconsin Idea – the longstanding vision aimed at preparing all Wisconsinites to have agency over their lives (Drury, 2020).

Competencies were in service of a task or goal. The original seven competencies reflected concepts found in leadership research and practice: self-awareness, interpersonal communication, supporting learning and development of others, honoring context and culture, decision-making, fostering bridge-building and collaboration and moving ideas into action. Outcomes occur at the intersection of a value and competency. The 89 original outcomes are generated when enacting a leadership competency while living leadership values and grounded in leadership principles.

A review of leadership frameworks in practice and literature

We conducted a review of campus-wide leadership frameworks developed for IHE contexts. This two-part review showed the importance of IHE leadership frameworks in the literature and in practice. We found campus leadership frameworks in published case studies and in our preliminary review of practice from IHE websites. Much variation was evident from the review. Institutional leadership frameworks, published and unpublished, varied in purpose, audience, concepts, perspective and approach. Frameworks were often developed collaboratively with many campus partners by assessing student and campus leadership needs and programming.

Leadership models and frameworks are prominent in leadership literature and practice, and yet these are regarded differently. Some campuses fully or partially adopt an existing leadership model from the literature. Models represent a theory visually and with a description (Dugan, 2017). Prevalent in higher education are student leadership development models, such as the social change model of leadership development (HERI, Astin, & Astin, 1996; Komives, Wagner, & Associates, 2017) and the student leadership competency model (Seemiller, 2013). Since our institutional effort did not adopt a specific leadership model, we instead focused our review on leadership frameworks developed for their IHE context. Leadership frameworks incorporate abstract ideas alongside practice-based concepts from their IHE contexts, which aligns with John Dugan’s definition of leadership framework as an “abstract representation of ideas, frequently conceptual or philosophical; have typically not been empirically tested (2017, p. 12).” Our review found that well-developed frameworks wove homegrown ideas from their context along with concepts from the literature.

Our review showed leadership frameworks are used to guide leadership development on college campuses. Leadership frameworks included philosophy statements, guiding principles, institutional values and core competencies. Some use a disciplinary lens, while others use an interdisciplinary one. A few offer assessments to determine needs and outcomes.

Leadership frameworks in practice

Our preliminary review of peer institutions’ websites was conducted in 2022, which included 146 IHEs with the R1 Carnegie Classification (doctoral universities with very high research activity). Of these, 35 institutions adopted leadership frameworks informed by their context and the literature. These institutions use discipline-based or interdisciplinary frameworks. Certain institutions specified that they had discipline-based leadership frameworks, which included Auburn University’s framework for students and staff in the School of Education or Clemson University’s framework for students in the College of Engineering, Computing and Applied Sciences. Other frameworks may specify various audiences or issue areas and may not specify if they were interdisciplinary. The University of Washington-Seattle campus emphasizes student leadership development, while Texas A&M University emphasizes staff and faculty development. Northwestern University and the University of Illinois Urbana-Champaign both focus on student leadership development with a campus-wide emphasis. North Carolina State University at Raleigh focuses on child welfare. A few institutions, such as Northwestern University, the University of Illinois Urbana-Champaign and the UW-Madison conducted needs assessments and created an online individual self-assessment. In our case, the UW-Madison Leadership Framework is interdisciplinary, has a campus-wide orientation and is for all institutional audiences: faculty, staff, students, campus units and across units.

Published case studies

The published case studies of IHE leadership frameworks show frameworks vary in complexity, audience and program focus. Some frameworks have a student focus, like the University of Illinois Urbana-Champaign (Rosch, Spencer, & Hoag, 2017). This campus-wide leadership education model is stewarded by students, faculty and staff and includes several components: a philosophy statement, skills and competencies and campus spaces for learning and practice. Another case study described the University of Iowa’s campus-wide framework, developed by all partners focused on student leadership development (Ashby & Mintner, 2017). Other cases had faculty and staff as their audience, such as Kansas State University’s community of practice for professional development (Scott & Tolar, 2009). Three case studies, using the UW-Madison Leadership Framework, focused on using the framework for specific leadership development purposes: faculty development for student academic achievement (Schmid, Gillian-Daniel, Kraemer, & Kueppers, 2016), library staff community of practices (Weltin & Schultz, 2019) and staff, student and community leaders sharing in ecological restoration work (Farrior, Kinsbury, & Herrick, 2024). Other frameworks were discipline-based, like the University of Central Florida’s framework for faculty and staff in their Engineering Leadership and Innovation Institute (Kotnour et al., 2014).

Limitations

Leadership framing in practice and in scholarship has limitations. The first points to the need for this study; we didn’t discover external content validation efforts for institutional leadership frameworks. The second set of limitations underscores shortcomings and posits ways diversity, equity and inclusion research can advance leadership framing. Many scholars acknowledged the deleterious effects of history and seek to address how some communities have been marginalized in higher education (Drury, 2020). Ospina and Foldy’s (2009) extensive review uncovered that many claim IHE perceptions and enactments of leadership framing are too narrow. IHEs often pursue modest diversity efforts, as these do not challenge institutional norms. Institutional cultural norms can thwart efforts or enhance culturally sensitive strategies and change efforts (Kezar & Eckel, 2002). Authors Ospina and Foldy’s review surfaced the social reality of race-ethnicity and how it informs leadership: the costs of color-blind approaches and of only considering people of color as special cases.

Many have called for the transformation of leadership. Ospina and Foldy’s (2009) research surfaced how deepening understandings of context, power and collective dimensions help to center marginalized voices, experiences and theories into leadership framing. Leadership educators in the International Leadership Association (2021) came together to provide guidance on transforming a wide range of leadership learning (training, development and education) in academic and co-curricular programs (https://ilaglobalnetwork.org/). Several authors advanced framing towards shared equity leadership (Kezar, Holcombe, Vigil, & Dizon, 2021; Santamaría, 2013; Ueda, Kezar, & Elizabeth, 2023), culturally sustaining leadership (Santamaría & Santamaría, 2016), culturally responsive leadership (Campos-Moreira, Cummings, Grumbach, Williams, & Hooks, 2020) or culturally relevant leadership learning (Bertrand Jones, Guthrie, & Osteen, 2016; Guthrie & Jenkins, 2018; Guthrie & Torres, 2021, p. 75). These and other authors center cultural responsivity into leadership. Ladson-Billings’ foundational culturally relevant pedagogy explains culture as an “amalgamation of human activity, production, thought, and belief systems” (1995, p. 75). We join the many scholar practitioners guiding curricular and co-curricular leadership programs who “ground leadership learning in a coherent conceptual framework within the context of the institution” (ILA, p. 1) and center leadership frameworks around cultural responsivity to inspire how marginalized voices can advance leadership frameworks.

Methods: our content validation process

Since the 2014 launch, the UW-Madison Leadership Framework has been widely used by students, staff, faculty and campus units in academic courses, co-curricular opportunities, employee development and campus change initiatives. In 2019, stewards of this leadership framework (organized into a collaborative group called Leadership @ UW) were conscious that the content had not been reviewed for over five years. Some potential adopters shared that a lack of validation was a barrier, especially since this campus leadership framework exists within a research institution. With only partial integration on campus, Leadership @ UW members wanted to ensure its ongoing relevance both within and beyond the institutional context and that it be broadly inclusive and culturally responsive to historically marginalized groups. CfLI leaders in consultation with assessment, research and evaluation campus experts were motivated to address the following research questions:

• (1)

  Is the UW-Madison Leadership Framework content aligned with existing and emerging leadership research?

• (2)

  Is its content inclusive and culturally responsive to historically marginalized groups?

• (3)

  Is its content clear, portable and reflective of the institutional context?

With these goals, we designed a rigorous external content validation process. We used a validation study – a type of evaluation often used to develop theory and improve programs (Kane, 2006; Taylor, 2013). Our approach aligns with an essential aspect or distinct type of validation study: content validation (Dellinger & Leech, 2007; Messick, 1989; Sireci, 1998; Taylor, 2013). Like validation studies outside the field of leadership, our approach explored the plausibility of our leadership framework.

We conducted a document review of the content validation process to gather information on how this process was designed and conducted. Mark Kueppers, a participant in the content validation process, shared internal documents with Ariel Kaufman including email communication, background and orientation materials, process instructions, survey instruments, feedback, analysis tools, meeting notes, recommendations, internal reports and validated the framework. This document review clarified the essential elements of the content validation process.

Coupled with a review of validation methodology (Bengtsson, 2016; Dellinger & Leech, 2007; Kane, 2006; Messick, 1989; Sireci, 1998; Taylor, 2013) and validation case studies (Andreasen, Lund, Aadahl, Gobbens, & Sorensen, 2015; Bengtsson, 2016; Davis, Stullenbarger, Dearman, & Kelley, 2005; Dellinger & Leech, 2007; Gabrenya, Moukarzel, Pomerance, Griffith, & Deaton, 2012; Rissi & Gelmon, 2014; Shupp, Wilson, & McCallum, 2018), the document review of our process surfaced essential elements that can guide future content validation processes. We outlined seven essential design elements of the content validation process.

• (1)

  Determine approach and scope;

• (2)

  Define roles and responsibilities;

• (3)

  Create materials, procedures and instruments;

• (4)

  Recruit, select and orient teams;

• (5)

  Collect content feedback;

• (6)

  Analyze content feedback;

• (7)

  Finalize revisions for a validated framework.

Determine approach and scope

CfLI leaders determined the approach and scope in consultation with process consultants and campus partners. The content validation process, conducted over two years, was designed to scrutinize all content of the UW-Madison Leadership Framework and produce a revised and validated framework. The content validated included 4 categories and 102 concepts. The process included two stages due both to the large amount of content to review and to the interdependence of the framework’s content – any substantive changes to the values or competencies would have a material effect on the outcomes. Therefore, stage 1 focused on the review of principles, values and competencies, while stage 2 focused on the review of outcomes. Although we conducted a content validation process in two stages with many distinct actors, another institution may conduct the process in one stage with a single set of actors.

Define roles and responsibilities

Content validation case studies in other fields highlighted significant roles and identified the best to serve in these roles. Several case studies detail how expertise from external and internal stakeholders offered in content validation processes informs program improvements (Andreasen et al., 2015; Bengtsson, 2016; Davis et al., 2005; Gabrenya et al., 2012; Rissi & Gelmon, 2014). In our process, planners deemed four roles as critical. In Figure 1, each quadrant has an accompanying rectangle defining the characteristics of those performing that role. The two quadrants on the left side of the circle show actors within the institution. Stakeholders within our university were considered internal stakeholders. The right side of the diagram shows actors mostly outside the institution, i.e. the external stakeholders.

External stakeholders, in our research and in other studies, help design the content validation process to explore relevance of results beyond the context studied (Bengtsson, 2016; Taylor, 2013). The process consultants and content reviewers were mostly external experts with no formal ties to our university. These two types of leadership experts informed the design and evaluated the degree of relevance of this content to leadership studies. Internal stakeholders were also vital to the process. The validation team and vetting team members were internal stakeholders. The validation team members, especially the project leads, defined and refined roles and responsibilities with guidance from process consultants. Vetting team members, students, staff and faculty at UW-Madison used their knowledge of the UW-Madison context and the Leadership Framework when evaluating content reviewer feedback. All four roles were important in both stages. Aware that this would be a significant time commitment, the validation team asked content reviewers and vetting team members to commit to only one stage. A few members committed to both stages, which offered helpful continuity, including a stage 1 content reviewer becoming the stage 2 process consultant.

Two roles were instrumental in designing and stewarding the process: the validation team and process consultants. The validation team, including the project lead, facilitated the process. The project leads, CfLI staff members, provided necessary support. Additionally, the validation team, in consultation with the process consultants, defined the process and key roles and responsibilities. CfLI staff and students and a student affairs assessment partner served on the validation team. Two esteemed leadership scholars provided guidance and oversight of the process. John Dugan (2019–2020) and Susan Komives (2020–2021), with no formal ties to our institution, served as process consultants to help sustain an ethical process. Additionally, internal assessment experts were consulted on the survey design used to collect expert feedback.

The other two roles were instrumental in reviewing content and analyzing content feedback. Validation studies, like ours, can use subject matter experts to review and provide feedback on content. The external subject matter experts, or in our case, the content reviewers, helped increase the generalizability of our leadership framework to the field of leadership (Dellinger & Leech, 2007). Content reviewers evaluated content based on pre-determined criteria. Eight content reviewers, racially and ethnically diverse women and men, completed the process. Jerlando Jackson (UW-Madison), Beth Hoag (University of Illinois Urbana-Champaign) and Susan Komives (eminent leadership scholar) were leadership experts in the content reviewer role in stage 1. The five content reviewers who completed the stage 2 review were within and outside higher education: Enrique Alemán, Jr. (Trinity University) and Darren Pierre (Loyola University-Chicago), Lorri J. Santamaría (University of Auckland and Principal Investigator of a community organization) and two leadership experts outside higher education, Deborah Meehan and Steadman Harrison III. Each received an honorarium as compensation for their time and expertise.

The vetting team members evaluated the relevance of the content reviewers’ suggestions regarding the institutional context. They remained open to suggested feedback and engaged in dialog and shared decision-making to ensure revisions remained relevant to campus stakeholders, values and purpose. Students, staff and faculty from diverse campus units and diverse identity groups were selected and participated in the vetting team, including several familiar with and who stewarded the UW-Madison Leadership Framework. CfLI professional staff and student interns also served on the validation and vetting teams.

Create materials, procedures and instruments

Due to the COVID-19 pandemic, materials, communication and procedures supported the mostly virtual format. Background materials were created to orient content reviewers. Feedback instruments were used by content reviewers to record their feedback. Outcome vetting sheets were used by vetting team members to analyze feedback. Agendas and meeting minutes shared plans, outlined the work and highlighted team discussions and decisions. Additionally, internal reports, created by the project leads, summarized the process and results at the culmination of each stage.

Procedures and instruments were used to collect and analyze content in the framework. Descriptive feedback gathered through surveys or interviews in one case study (Shupp et al., 2018) was comparable to our use of surveys and group discussion. We relied on assessment instruments to collect descriptive data, as validation scholars Kane (2006) and Taylor (2013) recommend. Instruments were developed and adapted to collect content reviewers’ feedback on the leadership framework. To analyze feedback, the vetting team used two main procedures: outcome vetting sheets to record comments and meeting notes to report group decisions and rationale. Differences between the stages included the amount of content validated, the number of reviewers and adapted tools and procedures.

Recruit, select and orient teams

Project leads, with the support of other validation team members, recruited and selected process consultants. Process consultants provided instrumental support when recruiting highly regarded leadership experts to serve as content reviewers. The validation team determined selection criteria and conducted outreach to leadership education and leadership development practitioners, inside and outside academia, to validate content. The goal was to have diverse content reviewers who were part of marginalized social identities, including different ethnic and/or racial groups and gender groups. Content reviewers had a stated focus on social justice and were recruited and selected from different job sectors. In stage 1, the plan was to use four experts; three were confirmed and engaged in the process. In stage 2, the validation team considered 25 potential candidates, selected 6 content reviewers and 5 completed the process. Project leads oriented content reviewers through email communication, phone conversations and background documents. This, along with an additional group orientation in stage 2, described framework development and the validation context.

Collect content feedback

Eight subject matter experts independently reviewed the framework, its categories and concepts. Due to the large number of concepts and their interrelated nature, the review process occurred in two periods over several weeks, with two review teams. Project leads collected expert feedback. Content reviewers considered existing and emerging leadership research and three criteria per stage (Table 2).

Moving from inclusivity to cultural responsivity was an organic progression determined by the validation team. Stage 2 content reviewers evaluated outcomes to determine which best represented the intersection and to limit redundancy. Reviewers selected a developmental option, from foundational to advanced, for each outcome.

Reviewers determined the value, quality and significance of each concept and offered general framework perceptions. They uncovered redundancies and suggested clarifications for concept descriptions. Content reviewers individually provided written feedback in the feedback instrument. Across the two stages, they commented on four main categories, 102 subcategories and the overall UW-Madison Leadership Framework.

Analyze content feedback

Vetting team members analyzed content reviewers’ feedback. Members offered experience as those familiar with the UW-Madison Leadership Framework, the campus environment and the criteria. Project leads collected and synthesized feedback, proposed revisions and facilitated group conversation and decision-making processes. Vetting team members reviewed feedback prior to group discussions while considering leadership framework development and the current campus environment.

Meeting minutes revealed divergence, convergence and shared decisions on revisions. Group dialog helped each vetting team reach majority consensus. In stage 1, members met as a group with the project lead to discuss feedback and recommend changes. In stage 2, members analyzed excerpts provided by the project lead while having access to all feedback. The project lead often proposed outcome language and called for votes on each recommendation. Vetting team members had an opportunity to discuss and provide recommendations.

Stage 2 adaptations were useful and more elaborate, due to more concepts, reviewers and feedback. Outcome vetting sheets were used to identify patterns, scope and substantive changes while tracking primary criteria of clarity, cultural responsivity and portability. Majority voting, introduced early in stage 2, helped members reach agreement on recommendations. If 50% or more of the team members present voted no and/or abstained, the conversation continued until majority consensus was reached. Coding each outcome with feedback into one of three levels was added to guide decision-making, indicating complexity, divergence and need for discussion. The stage 2 project lead excerpted feedback, often recommended changes and coded most outcomes.

Finalize revisions for a validated framework

Although discussed separately, this element is well integrated with analyzing content feedback. The project lead positioned proposed changes for final validation at the culmination of each stage. During this step, a few additional revisions were offered. For example, during stage 1, the process consultant and project lead advanced revisions to align competency titles. In another example during stage 2, the project lead suggested adding a new principle focused on cultural responsivity to account for the consistent feedback across many outcomes.

All four roles had the chance to review, revise and ultimately endorse final revisions, ensuring content was deemed valuable. The project lead and process consultant facilitated the sharing of proposed revisions, appreciations and evaluations of the process. In stage 1, this culminating step was organized via email communication, while in stage 2 a virtual final debriefing meeting was organized with content reviewers, the process consultant and validation team members. In addition to content revisions, the process yielded proposed strategic and visual enhancement opportunities.

Findings: revisions to the UW-Madison Leadership Framework

The content review process resulted in validated principles, values, competencies and outcomes. In summary, among the 102 total original concepts, most were revised, deemed relevant and thus validated. In total, 95 validated concepts were accepted into the UW-Madison Leadership Framework. The validated principles, values, competencies and outcomes are available at https://leadership.wisc.edu/leadership-framework.

Content modifications across the four categories of the leadership framework varied in nature (Table 3). Concepts were primarily revised, with only ten removed and one concept added to the leadership framework. Modifications focused on adding clarity, portability, inclusivity or cultural responsivity and were deemed relevant to existing and emerging leadership research and our institutional context.

Modification examples: principles

The three original principle statements were revised to provide fuller expressions, which conveyed more nuance. For example, the fourth principle (leadership for positive change) added “working toward” positive change, instead of only enactment of change, to enhance clarity. Additionally, a new principle statement focused on cultural responsivity emerged during the outcome review, following the validation team’s vetting of content reviewer feedback. Many suggested outcome modifications explicitly addressed inequity and discrimination of marginalized social identities. Given this pattern and the challenges of lengthening some but not all outcomes to account for this feedback, the stage 2 validation team proposed adding a new principle, elevating cultural responsivity and providing all stakeholders an opportunity to discuss, inform and endorse this change.

Modification examples: values and competencies

Revisions to the three original value descriptions offered greater clarity and accuracy. Modifications to the first value, i.e. integrity, were added, reordered and removed redundant concepts for greater clarity. The third competency, learning and development of others, was revised for flexibility for different people or sectors. Revisions created a tighter description, which became more inclusive and portable. Another example demonstrates both revisions and rejected feedback. Modifications made to the decision-making competency description enhanced inclusivity and portability, but the vetting team dismissed a request to remove the competency, deeming the concept essential to individual, organizational and community contexts. Table 4 provides examples of value and competency description modifications.

Original competency titles were inconsistent, offering both static and dynamic framing. The validation team agreed with a process consultant recommendation to align the title syntax for all competency titles. All roles endorsed the change, and the initial verb was removed from four original titles, as outlined below (Table 5).

Modification examples: outcomes

The stage 2 validation process resulted in 71 revised, 10 unchanged and 8 removed outcomes. Outcome examples selected (Table 6) illustrate a range of modifications. Numbered outcomes in the table include the competency-value intersection and the original and validated outcome statements, except for the no revision or removal examples.

Examples illustrate modifications made for greater clarity, cultural responsivity and portability. Overall outcome statements edited for clarity eliminated redundancies, such as when the vetting team edited Outcome 3 to address a reviewer’s request to clarify the meaning of “fact-based.” Outcomes 26 and 29 exemplify cultural responsivity criteria being met. These serve as good examples of feedback that informed adding a new principle along with revisions to these outcome statements. Outcome 26 was changed to add “including communities that are marginalized.” Outcome 29 added “centering and appreciating the contribution of perspectives that are historically silenced.” Outcome 84 exemplifies portability by changing the statement to focus less on the individual and more on working collectively, which extends its applicability.

Moving and removing outcomes were linked for Outcomes 11 and 30. Outcome 11 needed discussion, an additional meeting and voting. Outcome 11 was revised and moved to the intersection for Outcome 30, which the vetting team agreed seemed redundant. Ten outcomes were not modified, such as when feedback was rejected for Outcome 47 to move its intersection and revise its statement or when defining “groupthink” for Outcome 55 instead of removing it.

Strategic opportunities

Finally, external stakeholders who participated offered additional insights beyond the content review. Several strategic opportunities were mentioned that support ongoing validation and strengthen the delivery and impact of the UW-Madison Leadership Framework.

• (1)

  Advance leadership learning across audiences, settings and initiatives;

• (2)

  Develop group and community levels in the UW-Madison Leadership Framework;

• (3)

  Revise further with more accessible and culturally responsive language;

• (4)

  Layer in developmental complexity: outcome continuum from foundational to advanced;

• (5)

  Create culturally responsive validation cycles for ongoing relevance with research and

• (6)

  Enhance visual representations.

Recommendations

We tried something unique - to update and validate our campus-wide leadership framework. Universities are developing leadership frameworks that apply across various settings, and advancements in IHE leadership frameworks are necessary. Our content validation process did not dismiss what we had in place, and it only made the UW-Madison Leadership Framework more relevant and credible in the field of leadership. We would recommend designing an approach based on a few lessons learned. In the content validation process, we learned from what worked well and from the difficulties we faced.

Utilize both internal and external expertise in content validation to offer distinct yet complementary feedback about the general field of leadership and alignment with the institutional context. Campus leadership frameworks developed internally would benefit from inviting external content validation as a rigorous review to ensure validity. Leadership advancements call for broader views of leadership, making cultural responsivity central to our process and worthy of inclusion in other similar efforts. Our four main roles contributed considerably to the process, providing diversity of thought and experiences. Critical facilitation and process reflection amongst the roles is extremely beneficial.

Consider adding conversational approaches. More group conversations, especially between internal and external stakeholders evaluating content and those analyzing their feedback, would be valuable. Written independent feedback was beneficial in controlling bias. However, participants shared a desire for more group conversations to better understand and explain feedback. Clarifying written feedback could inform decision-making and final revisions. Subsequent participant interviews could be added to evaluate the process with all groups, like the stage 2 debrief with content reviewers.

Allow the process to evolve. During and between the stages, the process, tools and procedures were continually adapted. It was invaluable for the validation team to reflect throughout, consulting with participants serving in other roles. Project leads connected with other participants to learn about emergent issues. Due to challenges arising during the pandemic, tools were created to collect written feedback and survey instruments and procedures were adapted. The validation team guided the process, with critical facilitation and implementation from project leads and student notetakers. Adapting vetting team procedures facilitated decisions. Extending and altering time commitments limited involvement from some participants, but it was necessary to thoroughly review all content.

This may not be the exact approach others would use, but this process is adaptable. By engaging in a rigorous and intentional validation process, we have enhanced the content of our leadership framework, ensured its ongoing relevance and deepened our commitment to cultural responsivity in our practices while offering an invitation to the field of leadership studies to follow suit.

Figures

Figure 1

Main roles and stakeholder groups

Table 1

UW-Madison Leadership Framework content categories and descriptions

Categories (#)	Descriptions
Principles (3)	Theoretical foundation of this leadership framework
Values (3)	A set of cultural beliefs or ideals consistent with our institutional history and mission
Competencies (7)	Skills, abilities, or knowledge sets that can be taught or developed. All have indicators that allow them to be observed and measured
Outcomes (89)	Detailed, specific, measurable, identifiable, and meaningful statements about a knowledge, skill or ability

Source(s): https://leadership.wisc.edu

Table 2

Evaluation criteria

Criteria	Descriptions	Stage(s)
Clarity	Coherent and accessible	Stages 1 and 2
Inclusivity	Relevant to all individuals and groups, including those whose social identities are most marginalized	Stage 1
Culturally responsive	Centers and celebrates the perspectives and experiences of all people, especially those whose social identities are most marginalized	Stage 2
Portability	Useful and applicable across a variety of context and settings	Stages 1 and 2

Source(s): https://leadership.wisc.edu

Table 3

Modification summary

Category	Number of concepts	Type of modification
Principles	3	Statements revised
	1	Statement added
Values	3	Descriptions revised
Competencies	4	Titles revised
	5	Descriptions revised
Outcomes	71	Statements revised
	8	Statements removed
	3	Statements moved

Source(s): Authors’ own work

Table 4

Modification examples: values and competencies

Category: concept	Original description	Validated description
Value: integrity	Transparency and truth are central touchstones for integrity. We aim for transparency of information and processes because we believe openness and accessibility facilitate trust, particularly when there are diverse and divergent perspectives on an issue. We hold ourselves accountable to reach decisions through an ethical process and accept responsibility for acting in the interest of all stakeholders	We hold ourselves accountable to acting in accordance with our values, engaging in the constant pursuit of the truth, reaching decisions through an ethical process and committing to care for our entire community. We aim for transparent, reliable, just and equitable processes. Openness and accessibility advance trust, particularly when there are diverse and divergent perspectives on an issue
Competency: learning and development of others	Developing the capacity and engagement of individuals and groups through feedback and coaching	Encouraging opportunities for growth and supporting the learning of individuals and groups through active, responsive and multi-directional feedback
Competency: decision-making	Arriving at decisions that impact others and the organization in which the decisions are made; employing critical and strategic thinking to enable creative solutions to be considered and pursued; recognizing that with important systemic dimensions, analysis and ideas from multiple sources give way to implementation and evaluation	Employing critical, ethical and strategic thinking to consider and pursue creative solutions; assessing the systemic pressures and impact of decisions; providing space for diverse voices and ideas that inform implementation and evaluation

Source(s): https://leadership.wisc.edu

Table 5

Modification examples: competency titles

Category	Original title	Validated title
Competency	Supporting learning and development of others	Learning and development of others
Competency	Honoring context and culture	Context and culture
Competency	Fostering bridge-building and collaboration	Bridge-building and collaboration
Competency	Moving ideas to action	Ideas to action

Source(s): https://leadership.wisc.edu

Table 6

Modification examples: outcomes

Outcome	Intersection (competency and value)	Original outcome	Validated outcome
Outcome 3	Self-awareness and Integrity	Valuing fact-based information	Prioritizing gathering and verifying information
Outcome 11	Self-awareness and Inclusive Engagement	Creating ongoing opportunities for feedback and reflection	Thoughtfully and regularly giving, receiving and reflecting on feedback
Outcome 26	Interpersonal communication and connection and community	Cultivating a network of peers and colleagues in diverse communities	Cultivating a network of peers and colleagues in diverse communities including communities that are marginalized
Outcome 29	Interpersonal communication and connection and community	Openly acknowledging and appreciating the contributions of others	Openly acknowledging, centering and appreciating the contributions of other people, especially those from communities that are marginalized
Outcome 30	Learning and development of others and integrity	Providing and receiving feedback for the purpose of continued learning	Remove
Outcome 47	Context and culture and connection and community	Identifying goals that are meaningful to all	Not modified
Outcome 55	Decision-making and inclusive engagement	Recognizing “groupthink” and encouraging alternatives	Encouraging alternative perspectives when there is perceived pressure to conform with the group
Outcome 84	Ideas to action and inclusive engagement	Collectively defining and communicating a compelling vision that enables others to pursue change	Collectively developing and sharing a compelling vision for the pursuit of change

Source(s): https://leadership.wisc.edu

We are deeply grateful for the many who contributed considerably to our content validation process: Enrique Alemán, Jr., Anna Badamo, Meagan Blair, Katherine Charek Briggs, Amanda Butz, John Dugan, Delaney Egan, Jasmine Enriquez, Donna Freitag, Nancy Graff-Schultz, Jim Gray, Greg Harrington, Steadman Harrison III, So Hee Hyun, Beth Hoag, Jerlando Jackson, Larry Jolón, Nathan Jones, Barb Kautz-Wittwer, Susan Komives, Heidi Lang, Sarah Marty, Deborah Meehan, David Mott, Darren Pierre, Patrick Randolph, Lorri J. Santamaría, Ning Sun, Lauren Surovi and James Yonker. Without their commitment and contributions, our efforts would not have been possible.