Search results

The purpose of this paper is to demonstrate that common method variance, specifically single-source bias, threatens the validity of a university-created student assessment of instructor instrument, suggesting that decisions made from these assessments are inherently flawed or skewed. Single-source bias leads to generalizations about assessments that might influence the ability of raters to separate multiple behaviors of an instructor.

Exploratory factor analysis, nested confirmatory factor analysis and within-and-between analysis are used to assess a university-developed, proprietary student assessment of instructor instrument to determine whether a hypothesized factor structure is identifiable. The instrument was developed over a three-year period by a university-mandated committee.

Findings suggest that common method variance, specifically single-source bias, resulted in the inability to identify hypothesized constructs statistically. Additional information is needed to identify valid instruments and an effective collection method for assessment.

Institutions are not guaranteed valid or useful instruments even if they invest significant time and resources to produce one. Without accurate instrumentation, there is insufficient information to assess constructs for teaching excellence. More valid measurement criteria can result from using multiple methods, altering collection times and educating students to distinguish multiple traits and behaviors of individual instructors more accurately.

This paper documents the three-year development of a university-wide student assessment of instructor instrument and carries development through to examining the psychometric properties and appropriateness of using this instrument to evaluate instructors.

Collegiality is the modus operandi of universities. Collegiality is central to academic freedom and scientific quality. In this way, collegiality also contributes to the good functioning of universities’ contribution to society and democracy. In this concluding paper of the special issue on collegiality, we summarize the main findings and takeaways from our collective studies. We summarize the main challenges and contestations to collegiality and to universities, but also document lines of resistance, activation, and maintenance. We depict varieties of collegiality and conclude by emphasizing that future research needs to be based on an appreciation of this variation. We argue that it is essential to incorporate such a variation-sensitive perspective into discussions on academic freedom and scientific quality and highlight themes surfaced by the different studies that remain under-explored in extant literature: institutional trust, field-level studies of collegiality, and collegiality and communication. Finally, we offer some remarks on methodological and theoretical implications of this research and conclude by summarizing our research agenda in a list of themes.