Search results

1 – 3 of 3
Per page
102050
Citations:
Loading...
Access Restricted. View access options
Article
Publication date: 2 July 2020

Krzysztof Rybinski

This paper aims to analyse the relationship between two measures of university quality, the outcome and other characteristics of a mandatory accreditation and the university…

395

Abstract

Purpose

This paper aims to analyse the relationship between two measures of university quality, the outcome and other characteristics of a mandatory accreditation and the university position in the national ranking.

Design/methodology/approach

Natural language processing (NLP) models are used to calculate the sentiment indicators for 1,850 accreditation reports from the Polish Accreditation Agency. The sentiment indicators, accreditation frequency and outcomes for 203 HEIs are used in correlation analysis, automated linear regressions and quantile regressions with the university position in the Polish Perspektywy rankings as the outcome variable.

Findings

High/low frequency of accreditation visits, excellent/poor accreditation outcomes and low/high frequency of negative inclination words in the accreditation report are followed by high/low university rankings. Quantile regressions reveal that these relationships vary with the quality of the university.

Practical implications

Publishers of university rankings may consider adding the accreditation features to the set of indicators used in such rankings. The machine learning methodology presented allows cross-country inconsistencies to be identified in the approaches used by accreditation agencies in Europe. The authors of the accreditation reports should be aware they can be mined by machine learning models and this should be considered when the reports are drafted.

Originality/value

This is a novel application of NLP models for analysing the relationship between the accreditation and rankings of universities. In other research, the author has applied NLP models to test whether quality assurance agency (QAA) accreditation in the UK can predict how students rate their university on whatuni.com website.

Details

Quality Assurance in Education, vol. 28 no. 3
Type: Research Article
ISSN: 0968-4883

Keywords

Access Restricted. View access options
Article
Publication date: 11 December 2024

Elzbieta Kopciuszewska and Krzysztof Rybinski

This paper aims to investigate the usefulness and validity of student evaluations of teaching (SET) by estimating multiple biases and their cumulative effect, and assessing their…

4

Abstract

Purpose

This paper aims to investigate the usefulness and validity of student evaluations of teaching (SET) by estimating multiple biases and their cumulative effect, and assessing their implications for evaluating teaching effectiveness.

Design/methodology/approach

The study uses a rich dataset from a Polish university and applies linear and quantile regressions to estimate SET biases, including course difficulty, class size and instructor characteristics. The cumulative effect of these biases is measured, and changes during the COVID-19 pandemic are analyzed to assess their impact on SET scores.

Findings

The cumulative SET bias reaches more than one point on a 1–5 Likert scale, challenging the reliability of raw SET scores. Significant asymmetries exist between low and high SET scores. Poor initial evaluations of a teacher predict future low performance ratings, while top-rated teacher contests are often influenced by chance rather than teaching quality.

Practical implications

The findings suggest universities should discontinue using raw SET scores for faculty evaluation and instead implement adjustments for identified biases. This approach will provide a more accurate measure of teaching performance.

Originality/value

This paper builds on earlier studies that applied econometric frameworks to analyze SET bias predictors and offers a novel, comprehensive assessment of cumulative SET biases and their asymmetries. It is the first to evaluate the effects of multiple SET biases within a single model and the first to document how SET biases intensified during the pandemic, emphasizing the need for significant reform in teaching evaluation practices.

Details

Quality Assurance in Education, vol. ahead-of-print no. ahead-of-print
Type: Research Article
ISSN: 0968-4883

Keywords

Access Restricted. View access options
Article
Publication date: 7 April 2023

Krzysztof Rybinski

This paper aims to investigate the relationship between student emotions, professors' performance and course ratings and difficulty.

260

Abstract

Purpose

This paper aims to investigate the relationship between student emotions, professors' performance and course ratings and difficulty.

Design/methodology/approach

Natural language processing models are used to extract six basic emotions and several categories of professors' harmful performance from nearly one million student reviews randomly selected from the website ratemyprofessors.com. These features are used in regression analysis to analyse their relationship with numerical ratings of course quality and course difficulty.

Findings

Negative emotions and bad performance by professors are detected more often for low-rated courses and courses perceived as more difficult by students. Positive emotions are seen for highly rated and less challenging courses.

Practical implications

This paper shows that natural language processing tools can be used to enhance and strengthen the quality assurance processes at universities. The proposed methods can improve the often-contested student evaluation of teaching practices, help students make better and more informed choices about their courses and assist instructors to better tailor their teaching approaches and create a more positive learning environment for their students.

Originality/value

This paper presents a novel analysis of how student emotions and poor performance by professors, derived automatically from teacher evaluations by students, affect course ratings. Results also lead to a novel hypothesis that the student–course emotional match or student tolerance of bad behaviour by professors can affect the performance of students and their chances of completing their degree.

Details

Quality Assurance in Education, vol. 31 no. 3
Type: Research Article
ISSN: 0968-4883

Keywords

1 – 3 of 3
Per page
102050