Digital proctoring in higher education: a systematic literature review

Abstract

Purpose

To improve the academic integrity of online examinations, digital proctoring systems have recently been implemented in higher education institutions (HEIs). The paper aims to understand how digital proctoring has been practised in higher education (HE) and proposes future research directions for studying digital proctoring in HE.

Design/methodology/approach

A systematic literature review was conducted. The PRISMA procedure was adapted for the literature search. The topics were identified by topic modelling techniques from 154 relevant publications in seven databases.

Findings

Seven widely discussed topics in literature were identified, including solutions for detecting cheating and student authentication, challenges/issues of uptakes and students' performance in different proctoring environments.

Research limitations/implications

This paper provides insights for academics, policymakers, practitioners and students to understand the implementation of digital proctoring in academia, its adoption by HEIs, impacts on students' and educators' performance and the rapid increase in its use for digital exams in HEIs, with particular emphasis on the impacts of the systems on digitalising examinations in HE.

Originality/value

This review paper has systematically and critically described the state-of-the-art literature on digital proctoring in HE and provides useful insights and implications for future research on digital proctoring, and how academic integrity in online examinations can be enhanced, along with digitalising HE.

Citation

Han, S., Nikou, S. and Yilma Ayele, W. (2024), "Digital proctoring in higher education: a systematic literature review", International Journal of Educational Management, Vol. 38 No. 1, pp. 265-285. https://doi.org/10.1108/IJEM-12-2022-0522

1. Introduction

Digitalisation of higher education (HE) has been bolstered by the increasing availability of the internet and personal computers across the globe (Ugur, 2020). As a result, higher education institutions (HEIs) have undergone transformations in their teaching, learning and examination practices, offering courses online through learning management systems (LMS) (Muzaffar et al., 2021). This shift has provided HEIs with a strong foundation to deliver quality education to a broader student population, breaking free from geographical constraints, institutional boundaries and temporal differences. This has been exemplified through the development of Massive Open Online Courses (MOOCs) (Peters and Jandrić, 2018). However, the rapid digitalisation of HEIs has given rise to several critical challenges, such as maintaining academic integrity and security, and preventing dishonesty and cheating in online distance examinations (Pavlou et al., 2008). The emergence of the COVID-19 pandemic has further accentuated the need for new approaches to handle remote assessment and examination, disrupting both campus-based and online education.

To tackle these challenges, HEIs have rapidly developed and adopted digital proctoring systems (Nigam et al., 2021). Since its development, there has been a significant increase in the demand and usage of digital proctoring systems (e.g. Raman et al., 2021). Digital proctoring refers to the utilisation of digital tools and technologies to ensure that students (exam-takers) adhere to the prescribed guidelines and policies during examinations and assessments (Alessio et al., 2017; Udechukwu, 2020).

Literature in this domain shows that the rationale for investigating digital proctoring in HE is based on several compelling factors. Firstly, the preservation of academic integrity in an online environment is a paramount concern for HEIs (Pavlou et al., 2008). As literature shows, the transition of traditional face-to-face assessments to online formats is becoming a common practice in HE environment (Besser et al., 2022). Therefore, ensuring the authenticity of evaluations becomes increasingly challenging. However, digital proctoring, with its utilisation of advanced algorithms and surveillance technologies, offers a potential solution by deterring cheating and plagiarism, thus safeguarding academic integrity in online examinations (Alessio et al., 2017).

Secondly, digital proctoring systems are changing the practices of examinations and assessments in HE, which raise concerns amongst stakeholders, e.g. students and educators regarding their adoption and performance under proctored online environments (e.g. Andreou et al., 2021). HEIs endeavour to create open learning environments for all students, regardless of their geographical location or socioeconomic background, through initiatives like Massive Open Online Courses (MOOCs). To maintain openness and ensure equal access to knowledge, it is crucial for HEIs to address any disparities that may arise due to varying levels of supervision, access to resources and individual concerns. This approach is essential for advancing fair and equitable assessment practices (e.g. González-González et al., 2020).

Thirdly, the implementation of digital proctoring must be accompanied by careful consideration of relevant issues, e.g. ethical implications, privacy concerns and security issues. Continuous surveillance and monitoring during online exams have the potential to infringe upon students' privacy rights and generate feelings of discomfort (e.g. Conijn et al., 2022). Moreover, the presence of biases within monitoring algorithms can disproportionately impact specific student populations (e.g. Udechukwu, 2020). Therefore, HEIs and policymakers must pay heightened attention to the potential negative impacts of digital proctoring on students' human rights, prompting critical evaluation of the long-term usage of these systems.

Overall, the literature on digital exam proctoring encompasses a range of research studies. The various perspectives discussed in the literature on digital exam proctoring contribute to a comprehensive understanding of the topic, encompassing viewpoints from educators, students, technologists, policymakers and researchers. These diverse perspectives shed light on the benefits, challenges, ethical considerations and effectiveness of digital exam proctoring solutions in higher education contexts.

The primary objective of this research is to investigate the current state of digital proctoring in higher education settings and its implications for policy and practice. By conducting a comprehensive literature review on relevant topics related to digital proctoring, we aim to offer valuable insights into the challenges, benefits and potential best practices associated with its implementation. This focus on topics within the literature review allows us to explore various dimensions, such as the effectiveness of different proctoring methods, ethical considerations, student experiences and implications issues and challenges. By examining literature, our research strives to inform policymakers, educators and HEIs about the potential impact of digital proctoring on teaching, learning and assessment practices. To achieve this aim, we formulate the following research questions.

1. What is the state-of-the-art research on digital proctoring in higher education?

2. How has this phenomenon been studied in the literature?

To answer these research questions, we conduct a systematic literature review (SLR), searching relevant databases for peer-reviewed scientific publications (e.g. journal articles and conference papers) on digital proctoring, including ACM, ERIC, IEEE, ProQuest Central, PubMed, Scopus and Web of Science. We employ topic modelling methods to analyse the final dataset, which comprises 154 scientific peer-reviewed studies. The results of our analysis reveal three distinct categories encompassing seven key topics discussed in the literature. These categories include the technological advancements of digital proctoring systems, stakeholders' concerns regarding the adoption of digital proctoring and the issues and challenges that have emerged in implementing these systems.

This paper provides two main contributions. Firstly, the review results offer valuable insights into the research on digital proctoring, which plays a crucial role in improving academic integrity in online examinations and advancing the digitalisation of HE. Secondly, it guides academic practitioners and other stakeholders in HEIs on the implementation of digital proctoring systems and the assessment mechanisms required to evaluate their impacts on students and educators' performance. This knowledge will serve as a foundation for academics, policymakers and students to understand the implications of digital proctoring systems. Moreover, it will stimulate discussions regarding the continued usage of these systems and their long-term influence on the digital transformation of HE.

The paper is structured as follows: In Section 2, we provide a concise overview of the development of digital proctoring in higher education. Section 3 outlines the literature review method employed in this paper. We then present a summary of the results. Finally, we engage in discussions of the key results and propose future research directions.

2. Digital proctoring

Digital proctoring, also known as e-proctoring, virtual proctoring, remote proctoring, or online proctoring, is a process that utilises digital tools and technology to ensure that exam-takers adhere to prescribed guidelines and policies (Udechukwu, 2020). Allan (2020) defines an online exam as “high-stakes summative assessment events, mediated by digital technologies, which takes place in a defined place or time, and conducted under restrictions of access to course materials, notes or communication and, taking place in a secure condition such as invigilation” (Allan, 2020, p. 1). It serves as a mechanism to verify the authenticity and authorship of exams, while also preventing and detecting any unauthorised or unacceptable activities during online assessments (Udechukwu, 2020).

With the rise of MOOCs and open education, digital proctoring has facilitated the administration of remote exams without requiring in-person invigilation (González-González et al., 2020). Digital proctoring essentially can be classified into three main categories: Live Proctoring (LP), Recorded Proctoring (RP) and Automated Proctoring (AP) (Arnò et al., 2021; Nigam et al., 2021). Each category differs in its technical features and implementation. For example, LP involves the presence of a human invigilator who authenticates exam-takers and monitors their activities via screen sharing. One invigilator can oversee the activities of 10–12 exam-takers on a single screen, requiring additional invigilators for larger groups. In RP, no human invigilator is present during the exam. Instead, the behaviour of students is recorded for later review by invigilators to identify potential instances of cheating or misconduct. However, this process can be time-consuming, and students may contest proctoring decisions, leading to complaints. On the other hand, AP utilises artificial intelligence (AI) to monitor exams in near real-time. In AP records and analyses students' behaviour using audio-video analysis and automatically detects cheating. If cheating is detected, the exam may be paused or terminated. AP can be used in conjunction with LP or RP methods (Arnò et al., 2021; Duncan and Joyner, 2022).

Furthermore, existing literature demonstrates that several studies have addressed operational and technological concerns of various digital proctoring programmes in the context of maintaining academic integrity and preventing cheating, for instance, D'Souza and Siegfeldt (2017) as well as Ullah et al. (2019). Additionally, Karim and Shukur (2016) have examined the challenges associated with utilising technology and software effectively and efficiently for conducting digital exams. Berggren et al. (2015) have explored students' perceptions of online exams, noting their apparent preference for typing over writing exam responses. Moreover, James (2016) have conducted studies on students' experience of anxiety when using digital technologies during online exams.

Arnò et al. (2021) conducted a comprehensive review of 29 proctoring systems available in the market and commonly used in HEIs. Their study exemplifies the development and uptakes of the systems. Nigam et al. (2021) focussed on reviewing AI-related features widely employed in various digital proctoring systems. These technical advancements have significantly improved the accuracy of detecting misconduct during online exams, thereby enhancing academic integrity and security. The adoption of digital proctoring systems has become a global phenomenon, especially accelerated by the COVID-19 pandemic. HEIs in the United States and Europe have embraced proctoring systems such as Examity, ProctorU, Proctorio, Proctortrack and Inspera Exam Portal (Han et al., 2022). In Australia and New Zealand, ProctorU and Inspera are commonly used, while Examsoft and RPNow are prevalent in Asian countries like Singapore due to the campus closures prompted by the pandemic (Arnò et al., 2021; Swauger, 2020).

3. Method

3.1 Selecting review papers using the PRISMA approach

Literature reviews play a crucial role in scientific inquiry and knowledge accumulation, as noted by Webster and Watson (2002). Literature on digital proctoring was searched and selected from seven databases, including ACM, ERIC, IEEE, ProQuest Central, PubMed, Scopus and Web of Science. The search terms and retrieval process started with three main keywords: “proctoring”, “education type” and “type of exam”. Based on these three keywords, we decided the final search string would be:

(“Digital Proctoring” OR “Online Proctoring” OR “Online Exam Supervision” OR “Remote Proctoring” OR “Automated Proctoring” OR “e-Proctoring” OR “Proctoring Systems”) AND (“Higher Education” OR “University” OR “College” OR “Institute”) AND (“e-Exam” OR “Digital Assessment” OR “e-Assessment” OR “Formal Assessment” OR “Exams” OR “Online Testing” OR “Assessment”)

In the literature search process, we followed the guidelines of the PRISMA process (Page et al., 2021). The database search was conducted in October 2022, and 425 studies were found, ACM (38), ERIC (10), IEEE (44), ProQuest Central, (131), PubMed (112), Scopus (59) and Web of Science (31). After removing the duplicates (n = 71), 354 studies remained. We followed the inclusion and exclusion criteria to screen for the most relevant studies. The inclusion criteria were: (1) the article must study digital proctoring, e.g. online or remote proctoring, automated proctor, (2) the article must study digital proctoring within the HE context, either in universities or colleges and (3) the proctoring system is used for online examination or assessment purposes. We excluded papers that were: (1) not written in English, (2) not peer-reviewed and (3) grey literature published in newspapers or magazines. We manually checked the abstracts of these 354 articles, and 154 articles were considered relevant and included in the final analysis. To ensure the quality of the selected articles, we followed a rigorous process. The PRISMA diagram of the literature search and review process is shown in Figure 1.

The quality appraisal process for including the papers in the final review consisted of three steps. First, the articles must be peer reviewed, which is widely accepted as a measure of quality in academic research. We selected a combination of articles from conferences, journals, books and book chapters, as well as empirical, review and conceptual studies to ensure comprehensive coverage of the topic. Second, the studies were published by reputable and reliable sources and publishers, such as Springer, Emerald, Elsevier, ERIC or conference proceedings published by ACM, or IEEE. Third, the abstracts of these studies were required to contain sufficient information about the research aim, questions, research methods and results necessary for the topic modelling analysis.

Topic modelling is one of the methodological approaches used in text mining to find recurring themes (topics) in the text corpora. Probabilistic topic models, for example, Latent Dirichlet allocation (LDA) (Blei et al., 2003), are algorithms that can identify topics and assign a document to a topic by relating the co-occurrences of words, which are important in defining their meaning and the meaning of topics (DiMaggio et al., 2013). The LDA allows documents to be assigned to multiple topics, with varying degrees of probability associated with the topics. By following the recommendations of Debortoli et al. (2016) and Schmiedel et al. (2019), we choose to use the MineMyText (www.minemytext.com) cloud service to run the LDA analysis of our data sets (154 articles).

3.2 Analysing the selected studies by topic modelling methods

The quantitative analysis of the 154 articles was performed in three steps. First, we prepared and cleaned the data for topic modelling analysis by using minemytext.com. We exported the bibliographic data from Zotero to a single.csv file which included “date”, “publication year of the papers” and “text” (Abstract).

We tokenised the documents by using 2-g to produce strings. For example, to create “COVID and 19” as one word, which is “COVID19”. We removed stop words such as “IEEE”, “ACM” and “SPRINGER” and “COPYRIGHT”. We also considered “find, also, provide, course, study, take, many, high” as stop words since they are the most frequently used words in an abstract. Standard stop words such as “remove HTML tags” and “remove numbers” were also selected. We chose lemmatisation for analysis as it considers context and converts words to their meaningful base form, accounting for different word forms. However, we did not select the “stemming” option to analyse the dataset since stemming removes the last few characters of a word. In addition to that “noun”, “verbs”, “adjectives” and “adverbs” were selected as part of speech filtering to ensure the text corpus only retained those parts of speech that were important to the topic models. Second, we computed the optimal numbers of topics by computing the coherence score based on the algorithm by Röder et al. (2015), using the Python Gensim library (see Figure 2).

The coherence score measures the internal coherence and validity of a topic based on its semantic interpretability (O'Callaghan et al., 2015). A high coherence score indicates more interpretable topics, so we used the highest score to determine the number of topics. The results indicated the optimal number was seven. Third, we ran the topic modelling analysis with seven topics (as shown in Figure 2), using the LDA algorithm. To interpret the meanings of the topics, we qualitatively re-examined the words and documents that were highly related to each topic.

We coded meanings using two criteria: representative terms building a meaningful topic and abstracts closely related to the topic, with consensus reached through team discussions. During coding, we focussed on the relevance and exclusivity of topics and labelled them with descriptive names, following the method suggested by Blei et al. (2003). We relied on our domain knowledge and judgement for topic labelling (Schmiedel et al., 2019), ensuring reliability and validity in our qualitative analysis.

4. Results

The initial publication on digital proctoring dates back to 2008 (Pavlou et al., 2008). From 2008 to 2019, there were relatively few studies published each year on this topic, with the exception of 2010 when no studies were published. A sharp increase in publications was seen in response to the pandemic and campus closures between 2019 and 2021, followed by a decrease in 2022 (as shown in Figure 3). The 154 reviewed studies were published across natural sciences, medical sciences, social sciences and humanities. Figure 4 shows the most commonly used words in the corpus, including online, student, exam, proctor, use, test and assessments.

The literature encompasses various digital proctoring programs, including LP (e.g. Vazquez et al., 2021), RP (e.g. Davis et al., 2016; Lewis, 2020) and AP (e.g. Migut et al., 2018), as well as combined programs such as LP + AP, RP + AP, or LP + AP + RP, which have been extensively studied (e.g. Karim et al., 2014). Furthermore, there are papers that explore the development and examination of AI features to enhance digital proctoring systems (e.g. Jia and He, 2021). In addition to specific program-focussed studies, there are also papers that provide a broader perspective on digital proctoring in general (e.g. Fask et al., 2014; Kharbat and Daabes, 2021). Overall, the existing literature offers valuable insights into the different digital proctoring programs, their effectiveness and the emerging issues.

4.1 Topic description

Table 1 shows the seven topics obtained from the analysis. The most studied topics are Topic 4: students' performance in different proctoring environments (19%), and Topic 1: solutions for detecting cheats (17.4%).

To provide a better overview of the current research on digital proctoring in higher education (HE), we have grouped the seven topics into three categories.

1. Technological advancements for maintaining academic integrity in online examinations: This category encompasses Topic 1 and Topic 2, which focus on the technological advancements in digital proctoring.

2. Stakeholders' concerns (e.g. Students, Educators and HEIs) of using digital proctoring for online examinations and assessments: This category delves into the concerns raised by students, educators and HEIs regarding the new assessment practices in the online proctored environment. It includes topics such as students and educators' performance, and the impact of the pandemic on the uptake of digital proctoring discussed in Topics 3, and 4.

3. Issues and challenges that emerged in digital proctoring implementation: This category encompasses Topic 5, 6 and Topic 7, which shed light on the issues that arise during the implementation of digital proctoring systems.

In the following, we describe the categories and their associated topics.

4.1.1 Category 1: advances in proctoring technologies for maintaining academic integrity in online examinations

This category includes topic 1: Solutions for detecting cheats, and topic 2: systems for students' authentication.

1. Topic 1: solutions for detecting cheats

   This topic focussed on the technical solutions that were developed to detect students' cheating behaviours in a digital online (remote) examination. The solutions feature systems to capture audio and video data of exam-takers and their surroundings, as well as to analyse the captured data automatically for detecting suspicious behaviours. Prathish et al. (2016) developed a multi-model system using a webcam to capture audio and video in addition to active window capture. The results of the experiment showed that with such a system, misconduct behaviour can be detected via yaw angle variations, audio presence and active window capture. Madhusudan et al. (2022) also proposed a multimedia exam proctoring system based on face recognition and object capturing systems by making use of a webcam for face landmark detection.

   Automatic and collaborative approaches were also proposed to detect cheating behaviours in online exams. For example, Li et al. (2015) developed and proposed a Massive Open Online Proctoring framework, which consisted of three components: an Automatic Cheating Detector (ACD), a Peer Cheating Detector (PCD) and a Final Review Committee (FRC). In support of webcam video or other sensors, ACD monitors students' activities during the exam and automatically flags suspected cheating behaviour. In the event of abnormal behaviour, the information is sent to the PCD together with students' peer-review flagged webcam video to confirm suspicious cheating behaviours. All incidents of suspicious cheating behaviours are sent to the FRC to make the final punishment decision. Additionally, Duhaim et al. (2021) developed a recommendation system to analyse the students' answers to detect similarities and limit the extent of unintended student collaboration. Saba et al. (2021) concluded that these automatic solutions for detecting cheats should be more recognised and used in online exams for supporting invigilating tasks.

2. Topic 2: systems for students' authentication

   The studies on this topic discuss various aspects of technology integration and security mechanism in ensuring students' authentication (authorships). For example, Peytcheva-Forsyth et al. (2019) and Mellar et al. (2018) proposed an adaptive trust-based e-assessment system that can be used for authenticating the authorship of the student in the online exam assessment environment. In the same vein, Guillén-Gámez (2017) suggested that to verify the identity and to avoid or minimise academic fraud, HEIs need to utilise facial authentication software in the learning and teaching process when using online learning platforms.

   Muzaffar et al. (2021) emphasise that the verification and detection of examinees' abnormal behaviour are very significant characteristics in online exam environments. The identification has two main options: static and continuous verification. The online exam taker's identity is only confirmed once during static verification at the start of an online exam, while continuous verification is the process of continuously authenticating and verifying the examinee throughout the online exam. Similarly, Zhu and Cao (2021) proposed a biometric authentication and blockchain-based online examination scheme that is superior to the existing schemes in the experiment. In order to provide identity and authorship verification, Labayen et al. (2021) also proposed using multimodal biometrics technology in digital proctoring to conduct student authentication.

4.1.2 Category 2: stakeholders' concerns of using digital proctoring

1. This category includes the topic 3 and topic 4 that showcase the stakeholders concerns.

2. Topic 3: staff and students' perceptions during the pandemic

   This topic discussed the experience and students' perceptions of digital proctoring during COVID-19, which demanded the immediate transition from face-to-face teaching and learning to virtual environments. The pandemic has had disruptive impacts on HE, especially on online examinations (Itani et al., 2022). Cygan and Bejster (2021) studied the faculty's response to the transition and emphasised that more emotional support for the students and course re-organisation were important transition facilitators. The authors further pointed out that educators should learn from the emergency transition to capitalise on successes and mitigate challenges moving forward. In studies of Australian universities, Reedy et al. (2021) discussed different perceptions of staff and students towards digital proctoring, including how to redefine academic misconduct and emphasise the importance of ensuring exam integrity in online exams.

   Students' perceptions of digital proctoring are mixed. Kharbat and Daabes (2021) explored the experience of students' attitudes and concerns using an e-proctoring tool in their final exams. The authors found that students were mostly concerned with privacy and various environmental and psychological factors in relation to online exam platforms. The findings show that many of the students who were surveyed expressed their satisfaction with the digital proctoring platforms, while many also indicated they did not have a good perception of the systems, and they mentioned three main reasons for not being satisfied with online exam platforms, namely (1) that they took too long to set up, (2) technical difficulties and (3) personnel issues with proctors. Patael et al. (2022) and Elsalem et al. (2021), also studied the users' perception of digital proctoring and found that students rated the experience as “good”, but many also indicated that they preferred formative assessments as they faced pedagogical challenges such as pandemic-related stress, and online assessment challenges. Khalil et al. (2022) stated that students were concerned about privacy in terms of data protection and transparency.

3. Topic 4: students' performance in different proctoring environments

   This topic analysed students' performance in different proctoring environments and in proctored vs non-proctored online environments. Hylton et al. (2016) found no significant difference in exam scores between proctored and non-proctored groups, with slightly higher scores in the non-proctored group. Alessio et al. (2017) found that students scored 17 points lower and used less time in proctored online tests. Similar results were found in Alessio et al. (2018) and Alessio and Messinger (2021), with lower exam performance and shorter exam time with proctoring, as well as increased compliance with academic integrity. Goedl and Malla (2020) found unequal grade distributions in proctored and un-proctored exams and advised caution in using them to maintain grade equivalency. Baso (2022) reported lower scores in the proctored online system and found no influence of class or gender on performance.

   Lee (2020) and Andreou et al. (2021) found no difference in students' performance in online and offline proctored exams and asserted that the exam proctoring environment is unlikely to be related to student performance. Moreover, van Halem et al. (2021) found students preferred traditional exams six times more due to technical challenges and they showed lower uptake in proctored exams even after removing impediments, with student traits (characteristics) not linked to preferred method. Howard (2019) found similar exam scores for un-proctored exams, proctored in a testing centre and proctored online using the software, but un-proctored online students spent more time on exams. Hall et al. (2021, 2022) found that online proctoring had little effect on exam performance and was useful for protecting academic integrity and test security in distance learning. Additionally, Hall et al. (2022) study showed no consistent increase or decline in exam results with remote proctoring, even considering student GPA and test anxiety (p. 92). Woldeab and Brothen (2019) conducted a survey amongst 631 students and found that high test anxiety led to lower exam scores, especially in an online proctored setting. Law et al. (2020) found online proctoring led to greater engagement with course materials and improved exam performance, but formative assessments remained consistent. Reisenwitz (2020) study supports the use of proctored online exams to fairly assess student performance.

4.1.3 Category 3: issues and challenges emerged in the implementation

1. This category consists of three topics. Topic 5 discusses the updates and issues related to transitions to digital proctoring. Topic 6 focusses on the problems associated with proctoring in online exams. Finally, topic 7 addresses the issues of MOOCs and remote proctoring.

2. Topic 5: transitions to digital proctoring systems: uptakes and issues

   Digital proctoring systems are transforming assessment practice in universities. The pandemic has increased the uptake of the systems by HEIs. Ali et al. (2022) found that most of the providers of such systems reported experience in delivering high-quality products to the educational institutions. However, the authors suggested that educational institutions must make an informed decision and identify their explicit needs when deciding and choosing the most appropriate digital assessment platforms. Moreover, Du Plessis et al. (2021) stated that the COVID-19 pandemic has forced universities worldwide to immediately transit to distance learning and indicated the need for a web-based assessment.

   Some studies have discussed the issues and challenges of the use of digital proctoring systems. Pettit et al. (2021) mentioned that the immediate need to transition to digital assessment and digital exam proctoring has helped the providers of online examination software and remote proctoring platforms overcome some of the challenges such as students' authentication, cheating prevention, cybersecurity and IT failure. Selwyn et al. (2021) argued that universities faced several critical issues underpinning the adoption of digital proctoring platforms such as the surrender of control, the hidden labour required to sustain automated systems and the increased vulnerabilities of remote studying. Some authors also critically reflected on the negative consequences of digital proctoring (Conijn et al., 2022; Duncan and Joyner, 2022; Lee and Fanguy, 2022). A few studies (Sapawi, 2021; Vegendla and Sindre, 2019) attempted to provide and develop a methodology for administrators to assess the security and privacy risks of these platforms or considerations concerning issues such as the validity and reliability of the alternative and online assessment, the integrity of assessments and deterring plagiarism and cheating (Coghlan et al., 2021).

3. Topic 6: proctoring problems in online exams

   This topic discusses the proctoring problems of digital online exams. Hearn Moore et al. (2017) identified 10 different types of problems, for example, identifying the test taker, preventing the theft of the exam, combating the unauthorised use of textbooks and/or notes and preventing the use of mobile phones and hand-held calculators. Sasikala et al. (2022) also identified some of the major forms of malpractice during an online exam – such as (1) candidates switching between tabs of the exam and browser window, (2) changing position relative to the webcam and (3) the use of mobile devices.

   Several studies have proposed solutions to address the integrity problems. Haus et al. (2020) suggested that when a limited number of students are taking the online exam, direct monitoring by the teacher can be a feasible approach. However, when a larger number of students are taking the online exam, a proctoring system should be utilised. Cleophas et al. (2021) suggested data-driven techniques to analyse exam event logs and essay-form answers to reveal patterns of student collusion. Smith (2021) highlighted the importance of creating a supervised environment for online exams, where students' smartphones and devices are directly used to fully capture their workspace while taking the test. Notably, Fawns and Schaepkens (2022) argued that is necessary to improve trust in proctored online exams.

4. Topic 7: online teaching programmes (MOOCs) and remote proctoring issues

   The topic discusses the integration of digital proctoring to handle certificate valuation in MOOCs and online programmes. Maas et al. (2014) studied how Coursera, one of the largest MOOCs providers, verified students' credentials as the record of their performance by using biometrics methods. Staubitz et al. (2016) compared online proctoring and the current practices of MOOC platforms and concluded that online proctoring seemed to be a suitable way to verify the students. However, remote proctoring issues have also been raised. The “Telexetasis” system proposed by Pavlou et al. (2008) discussed the problems that e-examinations posed in open and distance learning, such as impersonation, collaboration and cheating. Schoenmakers and Wens (2021) discussed the fraud problems and proposed a web-based supervisor app for tracking and tracing students' behaviour. Karim et al. (2014) studied the problems of remote proctoring for Internet-based tests, issues such as perceived tensions, the invasion of privacy and negative reactions from the students. Rodchua et al. (2011), Li et al. (2015), also argued that the failure of HEIs to provide secure and reliable exams in distance education has resulted in the slow development of online programmes and MOOCs (Rodchua et al., 2011; Li et al., 2015).

5. Discussion

This paper provides a comprehensive analysis of the research on digital proctoring, spanning different disciplines and study areas. We aim to explore this phenomenon from multiple perspectives, setting it apart from previous literature reviews such as Arnò et al. (2021) and Nigam et al. (2021). Our review is distinguished by its broad focus and deliberate effort to locate relevant literature. Moreover, we utilised topic modelling analysis, which enabled us to identify seven distinct topics from the 154 articles reviewed. These topics are then grouped into three categories, i.e. technological advancements, stakeholders concerns and issues emerged in implementation. Furthermore, we delve into the research contributions, implications, future research directions and limitations.

5.1 Contributions to understanding digital proctoring in higher education

The review shed light on the innovative approaches and tools that ensure academic integrity in online exams. By exploring topics such as advanced algorithms and surveillance technologies, researchers aim to develop robust proctoring systems that can effectively detect and deter cheating and plagiarism. The findings demonstrate that the technical features of digital proctoring systems have improved, in tandem with both the innovations of education technology, as well as with advancements of AI technologies for detecting misconduct and malpractice in online examinations. Digital proctoring systems can, in general, provide a secure environment for online examinations, and can improve academic integrity with appropriate accuracy. However, technological advancements in digital proctoring systems are necessary and critical to overcoming the underlying challenges of the online exam, such as cheating. Nevertheless, no current proctoring systems can fully prevent and detect malpractices in online distance examination environments.

The review highlights stakeholders' concerns regarding the use of proctoring systems, with a particular focus on the perspective of students (exam-takers). The findings of this research showed that the exam-takers have generally positive attitudes towards the proctoring systems. The review results show evidence that the use of proctored online exams did not significantly influence, except for a few cases (e.g. Wuthisatian, 2020) students” academic performance in comparison to in-person or non-proctored examinations. We also found that the approach towards the online exam did not significantly improve the performance of students (e.g. van Halem et al., 2021). This may largely be due to the fact that the students were first-time users of proctoring systems and this situation produced technostress and anxiety during the exam (e.g. Woldeab and Brothen, 2019). These issues can be mitigated after the students gain experience in having this type of proctoring for their examinations.

The review critically examined ethical considerations, privacy concerns and security issues associated with continuous surveillance and monitoring. By identifying and addressing these issues, HEIs should mitigate any potential negative impacts. For example, on students' privacy rights, and prevent the presence of biases within monitoring algorithms that may disproportionately affect specific student populations. The review also investigated the perceptions, experiences and challenges faced by various stakeholders, aiming to address disparities and ensure equal access to knowledge and opportunities for all students. The review results point out the legal, security, privacy and ethical issues of using digital proctoring systems in HEIs, which need to be fully understood and addressed by all stakeholders and decision-makers. Otherwise, these issues, especially the legal issues, may limit and hinder the use of the systems (Slusky, 2020). The literature review demonstrates that in developing and enhancing the technical features of the proctoring systems, more AI and deep learning techniques should be used. However, the technological development of AI, such as face recognition, has posed threats to students' fundamental human rights (privacy, data prevention, etc.), ethics and privacy (Coghlan et al., 2021; Yaqub et al., 2022).

5.2 Practical implications

The review results provide several practical implications as well. For example, the findings show that it is necessary that educators, instructional designers, policymakers and decision-makers at HEIs understand the challenges and opportunities of online digital exams supported by digital proctoring systems. As one of the most recent methods of students' performance assessment, it is necessary to understand how the technology works and why such tools should be used, and then to carefully decide which solutions to adopt.

Though there are numerous challenges to implementing the proctoring systems, HEIs have accumulated valuable knowledge and practices, especially during the pandemic period. This research suggests that HE needs to develop an institutional approach for proctored online exams and define new principles for preventing and detecting misconduct. New examination practices supported by proctoring systems need to be advocated. Sufficient support and training for teachers and study administrators are needed, and arguably must be provided before implementing the digital exam proctoring systems. Students also need to be informed and prepared for the new examination practice. Supported by potential technical advances in proctoring systems, HEIs can enhance their control over the online distance examination environments, and thus improve the academic integrity of exams. This will be valuable for the institutions to develop and improve more online programmes and MOOCS for disseminating knowledge to more people (see Topic 7).

We argue that the implementation and the use of digital proctoring systems for online examinations is a crucial process for digitalising HE. The review results support our argument and show the insights gained from the digitalisation practices across all academic disciplines, though most of the usage of the digital proctoring systems was forced by the emergency disruption of traditional education due to the COVID-19 pandemic. However, we argue that the practices developed during the pandemic are valuable and can serve as departure points to continue using proctoring systems to improve the academic integrity of online examinations even after the universities return to normal “campus” education. Vermunt (2007) has strongly suggested that the ultimate goal of HE should be to help students to prepare for lifelong and self-regulated learning. This is also one of the United Nations' sustainability development goals (SDG #4), i.e. to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. We also assert that such lifelong learning should not be restricted by spatial-temporal restrictions and institutional boundaries when the world population has increased accessibility to the Internet and personal computers. We think it is crucial for HEIs to provide equal access to knowledge for everyone across the globe, and this equality should not be limited or hindered by the challenges of online examinations in comparison with traditional physical examinations.

However, we are also aware that the potential negative impacts of digital proctoring cannot be predicted with certainty. The review results highlight stakeholders' concerns regarding the protection of student data and privacy, the potential for bias or unfair treatment of different student populations in AI-powered assessment and monitoring programmes, and HEIs” lack of expertise in effectively responding to crises, such as the COVID pandemic. HEIs must proactively address these impacts by complying with relevant regulations and ethical guidelines to ensure these systems' fair and responsible implementation in the long run. In order to achieve this, HEIs can collaborate with regulatory bodies, privacy advocates and educational technology experts. Through dialogue and consultation, institutions can gain valuable insights and guidance on best practices for implementing digital proctoring systems. This collaborative approach will help HEIs strike a balance between enhancing academic integrity and respecting the rights and privacy of students and other stakeholders. By doing so, HEIs can sustain the implementation of digital proctoring for further advancing practices in online examinations and be well-prepared for any unforeseen crises that may arise in the future.

5.3 Future research

Depart from the review results, we propose future research directions to further advance our understanding of this research focus. The first important area for future research is the technological development of digital proctoring systems and the effects on online education practices. Comparative analysis studies can be conducted to examine the effectiveness and efficiency of different types of proctoring systems. Additionally, exploring effective pedagogical strategies for integrating digital proctoring into online teaching and assessment practices is crucial. This research can help instructors design proctored assessments. Future research can also focus more on the relationships between the use of digital proctoring systems and the effects on online programme and MOOCs development.

Secondly, understanding the impact of digital proctoring on students' and educators' behaviour and performances is another important research direction. Long-term effects studies can explore how proctoring systems influence students' academic performance, learning outcomes and overall educational experience. Additionally, examining the impact of digital proctoring on equity and access in HE is crucial, ensuring that underrepresented groups and students from certain populations (e.g. with disabilities or different skin colour) are not disadvantaged by these systems.

Lastly, to ensure the responsible and effective implementation of digital proctoring systems, addressing implementation issues is vital. Greenstein (2021) argues that the development of AI-related laws and regulations is slower than the technological development of AI and is bounded by national borders. Nonetheless, we need to further explore whether the proctoring systems are necessary and lawful for transforming the examination practices in HEIs from traditional physical locations to digital environments (Barrio, 2022). The rapid shift towards online exam transformation has serious legal implications for students (Nigam et al., 2021). We should pay more attention to investigating the legal issues and implications of using proctoring systems, especially since AI is an exam invigilator (Henry and Oliver, 2022). Ethical and legal considerations should be thoroughly investigated to understand the implications of using proctoring systems on student privacy, data protection and algorithm bias. Compliance with existing regulations and the development of guidelines and best practices is essential for institutions to navigate the legal landscape.

The research impacts of this paper are contingent upon the sustained utilisation of digital exam proctoring after the pandemic. As university campuses reopened, several factors come into play regarding the continuous usage of this technology. A crucial factor is a decision to return to in-person exams, which may reduce the need for digital proctoring as physical invigilation becomes feasible again. However, the adoption of hybrid approaches that combine online and in-person assessments allow for the continued relevance of digital proctoring, particularly for remote components of exams. The future of digital proctoring is also influenced by institutional policies and preferences, with some universities considering it as a permanent solution due to its convenience and scalability, while others perceive it as a temporary measure for exceptional circumstances. Therefore, it is crucial to comprehend the future trajectory of digital proctoring beyond the pandemic. This understanding will not only inform research interests but also drive the advancement of online examination practices, thereby enhancing our knowledge of its applications in HE.

5.4 Research limitations

Though we searched for scientific publications on digital proctoring systems from seven major databases, the queries and the search terms used in the search may not have retrieved all the relevant studies. However, it is suggested future work use different inclusion and exclusion criteria to see if new insights can be obtained. Nevertheless, the final corpus consisted of 154 articles, a sufficient dataset for a topic modelling analysis. It should be noted that because the literature review was conducted in October 2022, there may be new publications that are relevant but not included in this study at the time of publication of this research. In addition, the authors' subjective understanding may influence the labelling and interpretation of the topics. The relationships amongst the topics are not examined because of the small dataset, and the largest studies were found between 2020 and 2022 (October). The pandemic had a major effect on the adoption of the systems and led to disruptions in examination practices for both campus and online education. These effects are reflected in the topics, which weaken the exclusiveness of the topics.

6. Conclusions

This systematic literature review has provided valuable knowledge and insights into the implementation of digital proctoring in higher education, including technological advancements, stakeholders' concerns and implementation issues and challenges. The findings highlight the importance of understanding the impacts of digital proctoring systems on students and educators' performance, as well as the rapid increase in their use for digital exams in HEIs. This review paper contributes to the existing literature by offering critical analysis and insights on digital proctoring in HE, providing a foundation for future research aimed at enhancing academic integrity in online examinations and advancing the digitalisation of HE. The implications of this research extend to academics, policymakers, practitioners and students, who can benefit from a comprehensive understanding of digital proctoring implementation and its potential for improving assessment practices in the digital learning environment. In addition, the review provides guidance for future research and implementation of proctoring systems in HEIs. Challenges may hinder continuous use after campus-based education resumes post-pandemic, but digital proctoring should continue in order to ensure equal access and assessments in remote learning. Multidisciplinary collaboration is needed across academic disciplines, HEIs and educational policymakers, proctoring systems providers and society's lawmakers.

Figures

Figure 1

PRISMA literature screening process

Figure 2

Optimal numbers of topics

Figure 3

Development of publications

Figure 4

Most frequently used words

Table 1

Categories, latent topics and most frequent words

Categories	Topic label/percentage	Most frequent words
Advances in proctoring technologies	Topic 1: Solutions for detecting cheats, 17.4%	examination, system, online, student cheat detection, face
	Topic 2: Systems for students' authentication, 10.8%	system, student, authentication, development, data, cheat
Stakeholders' concerns	Topic 3: Staff and students' perceptions during the pandemic, 17%	student, online, learning, remote study, teaching tool, COVID 19 pandemic, challenges
	Topic 4: Students' performance in different proctoring environments, 19%	proctor, student, online, test, exam, result, performance, score, compare
Implementation issues and challenges	Topic 5: Transitions to digital proctoring systems: uptakes and issues, 15.9%	assessment, online technology, issue, digital, education
	Topic 6: Proctoring problems in online exams, 13.5%	exam, student, online, proctor, problem, solution
	Topic 7: Online teaching programmes and remote proctoring issues, 6.4%	test, programme, remote proctoring, candidate, results, issues

Source(s): Author's work