Increasing testing efficiency through the development of an IT-based adaptive testing tool for competency measurement

Abstract

Purpose

This paper investigates the potential of computerized adaptive testing for CMs to reduce test time. In the context of education and training, competency measurement (CM) is a central challenge in competency management. For complex CMs, a compromise must be addressed between the time available and the quality of the measurements. Increasing the efficiency of existing tests therefore poses a key challenge.

Design/methodology/approach

Results from a market analysis show a lack of integrated adaptive testing tools for CMs. The implementation, integration and evaluation of an appropriate adaptive component are presented for the example of the learning management system ILIAS used for a CM of health workers. The textbook scenario of a linear testing implementation is compared with results from the adaptive testing tool implementation. A simulation component is created to investigate the robustness of the adaptive test regarding answers that are inconsistent with the test person’s ability.

Findings

A test time reduction of 40 per cent is achieved by the adaptive test. The test shows high stability for up to 20 per cent of the answers being inconsistent with the test person’s ability.

Originality/value

Results indicate the high potential of computerized adaptive testing for CMs. The developed tool could be used for the implementation of combined multimedia computerized adaptive tests. Time savings could be utilized to improve measurement quality. Large-scale empirical studies on the interaction between competency dimensions could be facilitated. No other tool could be identified possessing these capabilities.

Citation

Kleinhans, J. and Schumann, M. (2015), "Increasing testing efficiency through the development of an IT-based adaptive testing tool for competency measurement", Interactive Technology and Smart Education, Vol. 12 No. 4, pp. 242-255. https://doi.org/10.1108/ITSE-09-2015-0023