Surveys that include skill measures may suffer from additional sources of error compared to those containing questionnaires alone. Examples are distractions such as noise or…
Abstract
Purpose
Surveys that include skill measures may suffer from additional sources of error compared to those containing questionnaires alone. Examples are distractions such as noise or interruptions of testing sessions, as well as fatigue or lack of motivation to succeed. This paper aims to provide a review of statistical tools based on latent variable modeling approaches extended by explanatory variables that allow detection of survey errors in skill surveys.
Design/methodology/approach
This paper reviews psychometric methods for detecting sources of error in cognitive assessments and questionnaires. Aside from traditional item responses, new sources of data in computer-based assessment are available – timing data from the Programme for the International Assessment of Adult Competencies (PIAAC) and data from questionnaires – to help detect survey errors.
Findings
Some unexpected results are reported. Respondents who tend to use response sets have lower expected values on PIAAC literacy scales, even after controlling for scores on the skill-use scale that was used to derive the response tendency.
Originality/value
The use of new sources of data, such as timing and log-file or process data information, provides new avenues to detect response errors. It demonstrates that large data collections need to better utilize available information and that integration of assessment, modeling and substantive theory needs to be taken more seriously.
Details
Keywords
Frank Fischer, Elisabeth Bauer, Tina Seidel, Ralf Schmidmaier, Anika Radkowitsch, Birgit J. Neuhaus, Sarah I. Hofer, Daniel Sommerhoff, Stefan Ufer, Jochen Kuhn, Stefan Küchemann, Michael Sailer, Jenna Koenen, Martin Gartmeier, Pascal Berberat, Anne Frenzel, Nicole Heitzmann, Doris Holzberger, Jürgen Pfeffer, Doris Lewalter, Frank Niklas, Bernhard Schmidt-Hertha, Mario Gollwitzer, Andreas Vorholzer, Olga Chernikova, Christian Schons, Amadeus J. Pickal, Maria Bannert, Tilman Michaeli, Matthias Stadler and Martin R. Fischer
To advance the learning of professional practices in teacher education and medical education, this conceptual paper aims to introduce the idea of representational scaffolding for…
Abstract
Purpose
To advance the learning of professional practices in teacher education and medical education, this conceptual paper aims to introduce the idea of representational scaffolding for digital simulations in higher education.
Design/methodology/approach
This study outlines the ideas of core practices in two important fields of higher education, namely, teacher and medical education. To facilitate future professionals’ learning of relevant practices, using digital simulations for the approximation of practice offers multiple options for selecting and adjusting representations of practice situations. Adjusting the demands of the learning task in simulations by selecting and modifying representations of practice to match relevant learner characteristics can be characterized as representational scaffolding. Building on research on problem-solving and scientific reasoning, this article identifies leverage points for employing representational scaffolding.
Findings
The four suggested sets of representational scaffolds that target relevant features of practice situations in simulations are: informational complexity, typicality, required agency and situation dynamics. Representational scaffolds might be implemented in a strategy for approximating practice that involves the media design, sequencing and adaptation of representational scaffolding.
Originality/value
The outlined conceptualization of representational scaffolding can systematize the design and adaptation of digital simulations in higher education and might contribute to the advancement of future professionals’ learning to further engage in professional practices. This conceptual paper offers a necessary foundation and terminology for approaching related future research.