Measuring the quality of workplace learning environments – a qualitative meta synthesis of employee questionnaires

Abstract

Purpose

This study aims to support researchers and practitioners in finding suitable instruments for future research studies and organizational quality assessments.

Design/methodology/approach

Employees’ success of learning at work is strongly influenced by the quality of the workplace learning environment. In the recent decades growing effort has been given to the development of surveys to measure the quality of workplace learning, resulting in a large number of available survey instruments. This study conceptually draws on a 3-P model and uses a qualitative metasynthesis to collect and categorize n = 94 surveys that intend to measure the quality of workplace learning (WPL).

Findings

The results underline that research on WPL environments is a highly interdisciplinary endeavor, where every discipline enriches the field by a new perspective and own foci. Overall, this study finds a focus on learning culture and working conditions, on social and functional inclusion of the learner and on support and feedback during training. Products of WPL such as professional competences or career aspirations play a minor role.

Originality/value

With the integration of quality measurement instruments from various research studies, this study produces an interactive online instrument map that gives a broad, yet organized overview of available quality measures in the WPL field.

Citation

Deutscher, V. and Braunstein, A. (2023), "Measuring the quality of workplace learning environments – a qualitative meta synthesis of employee questionnaires", Journal of Workplace Learning, Vol. 35 No. 9, pp. 134-161. https://doi.org/10.1108/JWL-06-2022-0074

1. Premise

Learning through observation and imitation, as typical for workplace learning (WPL), can be regarded as the natural and historically earliest way of knowledge acquirement (see the concept of mimesis, Billett, 2014). Yet, traditionally, most research activities in the learning sciences have been directed toward learning at schools and universities, eventually for work but outside of the workplace, rather than on workers’ learning processes through everyday work activities. The rise of knowledge economies and the concept of lifelong learning have rendered the neglect of workplaces as potential learning sites apparent and currently spur a vivid rediscovery of workplaces as rich learning environments. Policy agendas hence call for the recognition of non-formal and informal learning at workplaces [e.g. Organisation for Economic Co-operation and Development (OECD), 2010] – often based on considerations of social justice but also as an argument of economic efficiency. As a result, to mobilize the available workforce, many countries are now considering WPL as a source of untapped potential for human prosperity (Werquin, 2009).

Contemporary WPL research has, on the one hand, indeed shown that learning occurs at workplaces and that it is an adjuvant in developing specific knowledge, abilities and generic working-life competences (Kyndt et al., 2014; Klotz et al., 2015; Deutscher and Winther, 2018), intuition and expertise (Harteis and Billett, 2013) and positive personal traits and attitudes toward work such as a vocational identity (Billett, 2007; Klotz et al., 2015). On the other hand, large differences have become apparent in the extent to which workplaces are indeed rich learning environments (Negrini et al., 2016) affecting competence development and drop-out rates from educational WPL programs.

The fact that not every workplace is per se a rich learning environment has led to political discussions about learning-related work conditions and international agendas aiming to improve WPL conditions (OECD, 2020). These differences between workplace learning environments have also spurred a growing interest among employers, policymakers and researchers into what comprises learning at the workplace and how it can be facilitated by adequate conditions. The growing interest in and awareness of the quality of WPL is reflected in various research disciplines and research areas ranging from Economics and Business (e.g. educational economics, human resource management and development) to Psychology (e.g. cognitive learning psychology, work and organizational psychology), Sociology (e.g. educational sociology, sociology of work) and, of course, Education (e.g. adult education, vocational education); in all of these areas, learning at and through work nowadays has become a major theme.

A crucial precondition for research on learning through work is a valid description and measurement of WPL environments. The interdisciplinary WPL research community has already generated a considerable range of measurement instruments, as numerous studies deal with the operationalization and analysis of the quality of WPL environments. Nevertheless, the search for and the development of adequate measurement instruments continues. Because of different theoretical scopes, discipline-specific vocabulary and varying (also regional) accessibility and context, the state of research on WPL characteristics can hardly be called interconnected. This disjointedness seems problematic on the grounds of three interrelated phenomena detrimental to the progress of WPL research:

1. the existence of nonidentical but highly interrelated theoretical concepts;

2. an unmanageable number of measurement instruments in various research disciplines, often resulting in a (re)construction of already existing scales; and

3. a low comparability of empirical study results, which could, in part, be justified by different vocational fields and varying characteristics of workplaces but could also be attributed to different measurement approaches and instruments.

Those deficiencies – which are of naturally not exclusive to the WPL field but largely denote the WPL research landscape – might also explain why WPL research so far has mostly missed out on comprehensive meta-analytic contemplations (for rare exceptions, see e.g. Blume et al., 2010; Jones et al., 2016; Kleine et al., 2019) or at least comparative studies between different vocations (Kyndt and Beausart, 2017) [1].

We argue that it is key to the progress of WPL research to gain insight into the existing measurement instruments over different scientific disciplines. This insight is, in particular, key in gaining an interdisciplinary understanding of the effectiveness and efficiency of “workplaces” for learning processes. To this end, we provide an interdisciplinary overview of existing scales and items in survey instruments measuring characteristics of WPL environments. We focus here on the learner perception of WPL characteristics. To fulfill this goal, we identify and categorize with respect to their content accessible standardized test instruments from diverse disciplines and developed for different vocational settings. We then range the measures include in the instruments in a framework model for WPL learning (Tynjälä, 2013). Subsequently, we discuss the current measurement landscape with respect to common core areas and the statistical quality of the developed instruments (validation studies). The practical significance of this study lies in its support for researchers and practitioners in efficiently finding suitable instruments and avoiding unnecessary redevelopments when conducting new WPL studies. If, via such an overview, in form of a research instrument map, more instruments are used that systematically relate to the existing state of research, this use should, in the longer run, heighten the general comparability of study results and advance the feasibility of quantitative meta-analyses in the field.

2. Defining the quality of workplace learning environments

Defining the quality of WPL environments inevitably requires defining the three concepts of WPL, learning environments and quality.

1. We define WPL as the acquirement of professional competence (domain-generic and -specific knowledge, skills and attitudes), allowing for proficient performance at a workplace. The different phases or levels of action in which skills are upgraded and knowledge and attitudes are acquired may reach from observation to planning and execution and finally to reflection of work tasks (Benner, 2004). Such a broad definition of WPL includes advanced workers’ learning as well as learning as preparing to enter an occupation. However, in contrast to the concept of work-based learning, where learning takes place “either in the workplace, in settings simulating the workplace or outside the workplace” (European Centre for the Development of Vocational Training [CEDEFOP], 2015, p. 7), WPL always refers to learning processes at, but not outside, the workplace (e.g. in school-based or university-based vocational educational settings). With respect to its nature, WPL occurs through everyday practices accompanied by work-related interactions resulting in the learning of the individual employee and potentially also in the learning of the organization as a whole (Fenwick, 2008a; Collin et al., 2011). It is therefore noticeable that WPL takes place in the social context of a “community of practice” (Lave and Wenger, 1991), grounded on the dynamics between “individual actors” and “collectives” (Fenwick, 2008b).

2. For an understanding of workplaces as learning environments, a basic distinction between how workplaces afford or hinder opportunities for learning and how individuals elect to engage in activities and with the support and guidance provided by the workplace is crucial (Billett, 2001). This duality is the key determinant of the quality of learning in both formal and informal learning environments [2]. Workplaces differ substantially in how they support learning: “In particular, the readiness of the workplace to afford opportunities for individuals to participate in work activities and access direct and indirect support are key determinants in the quality of learning that arises from that participation” (Billett, 2001, p. 209). Such affordances of a WPL environment do not determine learning per se; they are mediated by the learners’ perception and elected use of the provided work environment. This perception is, in turn, premised on diverse individual learner characteristics (e.g. prior knowledge, individuals’ personal initiative, self-efficacy) (Hetzner et al., 2015).

3. In recent years, a growing research effort has been put into identifying and describing central characteristics supportive of WPL (for an overview see Kyndt and Baert, 2013). At the same time, the relation between learning conditions in the workplace and learning outcomes has been increasingly empirically analyzed (Janssens et al., 2017), allowing a deeper insight into the genesis and quality of WPL. “Quality” in the context of WPL is, however, characterized by a lack of conceptual clarity (Van den Berghe, 1997) and different connotations of the term prevail. With this study’s aim of describing the quality of WPL environments, a functional quality approach seems preferable over a notion of quality as excellence (which according to Harvey and Green (1993) is exceptional and relatively difficult to attain). This rather neutral approach allows for “good” as well as for “bad” aspects of a workplace to be studied, therefore allowing the nature of WPL to be fully grasped. But, even with a functional definition as such, an operationalization of the term “quality” is still problematic, as it is always a matter of perspective. It can be argued that quality conceptions are likely to differ between stakeholders. For instance, in the workplace context, employees’ categories for describing the quality of WPL environments might differ from those of their employers. We focus on the presumed perspective of the learner, as WPL learning is primarily a process within the learner and because we wish to acknowledge the individually mediated nature of WPL. Hence, when operationalizing workplace conditions, the judgment of the individual seems crucial. This importance of the learner’s evaluation and interpretation component is, inter alia, expressed in Tynjälä (2013) adaption of the 3-P Model of quality (Biggs, 1999), which stresses individual perception (Figure 1). To categorize and rank various test instruments related to the quality of WPL environments, we draw on that model, which differentiates WPL quality based on presage, process and product factors. Such sequential trinity for describing quality is internationally and interdisciplinary acknowledged (Biggs, 1999; Marsick et al., 2011; Ballantine, 2002; De la Fuente and Justicia, 2007; Anselmann, 2022) and, most importantly, widely accepted for the field of WPL.

The 3-P framework model by Tynjälä (2013) conceptualizes WPL by focusing on the “perceived quality” of an individual and therefore fits the duality notion of WPL learning research best. Both the learner factors and the learning context are represented within the presage dimension. The process dimension covers the description of WPL characteristics during interactional processes, including the structure and performance of work tasks or the learner’s interaction with others. Finally, within the product dimension, all learning-related outcomes are summarized; these are mainly focused on the individual’s personal and professional development (Tynjälä, 2013).

Summing up, for this paper we define the quality of WPL environments as the subjective perception of presage and process characteristics of the workplace (affordances) that, in their interaction with individual learner characteristics, are conducive to certain learning outcomes (products). Given this broad definition and framework, it is impossible to cover the field of quality research in its entirety. Instead, this paper provides a general overview of current research on workplace environments and their presage, process and product dimensions from the learner point of view. Accordingly, the measurement instruments related to individual learner characteristics (e.g. prior knowledge, agency) are excluded – albeit acknowledging that, for empirical studies, it is crucial to model the influences of the individual learner characteristics and traits that are relevant for the respective study purpose and sample.

3. Method

While there is generally broad consensus about the distinction between presage, process and product dimensions for describing WPL environments, the key characteristics specifying those dimensions vary greatly between studies and their respective measurement instruments. To list and categorize existing instruments capturing characteristics of the WPL environments, the method of qualitative metasynthesis seems particularly suitable. It combines a systematic literature search with a qualitative content analysis, thus allowing research to be systematically and fully integrated. Qualitative metasynthesis, sometimes also referred to as qualitative meta-analysis, represents a relatively new and, to date, rarely chosen form of analysis (Eisend, 2014) that is, however, spreading to more and more research areas. Like all meta-analyses, it can be described as a systematic summary of published studies with the help of a set of fixed rules (Leary and Walker, 2018). Following Lipsey and Wilson (2001), similarly to conducting a quantitative meta-analysis, the findings of several studies are amalgamated to develop a comprehensive overall new model (see also Leary and Walker, 2018), only that, in this case, the database is qualitative in nature and – unlike in a regular systematic review – is extended through qualitative content analysis and the formation of categories. Hence, this procedure exceeds a classic systematic literature review by not just providing a summary and possibly discussion of the state of research; it contains a systematic qualitative content analysis aiming to inductively generate a novel and integrative model.

For our study, the qualitative data base consists of the test items used in employee surveys. In accordance with the process logic of metasynthesis (Jensen and Allen, 1996), the first two steps comprise literature research and literature selection. The core of a qualitative metasynthesis then consists of inductively determining qualitative categories (item analysis and categorization). Finally, an integrative model of categories is built as a result of this methodological process (model building). These steps structure the following paragraphs of our methods section and will be described in greater detail. Where possible within the qualitative metasynthesis framework, these steps were performed in accordance with the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses‘ (PRISMA) recommendations (Liberati et al., 2009) to make the four blocks reproducible.

3.1 Literature research

To summarize the state of research into the measurement of WPL learning environments, we conducted a systematic literature search targeting various key words indicating learning at the workplace in combination with search terms indicating measurement instruments, resulting in 36 search combinations, displayed in Figure 2.

To exclude as few studies as possible (Jensen and Allen, 1996) to analyze a broad spectrum of WPL research, no limitations regarding publication type, profession, industry or country were set. We did, however, establish two general eligibility criteria. First, for the purpose of assembling an international online repository, studies and test instruments had to be published in the English language. Second, since the systematic recording of WPL using written survey instruments only began in the mid-1990s, the literature search was limited to publications from 1990 onwards. In addition, before the 1990s, WPL was often understood to involve only formal learning (Kyndt et al., 2014), leading to a limited focus of the developed instruments. With respect to the chosen information sources, this search strategy was implemented interdisciplinarily in five large databases in the fields of Education, Psychology, Economics and Business and Sociology [Education Resource Information Center (ERIC), PsycINFO, Business Source Premier, EconLit and Social Sciences Citation Index (SSCI)]. Following Walsh and Downe (2005), the keyword search was complemented by a snowballing procedure, marking relevant references of the identified studies, so as not to overlook relevant studies.

3.2 Literature selection

The initial search generated 69,514 results (including duplicates). A title and abstract analysis led to the majority of these hits being eliminated because they were related to a non-WPL context. After this elimination, 558 records were identified as fitting to the WPL context and another 29 records were added through snowballing (leading to a total of 587 records). The list of potential records was further reduced by removing duplicate texts, from both within and across the databases, resulting in 465 remaining records. For 31 studies, neither an abstract nor a full paper was accessible. This left 434 studies for the full text screening process. After screening the full articles, another 74 studies were excluded because of a misfit to the WPL context (360 studies remaining). In a second step, studies were eliminated if they either were only theoretically or conceptually founded or used no written survey (e.g. observation studies or interviews), focused solely on classroom or university settings instead of WPL focused on a point of view other than that of the employees – for instance, the perspective of training personnel – or focused exclusively on individuals’ personal learning characteristics at the workplace (e.g. agency, identity, learning types, mental or physical limitations) instead of the WPL environment. These exclusion criteria reduced the number of records to 360 studies. The reduction of search results to the included studies is comparatively drastic for two reasons. First, our original search string was broad to avoid overlooking relevant work in an interdisciplinary field. Second, most of the excluded studies mentioned the workplace but were not conducted in a WPL context (eligibility criterion 1) or did not contain empirical work in the form of a survey instrument (eligibility criterion 2).

Then the availability of the full study as well as the availability of the measurement instruments was checked for. For 77.22% of the studies, the underlying test instrument was neither included in an Appendix nor retrievable from supplemental materials or online resources. In such cases, the authors of the respective studies were contacted and asked if they wish to provide the test instruments for the analysis and/or the online repository. After a return time of eight weeks, 21.28% of the contacted authors responded by providing their instruments. Prior to the content analysis, it was necessary to identify instruments that had been used in more than one study to avoid multiple counting, which would have distorted the results. By checking for multiple uses, 92 different studies with 94 different test instruments could be extracted and formed the basis for the subsequent qualitative content analysis (Appendix Table A1 contains an overview of the included studies). The fact that there are more instruments than studies, even though some studies fall back on the same questionnaire, is because of the circumstance that many studies use more than one measurement instrument to describe the workplace environment.

3.3 Item analysis and categorization

To develop the WPL measurement model and the list of measurement instruments, all items of all identified measurement instruments were recorded in an Excel spreadsheet. This data collection resulted in 3,688 test items. These items were connected and synthesized by inductively determining categories based on Mayring’s (2000) qualitative content analysis, including generalization, selection and bundling. First, all items were collected from the measurement instruments in tabular form. Second, since the focus of this metasynthesis lies in measuring WPL environments, all irrelevant test items were excluded. This means that particularly individual factors (personal data or demographic factors, personal biography or personal traits) were excluded because they are not part of the learning environment itself, even though they almost certainly influence WPL and are collected in almost all measurement instruments. Then, every item was categorized separately on the basis of specific contents and classified within a general dimension of the 3-P model (Tynjälä, 2013). Within these dimensions, items with similar contents were grouped and subcategories were formed. Subsequently, keywords were assigned to categories, allowing a reasonable summary of the contents. A codebook for the keywords and categories was developed. To ensure and indicate the reliability of the item mapping the data was coded twice by two independent individuals[3] (intercoder reliability = 0.934). An exemplary assignment is shown in Appendix Table A2.

4. Results

The distribution of original hits to finally available test instruments can be retrieved from Table 1, reported separately for the different scientific disciplines.

The instrument output was highest for the disciplines of Education (ERIC) and Psychology (PsychINFO) (46 instruments each), followed by a significantly lower output for the Economics and Business database (BSP and EconLit) (13 instruments) and the Sociology database (SSCI) (12 instruments). Regarding the quality of the identified instruments, 81.08% of the instruments report reliability analyses (most often through a reporting of Cronbach’s alpha), while a lower percentage (63.06%) of the identified measurement instruments was additionally validated structurally through factor analysis. Advanced factor analyses were most common for studies identified through Sociology (100% within this database), followed by Psychology (71.79%) and Economic and Business databases (61.54%). In ERIC, 45.45% of the identified test instruments were structurally validated through factor analysis. Over all disciplines, in total, 94 test instruments containing 3,688 items could be analyzed. The resulting map consists of three dimensions, seven content areas and 22 categories. For space/design reason, the keyword catalog contains another 66 keywords. The created content areas and categories can be found in Figure 3, and the keyword catalogue is displayed in the Appendix Table A3 [4].

The presage dimension comprises the employees’ perception of general factors of their work environment that existed prior to or independent of their interaction in the company. Usually organizational conditions are largely determined for the employee, although they may significantly influence the further individual processes and outcomes of WPL. For the purpose of our analysis, it was useful to distinguish within the content area of Organizational Conditions between general Framework Conditions (e.g. number of employees, industry or city, profitability), Working Conditions (e.g. work status, vacation days, payments and rewards policy, safety standards) and Learning Culture (e.g. perception of learning atmosphere, shared views on learning, enforcement of common learning standards).

The process dimension targets aspects that shape WPL through an interaction of employees with their work and with others. Here, work tasks can be complex and meaningful in different ways. Furthermore, social interaction through collegiality and cooperation as well as pedagogical inclusion through support and feedback may vary and belong to this dimension. Correspondingly, the analysis of item content for the process dimension yielded the three major content areas Work Tasks, Social Interaction and Pedagogic Interaction. Within the first area, the following categories were determined as intuitive in assigning the items: Workload (e.g. time pressure, physical stress), Variation of Tasks (e.g. degree of reiteration of tasks, possibility of knowledge expansion), Autonomy of Tasks (e.g. decision-making freedom, scope of action), Meaningfulness of Tasks (e.g. relevance for the company, relevance beyond the company), Complexity of Tasks (e.g. difficulty of tasks, information availability) and the Work Requirements (e.g. skill fit for tasks, degree of challenge). The Social Interaction content area was subdivided into the categories Involvement in Expert Culture (e.g. appreciation and codetermination among colleagues), Functional Inclusion (e.g. collaboration on tasks) and Social Inclusion (e.g. friendships, sense of belonging). The third content area comprised Methods/Activities (e.g. learning and training methods such as e-learning), Training Resources (e.g. fixed learning times, learning budget), Support (e.g. Coaching, Mentoring) and Feedback (e.g. reflection, performance appraisal).

Finally, all items that could be interpreted, as the results of WPL were included in the product dimension. The subdivision of the dimension was adapted according to a conceptual suggestion by Kyndt et al. (2014). Correspondingly, a distinction was made between Generic, Job-Specific and Organizational-level outcomes of WPL. The categories describe the effects of employees’ learning experiences and mainly deal with the results employees expect in terms of their lives, their jobs and their companies. Generic outcomes can refer to acquiring Professional Competences (e.g. generic knowledge, occupational skills) or the employees Satisfaction with Learning Opportunities, sometimes in the sense of self-fulfillment. Job-specific outcomes may refer to Occupational Identity or to Occupational Career Aspirations (e.g. further education). The category Organizational-level outcomes target employees’ Organizational Identity and Organizational Career Aspirations (e.g. promotion). Via this identified empirical model, readers can be guided online to the respective measurement instruments covering their category or categories of interest (see Excel File). Here, they can find possible items of interest, the instrument name and bibliographical information of the studies that developed or used the respective item. Moreover, a search and filter of the WPL measurement inventory by dimension, by content area, by category, by study or finally by instrument is available.

Within this model, the frequency of study coverage of the different categories can be illustrated. Figure 4 gives an overview of the frequency of category usage in the different studies.

Here, clear foci are identifiable. For example, with respect to the presage dimension, there seems to be a strong focus on Learning Culture and on Working Conditions whereas in the process dimension, most attention is given to Functional and Social Inclusion while the analysis of the Meaningfulness of Tasks and Training Resources is relatively underrepresented. In the product dimension, the category of Occupational Identity is most prevalent while Occupational Career Aspirations and Professional Competences play a minor role.

Moreover, there seem to be differences in the coverage of categories dependent on the scientific discipline. Appendix Table A4 gives a numerical overview of the category coverage for studies stemming from the four different disciplinary contexts. The results display how the different disciplines take different interests in the topic of WPL. Educational studies take a particular interest in the concept of Learning Culture, Involvement in Expert Culture and in Methods and Activities during the learning process. Psychological studies show less interest in the presage dimension, taking instead a stronger interest in Social Inclusion and in the Work Requirements, Satisfaction with Learning Opportunities and Occupational Career Aspirations. Meanwhile, the focus in the discipline of Economics and Business is primarily on Working Conditions and professional outputs such as Professional Competences and Organizational Identity. Finally, Sociological studies are mostly concerned with Workload and Work Tasks, as well as with Functional Inclusion, Occupational Identity and Career Aspirations.

5. Discussion

The study collects and categorizes n = 94 measurement instruments on the quality of WPL environments from the perspective of the learner via a qualitative metasynthesis and integrates the identified 3,688 items into a research instrument map. This map consists of three dimensions, seven content areas, 22 categories and 66 keywords that characterize contemporary efforts to measure the quality of WPL environments. The categories and contents identified through our instrument analysis resemble the content dimensions identified in an integrative literature review on informal workplace learning by Jeong et al. (2018), who were looking empirically at what factors influence informal learning in the workplace. An analysis of our research instrument map allows insights into the quality and foci of WPL research instruments, differentiated for the research disciplines investigated. Here, first of all, it becomes apparent that the research on the quality of WPL environments must be characterized as a highly interdisciplinary endeavor, where every discipline enriches the field by a new perspective: while Education takes a didactic perspective with a focus on Learning Cultures, Methods and Activities, Psychology puts a focus on learner-side dispositions that interact with the work environment. The discipline of Economics and Business further contributes to the field by taking a relatively institutional perspective, targeting amongst other things Working Conditions and outputs that are of particular importance from an institutional perspective. This finding of a difference in perspectives is in line with the results of a scoping review by Brandi and Iannone (2021). It suggests that studies from the Human Resources field have chiefly focused on workplace learning (as a strategic means) for developing workforce abilities, as a form of capital to maximize business performance, while WPL studies tend to take a relatively deep interest in understanding the mechanisms between workplaces and different forms of learning processes (as an end). Sociology finally adds to the shared research field by analyzing socialization processes in and through work environments with a particular focus on the nature of Work Tasks and on influences on Occupational Identities and biographies. When analyzing the quantity and quality of the identified instruments, it is noteworthy that the disciplines of Education and Psychology have produced the highest output of measurement instruments in quantitative terms while instruments stemming from Sociology most often administer advanced validation procedures to report on the quality of the developed measures, followed by studies from Psychology and Economics and Business. This result may be explained by different reporting customs in the respective disciplinary contexts.

A glance at the research foci over all disciplines shows that in the input dimension there seems to be a strong focus on Working Conditions and Learning Culture whereas, in the process dimension, most attention is given to forms of Functional and Social Inclusion, to Involvement in Expert Culture and to the categories Support and Feedback. Meanwhile the analysis of tasks (meaningfulness and complexity) and Training Resources are underrepresented. These finding can be in part explained by the greater number of studies from the disciplines of Education and Psychology, compared to Sociology and Economics and Business. It is moreover noteworthy that the coverage of the whole product dimension (with the exception of the concept of Occupational Identity) is underrepresented. Here, the results are in line with – or may indeed explain – the phenomenon that there are comparatively few research findings on the learning outcomes of WPL (Janssens et al., 2017; Kyndt et al., 2014). This dearth can be attributed to a low number of studies and measurement instruments that cover this quality dimension of WPL, as our results show. While the disciplines of Sociology and Economics and Business are relatively active regarding the product dimension, these disciplines take a smaller share in WPL research compared to that of Education and Psychology, where the focus seems to lie primarily in the presage and process dimension.

Another aspect that came up when analyzing the product dimension is that quantifiable outputs of workplace learning, such as financial income, are largely neglected as outputs even though they might play a crucial role in the learning process. This aspect indicates that our overview can hardly show what is obscure in the research on the quality of WPL environments. Therefore, the study presented here is constrained by the identification, selection and aggregation of WPL characteristics that constitute former research activities. The generated research map cannot – and does not intend to – claim normative completeness. Moreover, since this paper aimed to operationalize WPL, no assumptions can be made about the influence of one factor on another. For example, despite the large number of assigned test items in the category Learning Culture, it cannot be concluded that a good learning climate and culture automatically lead to increased WPL. Such assumptions must be confirmed by quantitative meta-analyses. The large number of items in this category only reflects the fact that, within the 94 different test instruments, there are many test items related to Learning Culture. Here, the created measurement model of WPL in environments can serve as an overview and measurement instrument repository for future research to investigate relationships between the different identified categories of WPL environments.

Further, the results are influenced by several methodological issues we encountered during data collection and analysis. Collecting the different test instruments was time consuming, taking 26 months for one scientific assistant along with two student assistants. This length of time was mostly because of a large number of search results containing numerous duplicates of studies and test instruments and because of unavailability of instruments. Even though, where necessary, we contacted authors to retrieve as many instruments as possible, the studies and/or the test instruments were not always made available so that our results only depict the accessible instrument landscape. Moreover, the focus of this work was on written forms of employee questionnaires, which constitute the most frequent study approach in WPL (Berings et al., 2006). Therefore, no fundamental statements on the operationalization of learning conditions in the context of WPL for other research methods (e.g. via interviews) and actors other than the learner (e.g. training personnel) can be deduced from the analysis presented here.

Moreover, our analysis, which aimed at a quality and instrument map from the learner perspective (learner surveys), is bound to only one perspective on WPL quality. A metasynthesis on employers’ or other stakeholders’ perceptions of quality will possibly identify different measurement instruments and potentially also other quality categories. Here, we wish to note that quality aspects of workplace learning and particularly learning outcomes should ideally also be measured through multiple stakeholders, particularly to avoid single-source biases in cross-sectional and longitudinal analyses of learning processes. Additionally, wherever feasible, objective measurement approaches, e.g. tests for assessing competences, should be used to contrast self-report measures.

Finally, because of the focus of this paper, which is on in-company learning conditions, the individual learner characteristics and traits, such as intelligence quotient (IQ) or motivation, were excluded. However, during the categorization of the test instruments, it became apparent that, while differentiating between an individual’s traits and the workplace surrounding is theoretically straight forward, this simplicity is not always reflected when formulating the items. Instead, one item may contain individual traits as well as an evaluation of the work surrounding to provide a relative assessment of the work environment. Those items were then still included in the inventory and occur particularly often for the category Work Requirements, where learners were often asked for the work requirements in relation to their personal ability level. This combination can be interpreted as an instance where the individual and his or her perception of the surroundings are, in practice, closely aligned. This and the fact that previous research has shown that motivation influences learning outcomes and that, conversely, the organization can influence motivation (Wielenga-Meijer et al., 2010) indicate that including personal learner characteristics seems absolutely essential to fully understand the interactions between working individuals and their work environment. Furthermore, this interaction is less linear than depicted in the research map presented here, where the visualization has been simplified for the sake of a clearer overview.

Acknowledging these limitations, we nevertheless presume that the generated research map displaying the instrument landscape gives a broad and interesting overview of current measurement instruments operationalizing the survey research on workplace learning quality. We moreover believe that it will prove useful for several purposes. First, the sheer amount of measurement instruments from various research disciplines now seems manageable via this metatool. The ensuing map helps researchers in finding suitable instruments when conducting new studies in this field, allowing them to avoid a (re)construction of already existing scales and items as well as facilitating a later comparison of empirical study results through the (interdisciplinarily) shared instruments. Concurrently, by providing an online version of the research instrument map as an interactive tool, practitioners (e.g. employers, staff managers, learning managers) can use the developed tool when assessing the quality of WPL in their organizations. For these purposes, the developed tool gives wide and organized access to the identified research instruments, showing the item content as well as leading the user back to the links where the original instruments can be found. Here, it is important to point out that single items should not be taken from the resulting item list without considering their associated scale. Within the test instruments, the test items often refer to or are based on each other. Researchers should therefore use the generated research instrument map as a systematic starting point for their instrument assembly and also always retrieve and engage with the corresponding original studies to avoid misinterpreting items and scales and to get a better grasp of the theoretical rationale underlying the respective measurement scales.

Supplementary material

The supplementary material for this article can be found online at www.bwl.uni-mannheim.de/media/Lehrstuehle/bwl/Deutscher/Interactive_Item_Map.xlsx

Figures

Figure 1.

Three-p model of workplace learning

Figure 2.

Search string

Figure 3.

Measurement model of workplace learning environments

Figure 4.

Frequency of category usage throughout the identified studies

Table 1.

Literature and test instrument selection process and results

Database	Hits	Records	Full paper	Available relevant studies with test instruments	Available relevant test instruments
Education	22,032	192	184	39	46
Psychology	14,464	173	164	34	46
Economics and business	24,358	83	75	9	13
Sociology	8,631	110	101	12	12
Snowballing	29	29	29	25	25
Σ	69,514	587	553	119	142

Note:

The results of this table contain duplicates due to the utilisation of multiple databases

Table A1.

Overview of the included studies

Year	ID	Study	Test instrument
1974	001	Hackman, J. R., and Oldham, G. R. (1974). The Job Diagnostic Survey: An Instrument for the Diagnosis of Jobs and the Evaluation of Job Redesign Projects. Yale University, Department of Administrative Sciences, New Haven	Job Diagnostic Survey (JDS)1, a
1979	002	House, J. S., Wells, J. A., Landerman, L. R., McMichael, A. J., and Kaplan, B. H. (1979). Occupational Stress and Health among Factory Workers. Journal of Health and Social Behavior, 20(2), 139-160	Measures of Stressa
	003	Mowday, R. T., Steers, R. M., and Porter, L. W. (1979). The Measurement of Organizational Commitment. Journal of Vocational Behavior, 14(2), 224-247	Organizational Commitment Questionnaire (OCQ)a,b
1985	004	Herold, D. M., and Parsons, C. K. (1985). Assessing the Feedback Environment in Work Organizations: Development of the Job Feedback Survey. Journal of Applied Psychology, 70(2), 290-305	Feedback Culturea,b
1989	005	Ashford, S. J., Lee, C., and Bobko, P. (1989). Content, Causes, and Consequences of Job Insecurity: A Theory-Based Measure and Substantive Test. The Academy of Management Journal, 32(4), 803-829	Job Insecurity Scale (JIS)a
1991	006	Owens, T., and Cohen, C. (1991). Northwest Entry-Level Worker Study. Portland: Northwest Regional Education Laboratory	Northwest Entry-Level Worker Survey Tabulation
1993	007	Oregon State Economic Development Department (1993). Oregon Works: Assessing the Worker Training and Work Organization Practices of Oregon Employers	Oregon Employer Survey
	008	Morton, S. T. (1993). Socialization-related learning, job satisfaction and commitment for new employees in a federal agency. Dissertation, Virginia Polytechnic Institute and State University, Blacksburg	Survey of new employee's experiences on the joba,b
1994	009	Taormina, R. J. (1994). The Organizational Socialization Inventory. International Journal of Selection and Assessment, 2(3), 133-145	The Organizational Socialization Inventory (OSI)a,b
	010	Krueger, A., and Rouse, C. (1994). New Evidence on Workplace Education (NBER Working Paper 4831). Cambridge: National Bureau of Economic Research	Skills Classes Survey
1996	011	Martineau, J. W. (1996). A contextual examination of the effectiveness of a supervisory skills training program. Dissertation, Pennsylvania State University, State College	Frontline Leadership Studya,b
1997	012	Reio, T. G. Jr. (1997). Effects of curiosity on Socialization-related Learning and Job Performance in Adults. Dissertation, Virginia Polytechnic Institute and State University, Blacksburg	Job Performance Surveya + Workplace Adaptation Questionnaire (WAQ)a,b
1999	013	Edmondson, A. C. (1999). Psychological Safety and Learning Behavior in Work Teams. Administrative Science Quarterly, 44(2), 350-383	Learning Behavior in Work Teamsa,b
	014	Mikkelsen, A., and Grønhaug, K. (1999). Measuring Organizational Learning Climate: A Cross-National Replication and Instrument Validation Study Among Public Sector Employees. Review of Public Personnel Administration, 19(4), 31-44	Learning Climate Questionnaire (LCQ)a,b
2001	015	Washington State Workforce Training and Education Coordinating Board (2001). Workforce Training Results: An Evaluation of Washington State's Workforce Development System. Olympia: Workforce Training and Education Coordinating Board	Workforce Training: Needs and Practices of Employers in Washington State (1999)
	016	Strickland, A., Simons, M., Harris, R., Robertson, I., Harford, M., and Edwards, A. (2001). Evaluating on- and off-the-job approaches to learning and assessment in apprenticeships and traineeships. Leabrook: NCVER	Learning and assessment in traineeships and apprenticeships
2002	017	Smith, E., and Wilson, L. (2002). Learning and training in school-based new apprenticeships. Leabrook: NCVER	Learning and training in school-based apprenticeships and traineeships
	018	Ridoutt, L., Dutneall, R., Hummel, K., and Smith, C. S. (2002). Factors influencing the implementation of training and learning in the workplace. Leabrook: NCVER	Implementation of learning and training in selected industries
2003	019	Kirby, J. R., Delva, M. D., Knapper, C., and Birtwhistle, R. V. (2003). Development of the approaches to work and workplace climate questionnaires for physicians. Evaluation and The Health Professions, 26(1), 104-121	Workplace Climate Questionnaire for physiciansa,b
	020	Kirby, J. R., Knapper, C. K., Evans, C. J., Carty, A. E., and Gadula, C. (2003). Approaches to learning at work and workplace climate. International Journal of Training and Development, 7(1), 31-52	Workplace Climate Questionnaire (WCQ)2,a,b
	021	Marsick, V. J., and Watkins, K. E. (2003). Demonstrating the Value of an Organization's Learning Culture: The Dimensions of the Learning Organization Questionnaire. Advances in Developing Human Resources, 5(2), 132-151	Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b
	022	Van Yperen, N. W., and Hagedoorn, M. (2003). Do High Job Demands Increase Intrinsic Motivation or Fatigue or Both? The Role of Job Control and Job Social Support. The Academy of Management Journal, 46(3), 339–348	Job Demands and Motivationa,b
2004	023	Steelman, L. A., Levy, P. E., and Snell, A. F. (2004). The Feedback Environment Scale: Construct Definition, Measurement, and Validation. Educational and Psychological Measurement, 64(1), 165-184	Feedback Environment Scale (FES)a,b
2005	024	Clarke, N. (2005). Workplace Learning Environment and its Relationship with Learning Outcomes in Healthcare Organizations. Human Resource Development International, 8(2), 185-205	National survey of specialist palliative care healthcare organizations (hospices) in the UK
	025	Lohman, M. C. (2005). A survey of factors influencing the engagement of two professional groups in informal workplace learning activities. Human Resource Development Quarterly, 16(4), 501-527	Informal Workplace Learning Survey4,a
2006	026	Lohman, M. C. (2006). Factors influencing teachers' engagement in informal learning activities. Journal of Workplace Learning, 18(3), 141-156	Informal Workplace Learning Survey4,a
	027	Morgeson, F. P., and Humphrey, S. E. (2006). The Work Design Questionnaire (WDQ): Developing and Validating a Comprehensive Measure for Assessing Job Design and the Nature of Work. Journal of Applied Psychology, 91(6), 1321-1339	Work Design Questionnaire (WDQ)a,b
	028	Kersley, B., Alpin, C., Forth, J., Bryson, A., Bewley, H., Dix, G., and Oxenbridge S. (2006). Inside the workplace: Findings from the 2004 Workplace Employment Relations Survey. London: Routledge	Workplace Employment Relations Survey (WERS)
2007	029	Chang, S.-C., and Lee, M.-S. (2007). A study on relationship among leadership, organizational culture, the operation of learning organization and employees' job satisfaction. The Learning Organization, 14(2), 155-185	Learning Organizationa,b
	030	Hicks, E., Bagg, R., Doyle, W., and Young, J. D. (2007). Canadian accountants: Examining workplace learning. Journal of Workplace Learning, 19(2), 61-77	Learning Barriers Survey + Learning Facilitators Survey
	031	Schmidt, S. W. (2007). The Relationship Between Satisfaction with Workplace Training and Overall Job Satisfaction. Human Resource Development Quarterly, 18(4). 481-498	Job Training and Job Satisfaction Survey (JTJSS)a,b
	032	Jennings, M. E. (2007). Adult learning in a workplace setting: Key Factors Associated with the Development of Performance and Efficacy. Dissertation, Ann Arbor: ProQuest LLC	Job Satisfaction Scalea,b
2008	033	Chiva, R., and Alegre, J. (2008). Emotional intelligence and job satisfaction: The role of organizational learning. Personnel Review, 37(6), 680-701	Organizational Learning Capability (OLC) Questionnairea,b
	034	Vazsonyi, A. T., and Snider, B. J. (2008). Mentoring, competencies, and adjustment in adolescents: American part-time employment and European apprenticeships. International Journal of Behavioral Development, 32(1), 46-55	Mentoring Behaviora,b
	035	Doornbos, A. J., Simons, R.-J., and Denessen, E. (2008). Relations Between Characteristics of Workplace Practices and Types of Informal Work‐Related Learning: A Survey Study Among Dutch Police. Human Resource Development Quarterly, 19(2), 129-151	Learning from Police Work Questionnaire (LPWQ)a,b
	036	Berg, S. A., and Chyung, S. Y. (2008). Factors that influence informal learning in the workplace. Journal of Workplace Learning, 20(4), 229-244	Job Tasks + Informal Workplace Learning Survey4,a,b + Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b
	037	Cooney, R., and Long, M. (2008). Inter-firm cooperation in training. A National Vocational Education and Training Research and Evaluation Program Report. Adelaide: NCVER	Inter-firm Cooperation in Training
	038	Deilkås, E. T., and Hofoss, D. (2008). Psychometric properties of the Norwegian version of the Safety Attitudes Questionnaire (SAQ), Generic version (Short Form 2006). BMC Health Services Research, 8(1), 191	Safety Attitudes Questionnaire (SAQ) Generic version (Short Form 2006)a,b
2009	039	Burgard, S. A., and Ailshire, J. A. (2009). Putting Work to Bed: Stressful Experiences on the Job and Sleep Quality. Journal of Health and Social Behavior, 50(4), 476-492	Job Strains and Sleep Quality
	040	Taris, T. W., and Schreurs, P. J. G. (2009). Explaining worker strain and learning: How important are emotional job demands? Anxiety, Stress and Coping, 22(3), 245-262	Emotional Job Demandsa,b
	041	Kyndt, E., Dochy, F., and Nijs, H. (2009). Learning conditions for non-formal and informal workplace learning. Journal of Workplace Learning, 21(5), 369-383	Questionnaire on learning conditions for non-formal and informal workplace learninga,b
	042	DeSpain, L. (2009). The Relationship between Police Supervisor Training and Job Satisfaction Levels as reported by Patrol Officers. Dissertation, Ann Arbor: ProQuest LLC	Job in General Scale (JIG) (1997 Revision)5,a + Job Descriptive Index (JDI)6,a
	043	Schalk, R. and van Woerkom, M. (2009). Does age influence relationship between learning opportunities at work and employee wellbeing and mobility? Proceedings of the ECER VETNET Conference 2009: Papers presented for the VETNET programme of ECER 2009 “Theory and Evidence in European Educational Research”, Vienna, September 28–30	Learning Opportunities
2010	044	Yamazaki, Y., and Kayes, D. C. (2010). Learning and work satisfaction in Asia: A comparative study of Japanese, Chinese and Malaysian managers. The International Journal of Human Resource Management, 21(12), 2271-2289	Index of Job Satisfaction (IJS)7,a
	045	European Social Survey ERI. (2010). European Social Survey (ESS). London: ESS ERIC Headquarters	European Social Surveya,b
2012	046	Carpita, M., and Golia, S. (2012). Measuring the quality of work: The case of the Italian social cooperatives. Quality and Quantity, 46, 1659-1685	Indagine sulle Cooperative Sociali Italiane (ICSI, 2007)b
	047	Liu, Y. C., Huang, Y.-A., and Lin, C. (2012). Organizational factors’ Effects on the Success of E-Learning Systems and Organizational Benefits: An Empirical Study in Taiwan. The International Review of Research in Open and Distance Learning, 13(4), 130-151	E-Learning in Organizational Contextsa,b
	048	Martins, H., and Proença, T. (2012). Minnesota Satisfaction Questionnaire: Psychometric Properties and Validation in a Population of Portuguese Hospital Workers (FEP Working Papers 471). Porto: FEP	Minnesota Satisfaction Questionnaire (MSQ) short version8,a,b
	049	Westover, J. H. (2012). Comparative international differences in intrinsic and extrinsic job quality characteristics and worker satisfaction, 1989-2005. International Journal of Business and Social Science, 3(7), 1-15	General Social Survey (GSS)
2013	050	Kyndt, E., Raes, E., Dochy, F., and Janssens, E. (2013). Approaches to Learning at Work: Investigating Work Motivation, Perceived Workload, and Choice Independence. Journal of Career Development, 40(4), 271-291	Workplace Climate Questionnaire (WCQ)2,a,b
	051	Hoekstra, B. (2013). Relating Training to Job Satisfaction: A Survey of Online Faculty Members. Dissertation, Northcentral University, San Diego	Index of Job Satisfaction (IJS)7,a
	052	Molino, M., Ghislieri, C., and Cortese, C. G. (2013). When work enriches family‐life: The mediational role of professional development opportunities. Journal of Workplace Learning, 25(2), 98-113	Professional Developmenta,b
	053	Masier, D. J. (2013). An Exploratory Study of the Relationship between Self-Directed Learning and Senge's Five Disciplines Necessary to Become a Learning Organization: In a High-Tech Company. Dissertation, Ann Arbor: ProQuest LLC	Organizational Self-directed Learning and the Learning Organizationa
	054	Ariga, K., Kurosawa, M., Ohtake, F., Sasaki, M., and Yamane, S. (2013). Organization adjustments, job training and productivity: Evidence from Japanese automobile makers. Journal of the Japanese and International Economies, 27(C), 1-34	Skills Development in the Workplace
	055	Hosie, P., Jayashree, P., Tchantchane, A., and Lee, B. S. (2013). The effect of autonomy, training opportunities, age and salaries on job satisfaction in the South East Asian retail petroleum industry. The International Journal of Human Resource Management 24(21), 3980-4007	Job Characteristics and Job Satisfactiona
2014	056	Nikolova, I., Van Ruysseveldt, J., De Witte, H., and Syroit, J. (2014) Work-based learning: Development and validation of a scale measuring the learning potential of the workplace (LPW). Journal of Vocational Behavior, 84(1), 1-10	Task-based Workplace Learning Scale10,a,b
	057	Jason, E. (2014). Factors Affecting Employee Trust in Leadership. Dissertation, Ann Arbor: ProQuest LLC	Brief Index of Affective Job Satisfaction (BIAJS)a + Organizational Culture Survey (OCS)a
	058	Hobdy, A. K. (2014). Workplace Learning: Relationships Among Perceived Learning Opportunities, Job Satisfaction and Commitment in Small Business. Dissertation, Ann Arbor: ProQuest LLC	Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b+ Staying or Leaving Index (SLI)a,b + Job Satisfaction Survey (JSS)a,b + Training and Learning Opportunitiesa,b
	059	Kim, M. K., Kim, S. M., and Bilir, M. K. (2014). Investigation of the Dimensions of Workplace Learning Environments (WLEs): Development of the WLE Measure. Performance Improvement Quarterly, 27(2), 35-57	Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b+ Workplace Learning Environments (WLEs)a,b
2015	060	Fang, P., Luo, Z., and Fang Z. (2015). What is the job satisfaction and active participation of medical staff in public hospital reform: A study in Hubei province of China. Human Resources for Health, 13, 34	Minnesota Satisfaction Questionnaire (MSQ)8,a,b
	061	Preenen, P., Verbiest, S. Van Vianen, A., and Van Wijk, E. (2015). Informal learning of temporary agency workers in low-skill jobs: The role of self-profiling, career control, and job challenge. Career Development International, 20(4), 339-362	Job Challenge and Informal Learninga,b
	062	Steffgen, G., Kohl, D., Reese, G., Happ, C., and Sischka, P. (2015). Quality of Work: Validation of a New Instrument in Three Languages. International Journal of Environmental Research and Public Health, 12, 14988-15006	The Measuring Quality of Work questionnaire (MQW)a,b
	063	Maher, K. (2015). The Role of the Personality in the Advisory Relationship. Dissertation, Ann Arbor: ProQuest LLC	Advisory Working Alliance Inventory (AWAI-S)a
	064	Weinberg, F. J., Mulki, J. P., and Lankau, M. J. (2015). The impact of effort-oriented epistemological beliefs on mentoring support. Journal of Workplace Learning, 27(5), 345-365	Mentoring Supporta,b
	065	Messmann, G., and Mulder, R. H. (2015). Conditions for apprentices’ learning activities at work. Journal of Vocational Education and Training, 67(4), 578-596	Apprentices Learning Activities at Work and School–Work Alignmenta,b
	066	Yeh, H.-J. (2015). Job Demands, Job Resources, and Job Satisfaction in East Asia. Social Indicators Research, 121(1), 47-60	International Social Survey Programme (ISSP) Work Orientations III questionnaire (2005)9,a
	067	Van Dellen, T., and Heidekamp, I. (2015). How Dutch employees experience freedom of learning for work. International Review of Education, 61(6), 735-759	Freedom of Learning for Worka,b
	068	Zumrah, A. R. (2015). Examining the relationship between perceived organizational support, transfer of training and service quality in the Malaysian public sector. European Journal of Training and Development, 39(2), 143-160	Perceived Organizational Support and Transfer of Training Outcomesa,b
	069	Beverborg, A. O. G., Sleegers, P. J. C., and Van Veen, K. (2015). Fostering teacher learning in VET colleges: Do leadership and teamwork matter? Teaching and Teacher Education, 48, 22-33	Leadership and Teamworka,b
	070	Kim, J. (2015). The relationships among the learning transfer system, managers' creative learning transfer, and job performance. Dissertation, Texas A and M University, College Station	Learning Transfer and Job Performancea,b
	071	Fontana, R. P., Milligan, C., Littlejohn, A., and Margaryan, A. (2015). Measuring self-regulated learning in the workplace. International Journal of Training and Development, 19(1), 32-52	Self-Regulated Learning in the Workplace Questionnaire (SRLWQ)11,a,b
	072	Milligan, C., Fontana, R. P., Littlejohn, A., and Margaryan, A. (2015). Self-regulated learning behaviour in the finance industry. Journal of Workplace Learning, 27(5), 387-402	Self-Regulated Learning in the Workplace Questionnaire (SRLWQ)11,a,b
2016	073	Tseng, L.-M., and Yu, T.-W. (2016). How can managers promote salespeople’s person-job fit? The effects of cooperative learning and perceived organizational support. The Learning Organization, 23(1), 61-76	Cooperative Learning and Support in the Organizationa,b
	074	Suzan, Z. (2016). The Relationships among Job Satisfaction, Length of Employment, and Mentoring of Nursing Faculty. Dissertation, Walden University, Minneapolis	Job Descriptive Index (JDI)6,a + Job in General Scale (JIG) (1997 Revision)5,a
	075	Hasson, H., von Thiele Schwarz, U., Holmstrom, S. Karanika-Murray, M., and Tafvelin, S. (2016). Improving organizational learning through leadership training. Journal of Workplace Learning, 28(3), 115-129	Dimensions of the Learning Organization Questionnaire (DLOQ) short version3,a,b
	076	Yoo, S. J., and Huang, W. D. (2016). Can e-learning system enhance learning culture in the workplace? A comparison among companies in South Korea. British Journal of Educational Technology, 47(4), 575-591	Dimensions of the Learning Organization Questionnaire (DLOQ) short version3,a,b
2017	077	Naidoo, K. L., Van Wyk, J. M., and Adhikari, M. (2017). The learning environment of paediatric interns in South Africa. BMC Medical Education, 17(1), q	Postgraduate Hospital Educational Environment Measure (PHEEM)a,b
	078	Smith, E., Smith, A., Tuck, J., and Callan, V. (2017). Continuity and change: Employers' training practices and partnerships with training providers. Adelaide: NCVER	Employer Survey
	079	Janssens, L., Smet, K., Onghena, P., and Kyndt, E. (2017). The relationship between learning conditions in the workplace and informal learning outcomes: A study among police inspectors. International Journal of Training and Development, 21(2), 92–112	Informal Workplace Learning Outcomes Questionnairea,b + Learning Conditions Questionnairea,b
	080	Nafukho, F. M., Alfred, M., Chakraborty, M., Johnson, M., and Cherrstrom, C. A. (2017). Predicting workplace transfer of learning: A study of adult learners enrolled in a continuing professional education training program. European Journal of Training and Development, 41(4), 327-353	Transfer of Learning Instrumenta,b
	081	Serrat, O. (2017). Building a Learning Organization. In O. Serrat (Ed.), Knowledge solutions: Tools, Methods, and Approaches to Drive Organizational Performance (pp. 57-67). Singapore: Springer Singapore	Learning Organization Model
	082	Jutz, R., Scholz, E., and Braun, M. (2017). International Social Survey Programme: ISSP 2015 - Work Orientations IV; Questionnaire Development. (GESIS Papers, 2017/7). Köln: GESIS	International Social Survey Programme (ISSP) Work Orientations III questionnaire (2005)9
	083	Kyndt, E., and Beausaert, S. (2017). How do conditions known to foster learning in the workplace differ across occupations? In J. E. Ellingson and R. A. Noe (Ed.), Autonomous learning in the workplace. (pp. 201–218). London: Routledge	Informal Learning Conditions Questionnairea,b
	084	Ferreira, A. I., Martinez, L. F., Lamelas, J. P., and Rodrigues, R. I. (2017). Mediation of job embeddedness and satisfaction in the relationship between task characteristics and turnover: A multilevel study in Portuguese hotels. International Journal of Contemporary Hospitality Management, 29(1), 248-267	Job Diagnostic Survey (JDS)1, a, b + Job Embeddednessa,b + Job Satisfactiona,b + Turnover Intentionsa,b
2018	085	Jabeen, F., Friesen, H. L., and Ghoudi, K. (2018). Quality of work life of Emirati women and its influence on job satisfaction and turnover intention: Evidence from the UAE. Journal of Organizational Change Management, 31(2), 352-370	Quality of Work Life (QoWL)a,b
	086	Malik, M. S., and Kanwal, M. (2018). Impacts of organizational knowledge sharing practices on employees’ job satisfaction: Mediating roles of learning commitment and interpersonal adaptability. Journal of Workplace Learning, 30(1), 2-17	Impacts of Organizational Knowledge-Sharing Practices (KSP)a,b
	087	Hussain, S., and Soomro, F. Q. (2018). Role of Employee Training in Enhancing Perceived Performance through competencies in Services Industry: A Study of Pakistani Banking Sector. Pakistan Business Review, 20(1), 122-136	Role of Employee Training in Enhancing Employee Performance in Banking Sector of Pakistana
2019	088	Decius, J., Schaper, N., and Seifert, A. (2019). Informal workplace learning: Development and validation of a measure. Human Resource Development Quarterly, 30(4), 495-535	Informal Workplace Learning scale (IWL)a,b
	089	Battistelli, A., Odoardi, C., Vandenberghe, C., Napoli, G. Di., and Piccione, L. (2019). Information Sharing and Innovative Work Behavior: The Role of Work-Based Learning, Challenging Tasks, and Organizational Commitment. Human Resource Development Quarterly, 30(3), 361-381	Challenging Tasks and Goal Orientationa,b+ Commitment Oriented Work Systemsa,b+ Commitment to Organizationsa,b+ Task-based Workplace Learning Scale10,a,b
	090	Kortsch, T. and Kauffeld, S. (2019). Validation of a German Version of the Dimensions of the Learning Organization Questionnaire (DLOQ) in German Craft Companies. Zeitschrift für Arbeits- und Organisationspsychologie, 63(1), 15-31	Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b
	091	Lewis, S. (2019). A Quantitative Explanatory Examination of Job Training, Job Satisfaction, and Turnover Intentions among U.S. Retail Grocery Employees. Dissertation, Capella University, Minneapolis	Job Satisfaction Scale (JSS)a,b + Turnover Intentions Scalea,b
	092	Zahrah, N., Aziz, A., and Hamid, S. N. A. (2019). Workload and Work Engagement among Nurses in Public Hospitals: Moderating Role of Religious Spirituality. Asian Journal of Miltidisciplinary Studies, 7(6), 1–10	Workload and Work Engagementa,b

Notes:

1–11 test instruments used multiple times. ^aCarrying out a reliability test; ^bcarrying out a structural factor validity

Table A2.

Exemplary assignment of test items

Item	Dimension	Content area	Category	Keyword
How many hours a week do you generally work at this company?	Presage	Framework	Job-specific conditions	Working time/ overtime
Overall the climate of my workplace helps me learn	Presage	Learning environment	Work Climate	Working atmosphere
In my organization, leaders share up-to-date information with employees about competitors, industry trends and organizational directions	Process	Social interaction	Functional inclusion	Transparency
I received a lot of guidance from experienced company members	Process	Pedagogic interaction	Support	Support from superior/coworker
Do the skills you learned in class help you with your current job and/or would they help with future jobs at this company?	Product	Generic	Professional competences	Job-specific acquired skills/knowledge

Table A3.

Keyword catalog

Dimension	Content area	Category	Keywords
Presage	Organizational conditions	Framework conditions	• Number of employees • Industry • Subsidiaries • City/county/state • Business form • Corporate success • Trade union or staff association
		Working conditions	• Working time/overtime • Leisure/holiday • Payments/rewards • Work status • Job security • Health, safety
		Learning culture	• Learning climate • Learning culture • Organization of learning
Process	Work tasks	Workload	• Negative emotions • Positive emotions • Physical stress • Psychological stress • Time pressure
		Variation of tasks	• Variety in tasks and skills • Possibility of knowledge expansion
		Autonomy of tasks	• Decision-making freedom • Scope of action • Freedom in method • Freedom in planning • Freedom in implementation • Self-reliance
		Meaningfulness of tasks	• Responsible tasks • Relevance for overall business, other departments, beyond the company
		Complexity of tasks	• Simplicity/difficulty of tasks • Specialization • Information availability
		Work requirements	• Fitting of abilities/skills • Challenge
	Social interaction	Involvement in expert culture	• Appreciation • Codetermination
		Functional inclusion	• Work organization • Transparency • Collaboration • Accountability
		Social inclusion	• Attention • Open-mindedness • Friendship/collegiality • Sense of belonging
	Pedagogic interaction	Methods/activities	• Learning and training methods/activities • E. g. E-learning, on-the-job learning, off-the-job learning, learning companies, …
		Training resources	• Materials • Time for learning • Money for learning
		Support	• Support from superior/coworker • Handling errors • Coaching/mentoring
		Feedback	• Performance appraisal • Self-reflection • Reflection with others
Product	Generic	Professional competences	• Generic acquired skills/knowledge • Job specific acquired skills/knowledge
		Satisfaction with learning opportunities	• Summarizing judgement on learning opportunities
	Job specific	Occupational identity	• Fit with the profession • Interests • Boredom • Fun • Satisfaction • Re-election of the profession
		Occupational career aspirations	• Further education and career • Influence of family/friends • Desired occupation
	Organizational level	Organizational identity	• Fit to the operation • Re-election of the company
		Organizational career aspirations	• Promotion/advancement • Continuing education/study • Plans for the future • Goals/wishes

Table A4.

Frequency and number of items, test instruments and studies per dimension, content area and category

Dimen-sion	Content area	Category	A	B	C	D	E	F	G	H
Presage	Organi-zational conditions	Framework conditions	133	3.61	21	22.34	25	27.17	E:	9	32.14
									P:	5	18.52
									E&B:	2	22.22
									S:	1	8.33
		Working conditions	298	8.08	44	46.81	47	51.09	E:	15	53.57
									P:	14	51.85
									E&B:	5	55.56
									S:	7	58.33
		Learning culture	416	11.28	39	41.49	42	45.65	E:	16	57.14
									P:	10	37.04
									E&B:	4	44.44
									S:	5	41.67
Process	Work tasks	Workload	109	2.96	30	31.91	32	34.78	E:	7	25.00
									P:	9	33.33
									E&B:	4	44.44
									S:	7	58.33
		Variation of tasks	81	2.20	33	35.11	36	39.13	E:	10	35.71
									P:	12	44.44
									E&B:	2	22.22
									S:	6	50.00
		Autonomy of tasks	140	3.80	38	40.43	44	47.83	E:	11	39.29
									P:	14	51.85
									E&B:	4	44.44
									S:	7	58.33
		Meaningfulness of tasks	40	1.08	16	17.02	18	19.57	E:	3	10.71
									P:	3	11.11
									E & B:	2	22.22
									S:	4	33.33
		Complexity of tasks	67	1.82	28	29.79	30	32.61	E:	8	28.57
									P:	8	29.63
									E&B:	2	22.22
									S:	5	41.67
		Work requirements	119	3.23	37	39.36	39	42.39	E:	10	35.71
									P:	14	51.85
									E&B:	3	33.33
									S:	5	41.67
	Social interaction	Involvement in expert culture	124	3.36	39	41.49	47	51.09	E:	16	57.14
									P:	13	48.15
									E&B:	4	44.44
									S:	6	50.00
		Functional inclusion	261	7.08	50	53.19	55	59.78	E:	17	60.71
									P:	13	48.15
									E&B:	5	55.56
									S:	9	75.00
		Social inclusion	223	6.05	50	53.19	54	58.70	E:	14	50.00
									P:	19	70.37
									E&B:	5	55.56
									S:	8	66.67
	Pedagogic interaction	Methods/activities	345	9.35	36	38.30	37	40.22	E:	18	64.29
									P:	7	25.93
									E&B:	5	55.56
									S:	1	8.33
		Training resources	94	2.55	18	19.15	24	26.09	E:	11	39.29
									P:	5	18.52
									E&B:	2	22.22
									S:	1	8.33
		Support	180	4.88	40	42.55	48	52.17	E:	12	42.86
									P:	16	59.26
									E&B:	4	44.44
									S:	7	58.33
		Feedback	280	7.59	45	47.87	50	54.35	E:	13	46.43
									P:	14	51.85
									E&B:	2	22.22
									S:	7	58.33
Product	Generic	Professional competences	104	2.82	15	15.96	15	16.30	E:	5	17.86
									P:	4	14.81
									E&B:	3	33.33
									S:	0	0.00
		Satisfaction with learning opportunities	63	1.71	22	23.40	22	23.91	E:	7	25.00
									P:	7	25.93
									E&B:	2	22.22
									S:	2	16.67
	Job specific	Occupational identity	198	5.37	36	38.30	35	38.04	E:	10	35.71
									P:	10	37.04
									E&B:	3	33.33
									S:	5	41.67
		Occupational career aspirations	5	0.14	5	5.32	5	5.43	E:	0	0.00
									P:	2	7.41
									E&B:	0	0.00
									S:	1	8.33
	Organi-zational level	Organizational identity	86	2.33	22	23.40	19	20.65	E:	2	7.14
									P:	3	11.11
									E&B:	2	22.22
									S:	3	25.00
		Organizational career aspirations	67	1.82	25	26.60	27	29.35	E:	6	21.43
									P:	9	33.33
									E&B:	2	22.22
									S:	5	41.67

Notes:

A = Number of items per category (N_i = 3,688); B = Percentage of items per category (N_i = 3,688); C = Number of test instruments with recourse to category (N_t = 94); D = Percentage of test instruments with recourse to category (N_t = 94); E = Number of studies with recourse to category (N_s = 92); F = Percentage of studies with recourse to category (N_s = 92); G = Assignment of studies to databases with recourse to category; H = Percentage of studies per discipline

Notes

1. Disputable is whether it is valid to compare workplace learning across different vocations and industries, given varying specific occupational and workplace-related characteristics. However, Janssens et al. (2017) argue that research in one area can give at least an idea of what it might look like in another area. While, Kyndt and Beausaert (2017) find in a study with nurses and military employees that workplace learning can be researched across different areas, differences must be expected.

2. Formal learning is highly organized in terms of learning content, learning objectives, time and funding and is also intentional and usually leads to certification while informal learning tends not to be organized and is unintentional on the part of the learners (OECD, 2010).

3. We would like to thank two student assistants of the project (Lisa Müller and Herbert Thomann) for their continuing help in the project during data collection and analysis.

4. While some categories have up to seven keywords (e.g. framework conditions), others have only one (e.g. meaningfulness of tasks). This was primarily necessary to ensure an unambiguous assignment of items.