Student-led sustainability transformations: employing realist evaluation to open the black box of learning in a Challenge Lab curriculum

Abstract

Purpose

While sustainability-oriented education is increasingly placing importance on engaging students in inter- and transdisciplinary learning processes with societal actors and authentic challenges in the centre, little research attends to how and what students learn in such educational initiatives. This paper aims to address this by opening the “black box” of learning in a Challenge Lab curriculum with transformational sustainability ambitions.

Design/methodology/approach

Realist evaluation was used as an analytical frame that takes social context into account to unpack learning mechanisms and associated learning outcomes. A socio-cultural perspective on learning was adopted, and ethnographic methods, including interviews and observations, were used.

Findings

Three context-mechanism-outcome (CMO) configurations were identified, capturing what students placed value and emphasis on when developing capabilities for leading sustainability transformations: engaging with complex “in-between” sustainability challenges in society with stakeholders across sectors and perspectives; navigating purposeful and transformative change via backcasting; and “whole-person” learning from the inside-out as an identity-shaping process, guided by personal values.

Practical implications

The findings of this paper can inform the design, development, evaluation and comparison of similar educational initiatives across institutions, while leaving room for contextual negotiation and adjustment.

Originality/value

This paper delineates and discusses important learning mechanisms and outcomes when students act as co-creators of knowledge in a sustainability-oriented educational initiative, working with authentic challenges together with societal actors.

Citation

Holmén, J., Adawi, T. and Holmberg, J. (2021), "Student-led sustainability transformations: employing realist evaluation to open the black box of learning in a Challenge Lab curriculum", International Journal of Sustainability in Higher Education, Vol. 22 No. 8, pp. 1-24. https://doi.org/10.1108/IJSHE-06-2020-0230

1. Introduction

Education and learning are central in addressing complex sustainability challenges and realising desirable futures (UNESCO, 2014b; United Nations, 2015; Wals, 2009). Higher Education Institutions hold a special role in sustainability transformations, both in building knowledge and capabilities and in shaping values and perspectives (Cortese, 2003; Orr, 1994; Wals and Jickling, 2002). They are also frequently central in facilitating societal processes engaging with sustainability challenges pertaining to the common good (Leal Filho, 2015; Scholz, 2017; Trencher et al., 2014). However, these institutions often lack incentives and even the ability to engage with complex sustainability challenges in society that require institutional innovation and structural change to truly create space for meaningful inter- and transdisciplinary processes (Barth et al., 2016; Holmberg and Samuelsson, 2006; Lozano, 2006).

Sustainability-oriented education initiatives such as education for sustainable development (ESD) are increasingly interested in engaging students in such undertakings. These initiatives range from adjustments of single courses to whole-university approaches that engage with sustainability challenges in society (Mcmillin and Dyball, 2009). Such initiatives rely on learning spaces that address challenges co-constructed with societal actors and higher levels of student engagement, where learning unfolds from experiencing the inherent open-ended complexity and ambiguity of sustainability challenges (Dale and Newman, 2005; Sterling, 2011; Wals and Jickling, 2002).

Despite promising transformative initiatives where students engage with societal challenges (Barth et al., 2014; Blake et al., 2013; Bodorkós and Pataki, 2009; Brundiers et al., 2010; Leal Filho et al., 2019; Munro et al., 2016; Stauffacher et al., 2006), there is a lack of knowledge of, e.g. what, why, and how students learn in such activities (Tilbury, 2016; Van Poeck et al., 2018). Additionally, there is a general challenge in mainstreaming associated curriculum models that “work” into wider education systems (Barth et al., 2016; Macintyre et al., 2018; UNESCO, 2014b).

In a recent review on student engagement and leadership in higher education for sustainability, Tilbury (2016, p. 273) notes a prevailing tendency for teachers to assume control over learning processes and view students as passive receivers of knowledge yet-to-be-applied, rather than facilitate co-creative learning where students actively produce knowledge as agents of change. She further notes that:

[…] transformative learning ambitions, often associated with education for sustainable development, have not translated well into empirical research work. The contextual dynamics of institutions and narrow interpretations of the principle of ‘student engagement’ have led to a fixation on studies to establish student satisfaction levels rather than on research to improve participation levels or outcomes.

These concerns signal a need for studies on how transformative and sustainability-oriented learning approaches are situated in higher education institutions, and how such initiatives generate relevant learning for students.

Context and process are often absent in commonly used analytical frames. However, opening up the “black box” of learning (Van Poeck et al., 2018) facilitates the creation of knowledge with contextual awareness for “scaling” pedagogies and activities that are meaningful for student learning and in support of sustainability transformations (Caniglia et al., 2016; Macintyre et al., 2018; Mickelsson et al., 2019). In-depth qualitative research into how students participate in and experience their learning processes when engaging with societal challenges is considered key towards such ends (Lotz-Sisitka et al., 2015; Macintyre et al., 2018; Tilbury, 2016).

This paper draws on realist evaluation (Pawson and Tilley, 1997) as a promising analytical frame to guide such qualitative inquiry and to open the “black box” of learning. A key concept in realist evaluation is that of “mechanism”, encompassing interplays between resources provided by the intervention (e.g. teaching methods) and associated responses (e.g. learning experiences). As such, mechanisms explain how and why a certain intervention “works” (or not): They explain how and why outcomes of interest (e.g. learning outcomes) are produced with care for context. In realist evaluation, this knowledge is captured in a set of “context-mechanism-outcome configurations”, or “CMO configurations” for short. Mechanisms of this form can be transferred across cases and context without prescribing a particular “best practice”, as the way a mechanism unfolds and manifests must draw from the particular context, participating actors and related circumstances.

The educational initiative we set out to evaluate from this perspective – Challenge Lab, or C-Lab for short – supports students in leading transdisciplinary engagement with local/regional systemic sustainability transition challenges. Placing such challenges at the centre in C-Lab, learning unfolds in interaction with researchers and societal actors. C-Lab shares most ESD ambitions (Holmberg, 2014); a previous study into three student-led cases indicated that C-Lab has the potential for meaningful student learning and contributions to sustainability transformations in society (Larsson and Holmberg, 2018). The engineering education community is witnessing a similar surge of interest in sustainability-oriented education and learning approaches (Hadgraft and Kolmos, 2020). In this community, less is known about learning experiences and processes from a student perspective (Jamison et al., 2014).

In addressing these research interests, this paper seeks to generate explanatory knowledge of learning mechanisms and associated learning outcomes for students participating in C-Lab. To this end, the paper was guided by the following research questions:

RQ1.

What learning mechanisms are at play in C-Lab?

RQ2.

How can these be conceptualised in terms of CMO configurations?

Answering these questions involved three distinct methodological moves. First, a combination of ethnographic methods, including interviews and observations, were used to elicit student learning experiences in C-Lab. Second, this data was augmented with an in-depth understanding of the program theory underpinning C-Lab (Holmberg, 2014; Larsson and Holmberg, 2018), meaning an understanding of how C-Lab “works” from a teacher perspective, to provide insight into how and why these learning experiences come about – that is, insight into learning mechanisms. Finally, literature on ESD and societal transformations was reviewed to delineate potential learning outcomes stemming from these learning mechanisms. As such, this paper makes an original contribution to the ESD literature by delineating and discussing important CMO configurations that seek to explain how potential learning outcomes may be generated via mechanisms in context.

2. Study design

This study is part of a systematic evaluation of Challenge Lab (C-Lab), established in 2014 at Chalmers University of Technology in Gothenburg, Sweden.

2.1 Research setting

C-Lab is part of a longer tradition of the university seeking to transform and build institutional capacity towards relevance on societal challenges (Holmberg, 2014; Holmberg et al., 2012). Via a preparatory course and master’s thesis lab [1], C-Lab supports and creates space for students across educational backgrounds to build leadership capabilities in engaging with societal sustainability-transition challenges in multi-stakeholder settings. A majority of students have some kind of engineering education background, but in the most recent years students from economics/environmental social science backgrounds have also participated.

A key assumption in C-Lab is that students – due to their relatively neutral and unthreatening positions in society – possess the potential to bring people together on complex “in-between” sustainability challenges, challenges that are generally “in everyone’s interest”, but “no one’s responsibility”. These challenges present contradictions, and they create tension that without proper facilitation leads to polarisation and conflict rather than mutual understanding, trust and social collaboration.

In sustainability-oriented education, engagement with authentic sustainability challenges in society can enable meaningful learning if adequate support is provided (Brundiers et al., 2010; Caniglia et al., 2016; Lönngren et al., 2019). C-Lab builds on a methodology combining “outside-in” and “inside-out” dimensions of learning and change (Holmberg, 2014), where “outside-in” is oriented towards engaging with the external world and “inside-out” towards understanding and leading oneself together with others. C-Lab operates on a local/regional level with a thematic orientation (e.g. mobility, energy, urban futures). It hosts a particular space (outside a classroom context, at a science park), a process framework [backcasting with associated tools (Holmberg, 1998; Holmberg and Larsson, 2018)], and a learner-centred facilitation [affective/cognitive care for autonomy, competence and relatedness (Ryan and Deci, 2000)]. Space, process and facilitation together seek to create an environment for students to identify (in Phase 1) and address (in Phase 2) a complex “in-between” sustainability challenge at an intervention point with potential for a sustainability transformation.

C-Lab is situated in the context of the region West Sweden, where sustainability tends to be valued, and where a general flat/non-hierarchical organisational life enables students to connect and bring stakeholders together on issues of shared concern. Most students in C-Lab come with previous experiences from an engineering education context tending to value problem-solving and technical skills – sometimes at the expense of deeper reflection on, for instance, problem-framing, ethics and social/participatory aspects of sustainability. Additionally, engineering education has instrumental tendencies that may come at the expense of personal dimensions of learning and development.

2.2 Methodological and theoretical orientation

This paper draws on an approach for evaluating programs and interventions known as realist evaluation (Pawson and Tilley, 1997). Stöhr and Adawi (2018) note that:

Realist evaluation emerged largely as a reaction to the traditional approach to evaluating interventions, using an experimental or quasi-experimental design. Rather than focusing on the question “does it work?”, realist evaluation is more theory-oriented and pivots around questions such as “how or why does it work, for whom, and in what circumstances?” This shift in analytical focus has been described as a shift from a “black box” to a “white box” approach to evaluation [References have been omitted in this quote].

This theory-oriented nature of realist evaluation requires a qualitative research approach, attending to participants’ experiences and meaning making, ideally using multiple data sources (Bryman, 2012; Danermark et al., 2002).

To guide such qualitative inquiry, realist evaluation offers three analytical constructs: context, mechanism and outcome (Pawson and Tilley, 1997). An unpacking of mechanisms – explaining how or why a certain program or intervention “works” (or not) – lies at the heart of the analytical agenda, while acknowledging the contingency of context. Such mechanisms comprise resources provided by an intervention and the resulting responses by participants in terms of reasoning, what they say or do, in this paper referred to as learning experiences (Wong et al., 2012). Knowledge from realist inquiry is conceptualised as a set of context-mechanism-outcome (CMO) configurations, explaining how and why outcomes of interest are generated, taking context into account. In this study, we seek to provide insights into learning mechanisms at play in C-Lab (RQ1) and associated CMO configurations (RQ2). A conceptual overview is provided in Figure 1.

We approach learning from a socio-cultural perspective (O’Loughlin, 1992) which – like realist evaluation – emphasises the importance of context and process for understanding learning. It argues that learning originates from social and material interaction. As such, learning involves the “whole person” in wider settings where cultural, historical and relational aspects shape ways of knowing, being, acting and becoming (Wenger, 1998). From this perspective, learning is understood as identity-shaping (Illeris, 2014), with transformative potential in creating new concepts, objects and patterns of activity (Engeström and Sannino, 2010).

2.3 Data collection

Data related to the research questions was collected using a combination of ethnographic methods. Ten semi-structured interviews (45–80 min each) were conducted in 2018 towards the end of the students’ thesis work at C-Lab. Interviews were informed by a realist evaluation procedure (Manzano, 2016) in the sense that they drew from both students’ and teachers’ knowledge and experience of the C-Lab curriculum: The interviews started with a set of open-ended questions that focused on students’ learning experiences. This was followed by a dialogue on the C-Lab curriculum and underlying assumptions, resources and activities. Here, the interviewer introduced particular concepts and ideas to gain additional insights into student experiences and significant resource-response relationship – that is, significant learning mechanisms (see further below).

Two focus group interviews (60 min each) were conducted with all 14 students from the 2018 thesis process. These interviews deliberated on how students generally experienced their thesis process, and how they experienced change – internal as well as external. The first focus group was conducted at the end of Phase 1 and the second at the end of Phase 2. These interviews were preceded by evaluation surveys, in which students had an opportunity to individually reflect upon and share experiences participating in C-Lab.

These data sources were triangulated with participant observation and fieldnotes taken throughout the students’ master’s thesis engagement, including participation in weekly one-hour “check-ins” and “check-outs” where students share thoughts, emotions and experiences from particular situations they have encountered during their thesis work throughout the week. Evaluation surveys from the preparatory master’s course (2015–2019) were included as a secondary data source.

2.4 Data analysis

The data analysis included three iterative steps. First, interview transcripts from the individual interviews were coded for the three analytical constructs underpinning realist evaluation – context, mechanism and outcome – and themes were identified for each construct. This step followed the guidelines for carrying out a (theoretical) thematic analysis outlined by Braun and Clarke (2006).

Second, to answer the first research question (RQ1), themes emerging from interviews that related to learning mechanisms were subdivided into resources (parts of the C-Lab design) and responses (associated student learning experiences). Additional resources that were part of the C-Lab design were included to complement those resources students highlighted as important. This was followed by an analytical matching process that moved back and forth between response(s) and resource(s) to arrive at a first set of proposed mechanisms. For example, student experiences leaning towards “response”, such as themes of “feeling autonomous”, “connecting with self” and “issues contacting stakeholders”, were matched with related resources such as “teacher as facilitator”, “clarification of values” and “encourage stakeholder interaction”. This matching process informed further conceptualisation of potential mechanisms that would capture significant resource-response relationships in a mutually exclusive and collectively exhaustive way. These initial learning mechanisms were tested using additional and secondary data sources to strengthen or complement what students had focussed on during the interviews, including additional comparisons with the C-Lab design for comprehensiveness and validity. This step highlighted additional differences in experiences among students, the influence of context, and interplays across learning mechanisms. In total, the interviews generated 49 themes clustered in 20 broader themes throughout the analysis, eventually resulting in seven learning mechanisms.

Finally, to answer the second research question (RQ2), these learning mechanisms were analytically aggregated in broader themes to capture what students emphasised as the main and more general learning experiences. These broader themes are also strongly informed by the literature on education for sustainable development and societal transformations, and, in an attempt to outline potential learning outcomes resulting from these learning mechanisms in context, conceptualised as CMO configurations.

3. Learning mechanisms at play in C-Lab (RQ1)

A thematic analysis of the data led to the identification of seven learning mechanisms (M1-M7) at play in C-Lab. The names chosen for these mechanisms seek to capture the resource-response relationship in a succinct way. Illustrative excerpts from the interviews (the ten interviewees are referred to as [i1] – [i10]) are used to facilitate an understanding of the learning mechanisms. In keeping with realist evaluation, it is imperative to situate mechanisms in their proper context (provided in Section 2.1). During the interviews, it became clear that the contrast with the surrounding context and previous education is often what makes students highlight certain experiences as significant. The seven mechanisms are presented below.

3.1 Students in the Centre (M1)

Students in the centre refers to learning experiences resulting from participating in a space where they are acknowledged and empowered as co-constructive agents in the learning process. They are trusted to take ownership of their work, and teachers play a “space-creating” and facilitating role. One student described such learning experiences as a shift compared to previous educational experiences:

Sometimes I feel that [in my previous education] there was a lot of focus on ‘you are supposed to learn it this way, and if you don’t then you are doing something wrong’. And I think that I have always been, well, if I cannot learn something then [the teachers] are also doing something wrong. Challenge Lab is the first place where there has actually been more of a dialogue on how we are going to learn this together and help each other, compared to previously. […] You are quite vulnerable as a student […] and then I think it is quite beneficial to feel instead that you have something of value to bring, somewhere. [i10, authors’ translation from Swedish]

Conversations that explored the extent to which students have felt at the centre largely confirmed such views, but they also stressed the importance of support in making it possible for students to be in the centre. Below is an excerpt from a student who experienced ownership but also struggled throughout, asking for more support:

It’s a sense of personal satisfaction that comes from knowing you have been in charge of what you do throughout […] And I think that’s very consistent with what the c-lab wants it to be as well. It wants leaders who are self-aware to use values they have to lead the transition and I think that’s consistent because you let them, you let them interact with stakeholders. You use the lab as a space where you can come back to if you need help, but all the rest is, the world is yours. [i8]

Support was not available all the time, so it felt kind of overwhelming, and our supervisor also has a very different style of coaching where she lets you answer your own questions/…/which at that stage was not very helpful […] we also did not communicate with her so that’s also our fault in a sense, not telling her that we’re struggling. [i8]

Apart from various forms of support, continuous dialogue into roles and responsibilities seems to be key for making student-centering work. Often, students enter the space as experts who come up with solutions to problems, rather than exercising a listening-type of leadership where they bring together stakeholders around joint issues to surface complexity and perspectives. Similarly, teachers and involved researchers need to move from positions of steering and controlling students, into coaching facilitators. In other words, student-centering does not leave it all to the students; rather, it necessitates the creation of a “safe space” characterised by trust and empowerment. Here, main resources (deployed at C-Lab to various extents) include cognitive and affecting scaffolding that meets students “where they are”, and continuous dialogue/reflection on roles and responsibility, in a broader context where students are invited to co-shape what C-Lab is and should be.

3.2 Sense of togetherness (M2)

The relations between the students in the group are important for creating a sense of togetherness. Students often referred to a sense of togetherness as a particular atmosphere of, for example, openness, diversity, respect, sensitivity and awareness. They shared experiences on togetherness as enabling expressing honest thoughts and opinions, with judgement suspended out of respect for difference, characterised by caring, open and trusting relationships. This is borne out by the following comments:

Daring to be open, I mean what openness can lead to. I mean openness in how you think, what you think and what you value/…/it does so much for group cohesiveness and […] your own comfort in a group I think has made a huge difference in this group in particular, that we made this exercise in the beginning to sit down and talk eye to eye and really open up. I think that was very cool […] because the first time I met this group I was like ‘aha, oh, shit, it is, okay […] we are different, we are very different in here. How is this supposed to work out?’ [i7, author’s translation]

I don’t like speaking in groups […] and I’m not massively comfortable with new situations. But I thought that it was, because it was quite safe and comfortable all the time, it was quite a good opportunity/…/it was a space where you could be much more expressive. And have opinions, and sort of everybody seemed to respect each other quite a lot. [i1]

In relating the learning experiences above to resources, communicative abilities are exercised at C-Lab as resources provided via training in deep listening and dialogue, where openness, trust, and sensitivity, as well as sharing of experiences, come from resources in form of surfacing personal values and strengths/weaknesses, as well as by hosting weekly-check ins and check-outs.

3.3 Values (M3)

Values provide a starting point in C-Lab, which students experienced as important in both connecting with themselves and others on a level of intent. A clarification of values in the group early on seeks to build openness and trust in the group, while also supporting students in making their own moral/ethical compass explicit to guide purposeful reasoning and action in complex situations. Students stressed the importance of values in building group cohesiveness, but less so how values have been related to handling complexity throughout their thesis work, as the following comments express:

During the value exercise I got to understand and motivate for myself why some things are important for me. I also was able to understand why other values are more important for other people. Since everyone values different things, then it is important to think about this when working with the group. [survey]

I think it [responsibility and trust] also comes from both our personal values and experiences, but that foundation is laid in phase 1, because you’re like, we’re equals, we’re here for the same thing and we are gonna work together. [i8]

Clarification of values occurs in a facilitated session where students surface, prioritise and share personal values in smaller groups via story-telling exercises. Underpinning work with values, as well as related efforts to build openness and trust in the group, is the importance of teachers in facilitating with authenticity and care:

You show that you think this is important, it feels genuine from your side, that we could feel it or that I could feel it made it become real, it became serious. I think many, especially engineers, think that such questions are “blaha”, it is so fluffy, difficult to grasp, and perhaps sometimes experienced as unimportant. So, I don’t think we should underestimate how the people facilitating it act and are experienced by others in the group. [i7, author’s translation]

3.4 Sustainability orienting (M4)

If values seek to surface importance/desirability from the “inside-out”, sustainability orienting seeks to surface the same from the “outside-in”, referring to how students experience and relate to sustainability as a normative concern and ambition throughout their work. Primary experiences of engaging with sustainability arise through the creation of a joint framework of sustainability principles during Phase 1. Students experienced this activity as empowering in creating something shared together, and they reported changed, deepened and broadened understandings of sustainability, associating it with, for instance, desirability and hope, and connecting the “outer” with the “inner”:

I think that C-Lab is a very reflective process. Understanding sustainability […] I don’t know if I would call that personal development, maybe, but understanding sustainability, its complexities, and also C-lab’s values-based approach to sustainability. That’s completely new for me and that’s something I will take forward […] your understanding of sustainability, and how you can work together, that is something that I really understood at the challenge lab, and I’m grateful for that as well. [i8]

Students also expressed the importance of not only setting the sustainability orientation but also the opportunity of really enacting it:

To see how that [sustainability] can really be applied was very interesting. Not just as a checklist – okay, we wanna be more sustainable so we use this material instead of that material, but really to get to the core of where does it all come from. And where can it be at the core of a transition, how can a transition be started. [i2]

The approach towards sustainability in C-Lab is guided by a “sustainability lighthouse” framework (Holmberg and Larsson, 2018). Resources are provided to negotiate a shared framework of guiding principles for well-being, and for social, economic and ecological sustainability, and for using the framework in the following backcasting steps in, for example, illuminating “unsustainabilities” in present systems via systems analysis and stakeholder dialogues, and for motivating transformative potential.

While sustainability is preferably approached as a central concept from a critical angle, C-Lab is also fairly open for students to make their own choices. Some groups therefore assume sustainability in their thesis objectives (e.g. scaling of urban biking or solar photovoltaics), whereas other groups address intra-organisational problems in bi-lateral constellations with less relation to wider sustainability challenges and systems change. Groups that benefit the most from explicit sustainability orienting are those that align with the C-Lab intents of engaging with “in-between” challenges that demand stakeholder dialogue and negotiation of sustainability challenges in context for progress to be made [2]. These students also tend to experience C-Lab as the most rewarding. The following (final three) learning mechanisms describe how such work is characterised.

3.5 Multi-perspectivity (M5)

Multi-perspectivity refers to how students judge and balance multiple perspectives on complex issues, including, for instance, multiple sustainability aspects, systems-related factors, problem framings, boundaries and stakeholder perspectives. At first, multi-perspectivity was experienced as overwhelming due to the multitude of aspects and angles that are brought into the process. Over time, however, multi-perspectivity became a natural part of the process, especially when students started organising their work around a central question or topic of concern around which choices can be oriented. The following comment touches on this:

I thought it was a really good thing to have it broad in the beginning, even though it was quite messy. It helped to sort of get a very broad overview of where are we really and what [are] our delimitations. So, what did we chose to focus on and why is that, and what does it always tell us that if we find something within this narrow field, what does it tell us about the whole, or the system in general, the system itself. [i2]

An immediate tendency when students (in this case) are introduced to some problem or question is to pragmatically suggest immediate solutions rather than “staying with the question” by, for example, exploring it from multiple perspectives, asking critical questions, or reflecting upon framings. Resources for maintaining multi-perspectivity include tools for structuring multiple sustainability dimensions and aspects, collectively mapping socio-technical systems and boundaries and dialoguing for surfacing diverse perspectives on pressing issues. These resources are tied together and sequenced within a backcasting framework that opens up conversations at certain stages while putting a pin in others to maintain viability. Students in C-Lab have a saying that has held up over the years: “trust the process”.

From a teacher’s perspective, trust in the process may ease tendencies of trying to control uncertain situations by reducing complexity, rather than by exploring it. However, the same trust may also lead to a letting-go of critical and reflexive engagement in the process so that tasks are performed with less oversight or ownership.

3.6 Leverage point attraction (M6)

A key “output” from Phase 1 is an identified leverage point intervention with potential for a sustainability transformation that preferably is contextually grounded in a local/regional socio-technical system, connected to an ongoing process and/or a problematic situation of importance for multiple stakeholders across sectors and perspectives and aligned with students’ own motivation, interest, and knowledge. The understanding of a leverage point as an “area of potential” in some system was initially experienced as rather fuzzy, where understanding was developed via such resources as reasoning iteratively about the aforementioned three qualities and in dialogue with staff:

We hadn’t done much or done much reading or research on the topic itself yet, so it was clear that the leverage points were not very specific up to that point [end of phase 1]. And there were quite a few and during the meeting it was more really recalling a little bit what our leverage point exactly, what are they supposed to describe, what is the sort of, the size that we can aim for of the whole thesis. So that it’s not very very broad and way too big so it was a little bit I guess just just a little bit of input, maybe think about that, and don’t think you can go there, and that you can’t really […] Maybe it was a first feedback of what is realistic. What can we really achieve? Not that it was really put into words this is too big or anything, it was more the feedback was like where do you wanna go, what do you think you can do, where do you wanna start, or how do you wanna go about it? [i2]

[Stakeholders] were saying that one instance is doing something, another instance is doing something, which is completely like conflicting with another organisation […] and like kind of that the stakeholders knew the problems that were in there, they knew the problems but it was just like one thing for instance had led very little to do, they could not influence by themselves, they knew the problem but not what to do. [i4]

We realised halfway through that we students actually have a place in urban planning. We have a place and we must dare to take this place. And when we dare to take this place it can, it will, lead to something concrete. But we must start by believing in ourselves and believing in our questions, choosing a question that is close to what motivates us. [i5]

Students with leverage points that contained the aforementioned three qualities experienced greater relevance, motivation, and meaning in their contributions. Such engagement holds the potential to address complex issues that require commitment and engagement from multiple stakeholders across sectors and perspectives. Here, various stakeholders often hold their own (partial) perspectives on some complex or larger issue of importance.

A general process among student groups holding leverage point attraction follows a pattern of systematically mapping and listening in on multiple perspectives. In doing so, students start giving shape to a more comprehensive picture where commonalities and contradictions emerge across stakeholder perspectives. The students then bring stakeholders together in dialogue and joint workshops for further deliberation on how problematic situations can be understood as well as overcome. Similarly, when leverage point attraction is weak, neither students nor stakeholders feel attraction to the topic and the contribution typically becomes more traditional and more oriented towards desk research with extrinsic (rather than intrinsic) motivation and lower levels of engagement.

3.7 Holding an exploratory “in-between” space (M7)

This final mechanism is about consistency (or maybe even equanimity) when ambitions of students’ guiding principles, values and leverage point interventions meet present reality. Students seek to maintain relevance under shifting circumstances by staying open to change and adjustments that naturally happen along the way in response to, for example, new dilemmas, stakeholder views, and the knowledge attained. This learning mechanism is referred to as the holding of an exploratory “in-between” space, borne out by comments such as these:

We learnt to call it an iterative process […] I mean I guess that’s just what it is, and I felt, or it was good to make that experience that even though it may sometimes feel like you’re actually going, or taking a lot of steps in the wrong direction rather backwards, that we’re actually still learning from that. We just need to know when to say hey, this is enough. Here we’re not really getting any further for what we’re really doing. So, this is nothing really, doing more with the way we wanna go, here we cut it off. But still to have a lot of feedback, I mean what we did was just we talked to a lot of persons, to get an understanding of the whole system. And how it works, and who is involved, and who is not, and who should be. [i2]

No matter with whom we talked, it changed the angle of our project. Or they wanted to change it in some way, and then it was difficult to say what was, who was most important in the project or what voice is the most important. And sometimes it was a bit difficult for us to find – so that we did not lose ourselves, listening only to others and what they thought – in some way find ‘okay this is what we think is important’, because after all we are the ones who are going to do the work. [i3, author’s translation]

The possibility of “holding the in-between” largely hinges on the students and on them vis-a-vis their stakeholders and supervisors, where resources are situation-dependent/contingent on, for instance, the case, students and situation. To hold the space throughout the thesis work, any issues need to be framed so that they are relevant to the participating stakeholders and to broader systems change. Further, stakeholders and supervisors have to manage to set aside their ordinary agendas to instead create space for and trust the students. One student summed it up well:

We had no problems in engaging stakeholders […] [because] we chose a relevant/pressing question that engaged many. Wherever we went and with whomever we spoke, we got lots of support and encouraging words to continue with what we were doing […] even some surprising questions from the person developing the local food provision strategy, who asked us ‘how did you know that these were the people who needed to meet?’ Which I guess is a nice acknowledgement. [i5, author’s translation]

4. Conceptualising context-mechanism-outcome configurations (RQ2)

Below, we outline potential learning outcomes from the seven identified learning mechanisms in context, underpinned by research and theory from the learning sciences and related contributions on sustainability-oriented education. These insights were conceptualised as three context-mechanism-outcome (CMO) configurations – corresponding to opening up the ‘black-box’ of learning at C-Lab. The CMOs represent overarching aspects of C-Lab that students placed value and importance on:

• Learning space (CMO1), referring to the importance of engaging with complex “in-between” sustainability challenges in society with stakeholders across sectors and perspectives.

• Learning methodology (CMO2), referring to the importance of navigating purposeful transformative change via backcasting. This involves moving through the following four steps: framing sustainable futures at the level of principles – “guiding what should be”; illuminating challenges in present systems on a basis of principles – “what is? and why?”; identifying leverage point interventions for transformative change – “what could be different? where?”; and strategic experimentation in leverage points – “what can be? how?”.

• Learning within (CMO3), referring to the importance of “whole-person” learning from the inside-out as an identity-shaping process, guided by personal values.

These CMO configurations and their associated learning mechanisms are presented in detail below.

4.1 Learning space (CMO1)

A first CMO configuration at C-lab concerns the “learning space”. This CMO configuration brings into view the importance of engaging with complex “in-between” sustainability challenges in society with stakeholders across sectors and perspectives. It gravitates around two learning mechanisms – leverage point attraction (M6) and holding an exploratory “in-between” space (M7) – and presupposes two others – students in the centre (M1) and sense of togetherness (M2). Below, we discuss potential learning outcomes stemming from these learning mechanisms in context, summarised in Table 1.

Sense of togetherness (M2) with students in the centre (M1) supports the creation of an inclusive transdisciplinary space where learning unfolds in co-creative dialogue as trusting relationships between students, teachers and stakeholders. Such relationships are prerequisites for students to be able to take ownership of engaging with complex and authentic tasks and producing knowledge where teachers cannot be “nor should be in control” (Levy and Petrulis, 2012), and essential in surfacing and reshaping deeply held assumptions and world views (Sterling, 2011; Blake et al., 2013; Winter et al., 2015). Student-centred learning processes characterised by positive relationships tend to enhance motivation, engagement, involvement, and achievement (McCombs and Whistler, 1997; Lee and Hannafin, 2016; Hadgraft and Kolmos, 2020), and when stretched into co-creative learning, inclusivity, empowerment and enhanced agency represent additional outcomes (Kuh, 2009; Tilbury, 2016; Bovill, 2019). In addition, students in such spaces tend to increase their awareness and self-regulate their own learning processes and personal development (Lee and Hannafin, 2016).

Socio-cultural perspectives on learning stress this situated nature of learning, where participation in society and authentic experiences is what makes learning meaningful (Barab and Duffy, 2000). In C-Lab, learning and knowledge production processes are driven by an ambition to shape a better future, practiced in authentic situations as engaging with complex sustainability challenges. Outcomes of interest highlighted by research into student engagement with sustainability challenges in such and similar spaces (Svanström et al. (2008) include learning by:

• Collaborating in trans- and interdisciplinary constellations to identify, integrate and value multiple perspectives and learn from each other to reshape understanding and practice (Richter and Paretti, 2009; Vilsmaier and Lang, 2015; Heiskanen et al., 2016; Biberhofer and Rammel, 2017).

• Connecting to space, fostering social responsibility, building community awareness and entering caring relationships with others (Carrington, 2011; Biberhofer and Rammel, 2017).

• Coping with ill-structured, ambiguous and poorly defined situations, including conflicting/diverging knowledges, perspectives, norms, values and interests; formulating and addressing challenges (Wals and Heymann, 2004; Howlett et al., 2016; Biberhofer and Rammel, 2017; Vilsmaier and Lang, 2015).

• Linking theory, practice and critical reflective thinking; learning to make complexity of sustainability viable and actionable in context (Tilbury, 2016; Biberhofer and Rammel, 2017; Backman et al., 2019).

• Adding authentic experiences to personal and professional development (Biberhofer and Rammel, 2017; Hannon et al., 2018; Heiskanen et al., 2016; Munro et al., 2016)

In particular, “in-between” spaces in complex sustainability challenges are understood to provide meaningful spaces for learning (Vilsmaier and Lang, 2015). By bridging disciplines, sectors, and knowledge-action fields, such spaces encourage students to change perspectives, explore otherness and create mutual understandings. In addition, opportunities are provided for reflecting on diverse values, interests, objectives and cultures.

As this paper shows, these learning processes and underlying learning mechanisms do not necessarily come about naturally, instead requiring various forms of scaffolding. This is especially so for the CMO configuration below.

4.2 Learning methodology (CMO2)

A second CMO configuration at C-Lab concerns the “learning methodology”. This CMO configuration brings into view the importance of navigating purposeful transformative change via backcasting. It seeks to explain how students build capabilities to meaningfully engage with sustainability challenges, where open-ended complexity and a sense of “overwhelmingness” are contained via structuring and sequencing of actions. By approaching backcasting as a (meta-) cognitive scaffold (van de Pol et al., 2010; Vygotsky, 1978), it can be understood as supporting students in reasoning and taking action in ways they would not in the absence of such support. This includes an overall sequencing of the work (hence spanning over all learning mechanisms) and especially for “staying with the question” as identifying, formulating and framing sustainability challenges with an explicit normative orientation in a local/regional context before moving towards interventions and solutions. This pivots around three learning mechanisms: sustainability orienting (M4), multi-perspectivity (M5) and leverage point attraction (M6). Engineering practice tends to be particularly weak towards these ends (Lönngren, 2017). Below, we discuss potential outcomes of learning from these learning mechanisms in context, summarised in Table 2.

In initiatives where students engage with authentic and complex challenges where “answers do not yet exist”, scaffolding interventions are directed towards making students autonomous (Lee and Hannafin, 2016). This is done by, for example, creating spaces and processes within which students safely can explore open-endedness. Learning outcomes from such engagement emerge from processes that may require novel resources and creation of new concepts along the way (Engeström and Sannino, 2010). As a form of “learning what is not yet there” (Engeström, 2016), learning outcomes are characterised by (increased) capabilities and agency of facilitating and giving shape to future-oriented activity that overcomes constraints of existing systems (Feagan, 2018), where students work with backcasting to surface perspectives and address complex issues in dialogue with actors not necessarily used to working together.

When introducing backcasting to engineering students, Quist et al. (2006) identify learning outcomes on increased awareness of how particular tools and methods from their previous education may be useful for sustainability transformations. This is largely attributed to the positioning of tools/methods within a broader framework that supports reflection on their potential and limitations. Holmén and Holmberg (2020) identify that work with backcasting from principles that possess potential for building capacities of “beyonding”, “broadening” and “togethering”. In a backcasting-resembling model on dialectical processes in environmental learning, Schudel (2017) provides hints into “outcomes” of learning that may emerge from associated knowledge-generation processes:

• “what is and is not”: deepening understanding of sustainability issues and underlying causes;

• “what could be”: critical and sympathetic thinking that positive change is possible;

• “what should be”: reflecting ethically upon priorities, choice and desirability; and

• “what can be”: uniting theory and practice on what works in context.

The “wicked” nature of sustainability challenges demands careful problem formulation and boundary setting, including critical engagement with framing, values and assumptions (Rittel and Webber, 1973). However, most sustainability-oriented education initiatives tend to neglect this important dimension of learning by giving ready-made problems/challenges to students (Beckman et al., 2012; Le Grange, 2011). This may be particularly prevalent in engineering education contexts with tendencies to preclude alternative perspectives and worldviews (Lönngren, 2017). Beckman et al. (2012) find that important skills in learning problem formulation from a perspective of experiential learning and design include empathising, synthesising, diverging and iterating in communication with others. Using these categories as “heuristics”, potential learning outcomes in C-Lab can be translated into learning to empathise, understanding diverging stakeholder stories and perspectives on problematic situations; learning to synthesise, framing what is being heard in relation to, for instance, sustainability ambitions, system representation and self; learning to diverge, opening up and exploring a range of possibilities; and learning to iterate, systematically testing concepts in practice in communication with others.

4.3 Learning within (CMO3)

A third and final CMO configuration at C-Lab concerns “learning within”. It brings into view whole-person learning from the inside-out as an identity-shaping process, guided by personal values. It primarily encompasses the following learning mechanisms: students in the centre (M1), sense of togetherness (M2) and values (M3). As facts alone are insufficient for changing the world, this dimension of learning is increasingly being stressed in the literature (Joyner Armstrong et al., 2016; Wamsler, 2020). A focus on whole-person learning emphasises self-direction from one’s own inner motivation and curiosity in relation to others connected with one’s own life, values and identity (Freire, 1970; Grauerholz, 2001; Sterling, 2011). Below, we discuss potential outcomes of learning from these learning mechanisms in context, summarised in Table 3.

In C-Lab, inside-out learning opens up for students to surface meaningful purpose and build intrinsic motivation, aligned with their own values. Students witness a connection to the learning space and each other where they are seen as people and feel important, in an open, trusting and respectful atmosphere. Sustainability-oriented learning processes that emphasise inner and personal spheres of transformation highlight learning outcomes towards increased self-efficacy and confidence (Brower, 2011), motivation, agency and commitment to action (Bovill, 2019; Munro et al., 2016; Murray, 2018; Tilbury, 2016) as well as meaning and hope (Hooks, 2003; Wamsler, 2020).

Research into interdisciplinary student collaboration finds barriers relating to students’ abilities to recognise the role of their own disciplines in contributing to some complex issue, as well as value contributions from other disciplines, including structural constraints in doing so (Hannon et al., 2018; Howlett et al., 2016; Richter and Paretti, 2009). Richter and Paretti (2009) among others suggest that explicit reflection upon disciplinary strengths and limits during team formation has potential in overcoming some of these issues. Experiences from C-Lab point towards values as an important mechanism in supporting collaboration that critically reflects upon and crosses boundaries.

5. Conclusions

This paper set out to open the “black box” of learning at C-Lab via a realist evaluation approach. Drawing on a socio-cultural perspective on learning and using ethnographic methods, three context-mechanism-outcome (CMO) configurations of importance to students in developing capabilities for leading sustainability transformations have been conceptualised. These are learning space, referring to the importance of engaging with complex “in-between” sustainability challenges in society with stakeholders across sectors and perspectives; learning methodology, referring to the importance of navigating purposeful and transformative change via backcasting; and learning within, referring to whole-person learning from the inside-out as an identity-shaping process, guided by personal values. These CMO configurations tie together seven interrelated learning mechanisms that together seek to explain what works, for whom, why and in what circumstances in a C-Lab curriculum in context (summarised in Table 4).

5.1 Implications for practice

Initiatives with transformational sustainability ambitions where students engage with complex societal challenges in society, such as C-Lab, seek to host inter- and transdisciplinary spaces that stretch established roles among university, society, students and staff. In such initiatives, academic traditions with disciplinary excellence and objectivity tend to conflict with societal responsibility and relevance. Previous research has established that such curriculum initiatives require leadership and supportive institutional embedding, including incentives and capacities to educate, learn and value deep engagement with societal actors (Astin and Astin, 2000; Barth and Rieckmann, 2012; Lozano et al., 2015; Tilbury, 2016). One way to create space for such educational initiatives in the context of an established university structure is to focus on their complementary roles in, e.g. exploring and experimenting with new ways of “doing” education, rather than immediately challenging and replacing the established structure. Concepts, knowledge and experience from initiatives that work (and not) can inform wider organisational learning processes in re-shaping the institution and associated education in becoming more relevant for the future. Such an approach to institutional change is organisationally “ambidextrous” in the sense that it balances processes of exploitation and exploration (O’Reilly and Tushman, 2013). A concrete example of how such strategy enabled an emergence of C-Lab within its established institutional context is presented in Holmberg et al. (2012), and additional reports on community and societal value can be found in Larsson and Holmberg (2018).

The C-Lab curriculum model shares important features with well-established pedagogical models, such as service learning, problem- and project-based learning, transdisciplinary case studies and experiential learning. Depending on initiative and context, our findings may add to existing educational initiatives and future developments: Notably, students placed value on not only addressing challenges but also identifying them by means of contextual grounding in dialogue with stakeholders underpinned by explicit normative orientation. Also, students placed value on being in the centre of the engagement process, inviting actors across sectors and listening in on multiple perspectives around shared challenges and questions of concern. Further, scaffolding and methodological support via backcasting were seen as key when engaging with complex sustainability challenges. In short, the three CMO configurations with their seven interrelated learning mechanisms package knowledge and experience in a way to inspire design, development, evaluation and comparison of such and similar transdisciplinary sustainability-oriented curriculum models across institutions, while leaving freedom for contextual negotiation and adjustment. This form of result also allows for a “cherry-picking” of particular activities, but attention needs to be paid to what extent those activities might lead to different results when not part of a similar whole.

5.2 Implications for research

C-Lab supports students in developing leadership capabilities to engage with complex sustainability challenges in society. While leadership is central in C-Lab, it is generally absent in sustainability-oriented education (Tilbury, 2016) and in prevalent competence frameworks (Wiek et al., 2011). The CMO configurations developed here highlight how important capabilities for leading sustainability transitions are generated from learning mechanisms in context rather than being predetermined by design. Further research would do well to explore and connect generative mechanisms, leadership capabilities and sustainability competence frameworks to support developing further knowledge on curriculum models with transformational sustainability ambitions (Chikamori et al., 2019).

Realist evaluation claims to be a promising analytical framework for opening up the “black box” of learning. However, it is also demanding in, for example, distinguishing context from mechanism, understanding dependencies among mechanisms, and judging various levels of outcomes and their mutual dependencies. Realist evaluators generally struggle with such issues (Alvarado et al., 2017; Jolly and Jolly, 2014). Less resource-demanding evaluation might focus on, for example, assessing to what extent students have benefited from experiences assumed important by curriculum designers. Alternative conceptual frameworks of use in unpacking the how and why of learning in such curriculums could include a focus on situation, activity, and experience (Wenger et al., 2011) or condition, process and outcome (Rodríguez Aboytes and Barth, 2020) in contrast to context, mechanism and outcome.

A key idea in C-Lab has been to create space for learners to lead and leaders to learn – and to think big, start small and act now. The UNESCO roadmap for Education for Sustainable Development (ESD) (UNESCO, 2014a) acknowledges that students can play a central role in all five prioritised action areas if empowered (Area 4) within a transformative learning environment (Area 2). Under such conditions, students are in a favourable position to support systemic change (Area 1) and accelerate sustainable solutions at local levels in multi-stakeholder settings (Area 5). By creating space and supporting such co-creation processes, capacities of educators to be involved in ESD activities may also be enhanced (Area 3). The learning mechanisms with their associated CMO configurations outlined in this paper should be possible to adapt in giving shape to such educational initiatives where space is created for students to develop leadership capabilities in transformative engagement with complex sustainability challenges in society, in line with UNESCO’s ambitions and possibly beyond.

Figures

Figure 1.

An illustration of the approach to evaluating C-Lab, focusing on the relationship between the three analytical constructs underpinning realist evaluation: context, mechanism and outcome

Table 1.

CMO1 at C-Lab – learning space – pivots around the importance of engaging with complex “in-between” sustainability challenges in society with stakeholders across sectors and perspectives

Context	Mechanisms	Potential learning outcomes
Institutional valuing of inter- and transdisciplinary work with adequate supervision that trusts students	Sense of togetherness (M2) with students in the centre (M1), gathering around shared intents	Empowerment, agency, and increased awareness/regulation of one’s own learning processes
Master’s thesis providing freedom in the curriculum for students to engage with open-ended issues	Leverage point attraction (M6) holding an exploratory “in-between” space (M7) in multi-stakeholder settings anchoring sustainability challenge of interest in context	Learning to […]
Willingness of stakeholders to participate		Collaborate and build relationships in inter- and transdisciplinary constellations connecting with local/regional space and sustainability issues of concern
		Make sense of, engage with and contribute to addressing complex sustainability challenges in ill-defined situations by taking purposeful action in context

Table 2.

CMO2 at C-Lab – learning methodology – pivots around the importance of navigating purposeful transformative change via backcasting

Context	Mechanisms	Potential learning outcomes
Engineering education setting emphasising problem solving with less attention to problem framing and normative considerations	Sequenced moves within backcasting framework weaving together all learning mechanisms for containing complexity and any sense of overwhelmingness […] with particular attention to “staying with the question” as […] sustainability orienting (M4), multi-perspectivity (M5) and leverage point attraction (M6) where sustainability challenges are identified, formulated, and grounded in local/regional context	(Increased) capabilities and agency in facilitating and giving shape to future-oriented activity that questions and overcomes constraints of existing systems Identify, frame, and formulate complex challenges in structured and systematic ways (synthesise) Understanding of sustainability issues, underlying causes and that positive change is possible (empathise) Ethically reflect upon choice and desirability in unity of theory and practice (diverge and iterate) Positioning tools and methods in broader framework including awareness of their respective aims and limitations

Table 3.

CMO3 at C-Lab – learning within – pivots around the importance of whole-person learning from the inside-out as an identity-shaping process, guided by personal values

Context	Mechanisms	Potential learning outcomes
Educational institution with tendencies towards instrumental soley “outside-in” approaches to education and learning Structural constraints for collaboration across borders	Meaningful purpose and intrinsic motivation aligned with one’s own values (M3) Students in the centre (M1) and sense of togetherness (M2) fostering relations of openness, trust, and respect	Self-efficacy and confidence Motivation, agency, and commitment to action Meaning and hope Valuing contributions across knowledge fields and perspectives

Table 4.

A Summary of the seven learning mechanisms (M1-M7) at play in C-Lab, grouped into three CMO configurations

C-Lab supports students in meaningfully leading on sustainability transformations through […]
CMO1		CMO2		CMO3
Learning space		Learning methodology		Learning within

Encompassing the following learning mechanisms:
M1	M2	M3	M4	M5	M6	M7
Students in the centre	Sense of togetherness	Values	Sustainability orienting	Multi-perspectivity	Leverage point attraction	Holding an exploratory in-between space
						[…] in context

Notes

1. C-Lab includes a preparatory 7.5 ECTS course “Leadership for sustainability transitions” (40 students from different educational programmes), where key concepts are introduced, practiced on campus-related challenges, and reflected upon from a leadership perspective, followed by a 30 ECTS master’s thesis lab (20 students) where the students work with local/regional issues. An additional course is being developed within the C-Lab curriculum, “Systems interventions for sustainability transitions”, starting in the fall 2020 and replacing “Phase 1” that is currently being done within the scope of the thesis lab.

2. Three in-depth examples of such cases are presented in Larsson and Holmberg (2018).