Ethical challenges of researching emergent socio-material-technological phenomena: insights from an interdisciplinary mixed-methods project using mobile eye-tracking

Abstract

Purpose

This paper aims to discuss research ethics in mixed-methods research (MMR) and MMR development with a focus on ethical challenges that stem from working with technical instruments such as mobile eye-trackers.

Design/methodology/approach

The case of an interdisciplinary mixed-methods development study that aimed at researching the impacts of emerging mobile augmented-reality technologies on the perception of public places serves as an example to discuss research-ethical challenges regarding (1) the practical implementation of the study, (2) data processing and management and (3) societal implications of developing instruments to track and understand human practices.

Findings

This study reports challenges and experiences in ethical decision-making in the practical implementation of the study regarding the relationship to research subjects, the use of mobile research instruments in public places and the interdisciplinary cooperation among research team members. Further, this paper expounds on ethical challenges and recommendations in data processing and management and with a view to societal implications of method development and the aspirations of transdisciplinarity. This study concludes that institutionalized ethics need to become more flexible, while applied ethics and reflection must make their entry into university curricula across disciplines.

Originality/value

Complex interdisciplinary mobile and mixed-methods projects that involve sensors and instruments such as mobile eye-trackers are on the rise. However, there is a significant lack of engagement with practical research ethical challenges, practices and requirements in both mixed-methods and method-development literature. By taking a context- and process-oriented perspective focusing on doing ethics, the paper contributes a concrete empirical case to these underdeveloped fields.

Citation

Kaufmann, K., Bork-Hüffer, T., Gudowsky, N., Rauhala, M. and Rutzinger, M. (2021), "Ethical challenges of researching emergent socio-material-technological phenomena: insights from an interdisciplinary mixed-methods project using mobile eye-tracking", Journal of Information, Communication and Ethics in Society, Vol. 19 No. 3, pp. 391-408. https://doi.org/10.1108/JICES-01-2021-0007

1. Introduction

In today’s world, researchers are faced with increasingly complex digital and hybrid mobile phenomena such as locative media and mobile augmented reality (AR) that once again challenge empirical research methodologies (Humphreys, 2013; Liao, 2019). The use of mixed-methods research (MMR), new digital tools and instruments and multi- and interdisciplinary competence are seen as a solution to this rising complexity of blurring boundaries and the ensuing challenges (Plano Clark et al., 2008, p. 365; Hesse-Biber and Johnson, 2013). As a result, not only the complexity of the object of research but the methodological complexity of research projects is becoming increasingly higher (Cheek et al., 2015). However, a method is much more than what seems to be meant at first sight. Rather, “what we call simply ‘method’ is a multilayered set of inductive and non-linear processes, guided by the context and research questions” (Markham, 2006, p. 46). This makes methodical quality an ethical matter, as Mark and Lenz-Watson (2011, p. 199) point out (Cain et al., 2019).

It is noticeable that there is also an increasing focus on research ethics in research institutions, partially motivated by increased attention being paid to the topic by funding bodies and the interest of institutions in avoiding litigation risks and reputational damage. This phenomenon can be evidenced in the increase of research-ethics bodies across disciplines. However, these often heavily regulated ethics environments are problematic, as Markham (2018, p. 2) stresses: “[e]rror avoidance and concept-driven models, which lie at the heart of all regulatory guidelines and procedures, fail repeatedly in contemporary research situations”. The standard operating procedures of research-ethics review bodies, which focus on making reviews based on prior as well as current research and guidelines, are not conducive to identifying or addressing new concerns beyond those that can be anticipated and/or those that have become standard issues (Dawson et al., 2019). What today’s research needs is a “process- and context-oriented approach” (franzke et al., 2020, p. 4) that views ethics “as a dialogic process rather than a set of values or principles” (Markham, 2006, p. 50) and complements traditional criteria for quality such as validity and reliability with responsibility and accountability (Markham, 2006, p. 50; Genus and Sterling, 2018). Such a shift would appear to imply a broader change in our understanding of what it means to do research ethics as it is practised in standard ways by ethics-review bodies, research institutions and individual projects. In making a case for retrospective review, Dawson et al. (2019, para. 2) point out that, “[t]he contemporary review process lacks ways to develop the reflexivity and ethical sensitivity necessary to promote decision-making skills during ethical challenges and dilemmas” (Walker et al., 2013). A useful distinction made by Guillemin and Gillam (2004) between ethics in practice, which refers to dealing with issues that surface in day-to-day research work and procedural ethics, which relates to compliance processes, makes a similar point. Following the notion that “ethics as inextricably interwoven with a method often leads to better research” (franzke et al., 2020, pp. 4–5, emphasis in original), an ethics discussion seems even more urgent in the field of methodological innovation (Wiles and Boddy, 2013, p. 2; Cain et al., 2019, p. 8).

However, until now, it is rarely acknowledged that method innovation is always also an inherently ethical endeavour (Nind et al., 2013; Markham et al., 2018). Particularly in the comparatively new strand of MMR, the discussion on ethics and reflexivity is waiting to be shaped (Plano Clark et al., 2015; Cain et al., 2019; Collins et al., 2013; Evans et al., 2011). According to Hesse-Biber and Johnson (2013, p. 106), the dynamic character of MMR makes it essential for researchers “to continually reflect on their ethical standpoints throughout their research project, as they may encounter thorny ethical issues after a [MMR] project is in progress”. Demands on MMR also appear to be higher than those on mono-method studies as more (and therewith less controllable) dimensions affect the implementation of research, making it more difficult to consider possible impacts e.g. on participants. Furthermore, MMR often combines methods in unique and previously unexplored or at least less explored, ways. Researchers applying MMR, thus more often find themselves engaged in method development at the same time.

With this article, we want to contribute to this underdeveloped discussion by reflecting on the research-ethical challenges of our interdisciplinary mixed-methods development project that includes mobile eye-tracking. Eventually, this contribution adds to three – in our case inseparable – ethical debates: (1) MMR research ethics, (2) research ethics of methodological development, and (3) research ethics in projects that use technical instruments.

By discussing ethics in our project, we seek to leave our “comfort zone” and think outside our “methods box”, as called for by Hesse-Biber and Leavy (2008). Hence, we aim to contribute to the discussion on “very real-world ethical challenges faced by empirically-oriented researchers across multiple disciplines” (Ess, 2020, p. 3), which is driven, among others, by the Association of Internet Researchers (AoIR). As a community, AoIR brings to life the underlying idea that all empirical endeavours can benefit from acknowledging the ethical dilemmas we face in our research, not as failures but as signs of strength, striving to do better research (Ess, 2020). In so doing, AoIR exemplifies the virtues of being good – ethically reflective – researchers and what it means to research with “an amalgam of consciousness, mindfulness, honesty and sensitivity”, as Markham (2006, p. 43) aptly puts it.

Firstly, we look into the relationship between MMR development and ethics before presenting our ongoing interdisciplinary project and the data collection we have recently conducted. We then analyse three dimensions of research-ethical challenges we have identified to be facing in the conduction of our project: the research ethics (1) in the practical implementation of the study design, (2) concerning data processing and management, and (3) with a view to the societal implications of developing instruments to track and understand human practices. After that, we discuss the main recommendations from our case and conclude with an overview of the issues involved in doing ethics under conditions of complexity and uncertainty.

2. The ethics of mixed-methods research development

MMR has established itself as the third strand besides entirely qualitative and entirely quantitative research (Hesse-Biber and Johnson, 2013). The combination of existing approaches, however, leads to new challenges: “[w]hat troubles and puzzles researchers are, firstly, the trade-offs and balances of guiding principles and practices and, secondly, the unexpected developments that occur in conducting research or other human activity” (Preissle et al., 2015, p. 146). Despite the enormous growth of the practice of MMR in recent decades (Preissle et al., 2015, p. 157), issues of ethics and reflexivity in MMR are rarely discussed (Cain et al., 2019, p. 9). This may also have to do with the preponderance of quantitative-dominant MMR studies in many disciplines (Collins et al., 2013, p. 281) and the tendency of researchers to “write reflexivity out of reports of the research process” (Cheek et al., 2015, p. 752) due to limited word counts.

Yet, especially in MMR, scholars should “address their ethical puzzles” (Preissle et al., 2015, p. 146), as the complex MMR designs “might complicate matters of reflexivity and ethics” (Cain et al., 2019, p. 2). Describing ethics and reflexivity further is of paramount importance because “high-quality methodological practice requires reflexivity and demonstration of ethical inquiry to establish credibility and legitimation” (Cain et al., 2019, p. 1; see also Evans et al., 2011, p. 283). Ethical challenges in the “messy complexity” (Cheek et al., 2015, p. 760) of MMR and MMR development are indeed manifold. On the one hand, they concern the ethical measures that must be taken with a view to all involved individuals such as the participants, researchers, stakeholders and their dignity, well-being and privacy rights. On the other hand, this refers to the data management of mixed-methods data sets, the dissemination of results, the implications for socio-technological futures and their potential impacts on future research and policy practices. Where MMR does not only combine established approaches but aims to integrate new technological tools and instruments, ethical and societal implications are even harder to assess.

In the following, we want to share our experiences with the ethical challenges that came up in the course of our interdisciplinary mixed-methods development project “The Digital, Affects and Space” (DigitAS), which involved the instrument of mobile eye-tracking. The core of the empirical project was the implementation of a quasi-experimental field study, which combined qualitative research methodologies with mobile eye-tracking to research people’s in-situ use of digital augmented media in public parks and its influence on their perception of such places. Combinations of mobile eye-tracking in the context of MMR designs have increasingly been applied for a wide variety of different research purposes (Wang and Sparks, 2016; Hoy and Levenhus, 2018; Jankowska et al., 2018; Stark et al., 2018). MMR designs using eye-tracking are usually applied in lab conditions, whereas field experiments (Vater et al., 2019) are comparatively rare. Given the newness and experimentation with mobile eye-tracking in MMR, there is so far a notable lack of discussions on the concomitant ethical challenges and of recommendations for how these can be tackled.

3. Case study: The Digital, Affects and Space (DigitAS)

3.1 The DigitAS project: MMR development for research into the emerging socio-technological entanglements of everyday life

Digital media technologies have been a “natural part” (Cascone, 2000, p. 12) of people’s everyday lives for quite some time now. Evolving digital media technologies are subtly co-producing social life and entangled socio-material-technological spaces (Kitchin and Dodge, 2011; Sumartojo et al., 2016; Bork-Hüffer et al., 2021). Thereby, technologies change practices and, by doing so, also societal value systems (Genus and Sterling, 2018). Mobile AR technologies are a case in point, bringing the digital into public space and altering, as previous digital innovations did, established concepts such as privacy (Kudina and Verbeek, 2019). AR is a field of technologies that import digital images into field settings “in such a way that the virtual content is aligned with real-world objects and can be viewed and interacted with in real-time” (Dey et al., 2018, p. 1). Technologies that produce augmented realities and, subsequently, augmented spaces such as smartphones, smartwatches and smart glasses, are increasingly important in the everyday lives of many people (Boeckler, 2014).

Such technologies also increase the complexity of people‘s perceptions of the world (Lemos, 2008; Malpas, 2008; Bork-Hüffer, 2016; Dey et al., 2018; Felgenhauer and Gäbler, 2018). Given both the huge variety of new technologies and their constant evolution and innovation, the state of methods to research these digital technologies’ impacts on the perception and making of spaces is lagging. Furthermore, there is an urgent need for methodologies that analyse practices, subjects and technologies while interacting and unfolding in field settings (Ricketts Hein et al., 2008; Verne, 2012; Merriman, 2014; Spinney, 2015). Against this backdrop, we developed the DigitAS research project, which pursued the following main research question: what is the potential of the mobile mixed method(ologie)s for researching the effects of digital media on the affective-emotional experience of public places?

We successfully obtained funding from the go!digital Next Generation funding scheme for Digital Humanities research by the Austrian Academy of Sciences to pursue these objectives. The interdisciplinary team consists of researchers from the fields of geography, geoinformation science, technology assessment and communication studies. We also established an interdisciplinary scientific advisory board (SAB) including a research ethics expert to support our multi-layered project.

3.2 The DigitAS MMR design and methodology

Based on current methodological debates, we chose a research design that integrates established representational qualitative methodologies with more-than-representational mobile eye-tracking in a quasi-experimental field study. It consisted of three data-collection steps: (1) initial narrative-biographical interviews; (2) mobile eye-tracking in an outdoor quasi-experimental set-up (i.e. an experimental design without random assignment of experimental units to treatment conditions; see Mark and Lenz-Watson, 2011, p. 187); and (3) retrospective think-alouds based on videos recorded by the mobile eye-tracker:

1. Representational methods such as narrative-biographical interviews empower subjects to narrate their agency and positions (see, on the importance of these, DeLyser, 2010; Hitchings, 2012). In the project, the initial narrative-biographical interviews aimed to investigate the research subjects’ own reflected, verbalizable accounts – among others, concerning their outlook and experiences with public places and digital media use as well as their personal backgrounds, all of which potentially played a role in their perception of place and digital media choice and use.

2. Mobile eye-tracking: mobile wearable eye-trackers such as Tobii Pro Glasses 2 encompass audiovisual recording with a front-mounted camera, which provides documentation of the user's general visual frame. Within this frame, the eye-tracking technology follows and records the visual fixation of attention by the person wearing the eye-tracker, allowing for an analysis of fixations, attention allocation and subsequent calculations of cognitive indexes. The device also offers audio recording, an accelerometer and a gyroscope. In the project, eye-tracking should help to foreground elements of the affective, unconscious and pre-cognitive perception of public places and digital triggers (Bradley et al., 2008; Granholm and Steinhauer, 2004; Uzzaman and Joordens, 2011).

3. Retrospective think-aloud aims at elicitation by asking participants to comment on videos that have captured their experiences during a certain situation (Konrad, 2010). In this way, the approach allows an in-depth reconstruction of situations experienced by participants. Retrospective think-clouds have been applied as part of MMR designs that involve eye-tracking (Hoy and Levenhus, 2018; Vater et al., 2019) to elaborate self-perceived elements of a subject under investigation. In our study, we showed the videos of their walks as recorded by the built-in world camera of the mobile eye-tracker to the respective participants in a quiet room immediately after their walks. We then asked the participants to comment – think aloud – on everything that had been going through their minds during their walks, which were shown in the recorded videos to elicit their conscious perception of the public places.

3.3 The mobile data-collection process

For our study, we chose two highly debated Austrian public parks (the Venediger Au-Park in the Austrian capital of Vienna and the Rapoldi Park in Innsbruck, the capital of the state of Tyrol) as research sites. Media and public discourses reflect the parks’ controversial history, particularly for gang and drug activities and subsequent safety measures such as the installation of security cameras.

The sample consisted of students studying at Tyrolean (n = 10) or Viennese Universities (n = 10), 19 to 27 years of age, with an equal share of female and male participants. They were recruited via local internet platforms and we selected subjects who were new to the city or, in the case of Vienna, at least unfamiliar with the district where the park was located.

For the quasi-experiment, participants were asked to walk twice along a preselected path in a park. Participants first walked along the path without digital triggers. Subsequently, they walked the same path and received digital triggers relating to the park on a smartphone they were carrying. The digital triggers consisted of original textual social-media content collected from popular social media platforms in Austria (YouTube, Google Maps, Instagram, Twitter, Facebook and Yelp), i.e. content which would also have been available to users if they had searched for the park’s name on social media platforms on their own. The content had previously been collected, categorized and analysed by the research team and we selected suitable and comparable content about both parks as digital triggers, thematizing equal shares describing negative aspects of the parks such as littering and alcohol and drug consumption and positive aspects such as playgrounds, recreation and sports. For each digital trigger, we combined original social-media content with a photo we had taken from the preselected paths in the parks to make the connection between the social media content and the immediate park surroundings more obvious to the participants (Figure 1). Throughout both walks, subjects wore the mobile eye-tracker Tobii Pro Glasses 2 (MacInnes et al., 2018), which measured and recorded their bio-sensing responses and their engagement with the public park and/or media content. In addition, a global positioning system unit was handed to the participants to track their position during their walks for a later experimental georeferencing of the mobile eye-tracking data (for results, see Kollert et al., 2021). In the retrospective think-aloud, the more conscious, verbalizable emotions and their changes between the walks were elicited based on the video recordings.

Data collection took place in September and October 2020. Participants received a financial remuneration of €60 after completion of all data-collection steps. The material is subjected to parallel analyses (qualitative content analysis; visual and quantitative eye-tracking analysis) and combined for method comparison. As a further step, we are going to conduct a scenario-building exercise, using a Delphi-scenario method (Nowack et al., 2011; Wright et al., 2013). Here, we will discuss possible future developments regarding urban planning and park design in the digital age with relevant experts and stakeholders (including representatives of the city administration and planning, policymakers, community organizations) based on the empirical results.

4. Research-ethical challenges in the conducting of our project

In the following section, we discuss ethical challenges and dilemmas we have been facing in the project, to contribute to the recognized need (as discussed above) for “transparency through the use of reflexive practice” (Preissle et al., 2015, p. 158) in MMR (Cain et al., 2019; Collins et al., 2013). Initially, following Markham’s (2006, p. 48; Preissle et al., 2015) recommendation, we sought ethical advice from one of the authors who is a research-ethics expert at an Austrian university and a member of the project’s SAB. Also, we applied for and received ethical approval from the research ethics committee of the University of Innsbruck before the commencement of the data collection. Due to the COVID-19 pandemic, we had to postpone our data collection. We incorporated the necessary pandemic precautions into the ethics assessment application, which we submitted and which was approved in early summer 2020, before collecting data in early autumn 2020. When applying for the ethical approval, we provided standard documentation required by most human-research-ethics committees and as was expected by the review committee of the University of Innsbruck, including a detailed explanation of the research design, drafts of the interview guidelines, a participant information sheet and the declaration of informed consent detailing the process of data collection, the kind of data that were to be collected and what would happen with the data after the completion of the collection. The university ethics committee, as in the case of most similar review boards, followed the typical anticipatory, prospective framework of review to prevent, among other risks, risk of litigation and did not foresee the need for follow-up and/or continued guidance after approval. In other mixed-methods contexts, current review procedures have been found to be limited concerning their ability to identify and address concerns beyond a limited standard set of issues or to respond to dynamic settings. In human-computer interaction (HCI), which relies on active user participation, Frauenberger et al. (2017), call attention to the poor fit between the fluidity of methods used and the rigidity of formal ethics assessing those methods. They claim that “situated, explorative and design-oriented HCI projects raise issues of ethical importance that formal ethics is struggling to manage” (p. 220). As we saw above in the introduction, general codes may not sufficiently cover the actual ethical challenges or help identify potential new ethical issues as they arise in doing research and their relevance for actual research practice has been questioned (Guillemin and Gillam, 2004).

In the following, we discuss the challenges in conducting our project based on distinguishing three dimensions of ethical concerns that were to be considered: the requirements of research ethics (1) in the practical implementation of the study design, (2) concerning data processing and management, and (3) with a view to the societal implications of developing instruments to track and understand human practices.

4.1 Research ethics in the practical implementation of the study design

Even though we complied with the expectations and requirements of the ethics committee and tried to anticipate the main ethical concerns in the planning phase of the empirical project (Preissle et al., 2015, p. 153), once the project had started, we faced several challenges in the practical implementation of the MMR study design.

Research subjects, informed consent and respect for persons.

Most prominent in an MMR design that collects data with human research subjects is the ethical imperative to do no harm. Among the issues that can contribute to following this imperative is the careful dealing with participants. Indeed, “the selection and involvement of participants is the […] arguably most important, ethical consideration for mixed methods sampling” (Preissle et al., 2015, p. 150). Two cornerstones of research ethics – ethics review and informed consent – have been established to safeguard participants’ safety and ensure the appropriateness of research, as well as for the purposes of “protect[ing] the (deontological) norms of autonomy and equality” (franzke et al., 2020, p. 8, emphasis in original). According to Preissle et al. (2015, p. 151), informed consent should ensure that “participants understand how their role fits within a mixed methods study”. In experimental designs, ethical precautions such as informed consent need extra attention, as “experimental intervention intentionally alters the participant’s environment and can produce unintended and unforeseen consequences” (Teele, 2014, p. 120) and data.

Aware of the intervention we were planning in our quasi-experiment (Section 3.2), we put particular effort into the wording of the informed-consent sheet which was later also approved by the research ethics committee. It was supposed to advise individuals particularly sensitive to negative media content against participating, in the spirit of risk-averse ethics, while hopefully not overly deterring interested candidates from taking part in our experiment. Potential participants who contacted us in response to the study announcement were asked about their characteristics relevant to recruitment (such as area of residence, possession of a smartphone, field of study) were sent the participant information sheet and the informed-consent declaration form via email and invited to get back to us in case of questions as well as to inform us of their continued interest in participation. Both sheets were then also printed and handed out to the participants in person, giving them sufficient time to read the documents and ask questions on the occasion of the first interview before signing the consent form on-site. None of the participants refused to participate based on the informed – consent form. However, after receiving the material via email, a few female candidates made use of the possibility to ask questions about the level of violence shown in the media content: even so, those few did not opt-out. Still, the self-selection process of participants caused by our warning may have led to a potential bias towards excluding those individuals who may be particularly affected by digital media, and thus those who would potentially have produced the societally most relevant results in the context of this research object – and who might even have had the most to contribute and/or to gain (Valentine, 2005; Preissle et al., 2015).

In addition to the detailed information in the run-up to the data collection, we conducted a debriefing with each participant directly following the think-aloud as the final data-collection step. In the debriefing, we described the project’s background, the research object and the various steps of the project. Thus, the debriefing was not limited to the quasi-experiment itself but gave participants insight into the whole project, including still-pending parts of the project, to enable the participants to fully understand the purpose and background of the basic research project they had been part of. In contrast to the interview with the think-aloud, the debriefing was purposefully not audio-recorded, to signal the end of the official data-collection process and to allow participants to relax and talk freely about the study and their impressions. Even though the informed-consent sheet and the accompanying information sheet had included a detailed description of the data-collection process, the debriefings seemed to be of high value to the participants as they were able to ask questions under the immediate impression of the field experiment and an intrinsic thirst for knowledge was tangible in many of these encounters.

The importance of such a space of reflection between participants and researchers is exemplified by the case of a young female participant in our sample who was interested in taking part but who voiced concerns regarding extremely violent social-media content during the initial written correspondence. Knowing the social-media content that had ultimately been chosen for use in the experiment, the first author explained to the potential participant the level of violence to be shown as comparable to that of the evening news and stressed that she would completely understand the participant’s withdrawal. The young woman decided to take part, however, and in the first narrative-biographical interview did not voice any further concerns. Before the experiment, she even seemed positively interested in the scientific work. It was only after the completion of all data-collection steps that the participant declared in the debriefing how anxious and afraid she had been before the experiment in anticipation of what she would have to do and see. She further described herself as being very cautious in how she moved through public space and parks in particular – causing the first author, in turn, to worry that taking part in the experiment had possibly deterred this young woman even further from participating in public life. The informal and trusting atmosphere of the debriefing had, thus, allowed the participant to open up and provide insights beyond the object of study into how she had experienced the intervention the researchers had caused in her life, as well as giving the researchers even greater appreciation for the sorts of risks they were asking their subjects to take.

Negotiating privacy in the public use of new technical instruments.

In our project, part of the data collection took place in public parks where we deployed the mobile eye-tracker. The mobile eye-tracking glasses entail audiovisual recording, a feature that was highly criticized in the case of the AR device Google Glass due to the perceived invasion of privacy (Kudina and Verbeek, 2019; Rodriguez and Opsahl, 2020). How should a mobile eye-tracker then be appropriately used for research in public space? After all, privacy is a basic right, to be carefully protected. Neglecting it can cause harm as well as distrust of technology which also would limit the actual potential technology could have (Georgiadou et al., 2020). On the one hand, it, thus, seemed ethically appropriate in our case to make passers-by aware of the recordings that were to happen, for example, by putting up signs informing passers-by that we were conducting studies in the parks. On the other hand, such announcements pose a risk to external validity, limiting the very advantage of the instrument’s mobility and portability and its accompanying potential for research in field settings. Ultimately, in this trade-off between ethical decision-making and the experiment’s validity, we decided not to warn passers-by of the recording. During the data collection, no passers-by seemed to be disturbed by our participants wearing the mobile eye-tracker and there was no occasion on which we were questioned about the field experiment itself. Only once did a park user raise concerns. That was, however, related to an additional recording of one participant’s walk, this person having consented to the documentation of the walk with a regular consumer video camera. The recording was essential to be able to exemplify the experimental set-up later on to the stakeholders participating in the (at that point still scheduled) scenario-building exercise (see also below). During the recording, an uninvolved female park user felt her privacy rights affected by the researchers’ filming of the running experiment and confronted the team, though not the participant during their walk, in the middle of an experimental trial. Here, it proved useful to have an information sheet with us at all times while moving in public space which contained details about the project as well as contact details of our university’s public-relations department, which we had informed ahead of the data collection. The park user who feared a violation of her privacy rights was satisfied with the written information we were able to provide to her. Still, the example illustrates that the concept of privacy contains a “multitude of definitions-in-context that are used by everyday people to make sense of their relationship with their own data” (Markham, 2018, p. 4) and particularly research teams that operate in public space need to be aware of diverging perceptions (Humberstone and Cutler Riddick, 2021).

Interdisciplinarity in complex problem-oriented research settings.

In complex problem-oriented research, a single discipline’s views are mostly too limited to find appropriate answers, and thus interdisciplinary discussions are needed (Decker, 2004; Collins et al., 2013). After all, “research and scholarship are community activities” (Preissle et al., 2015, p. 151). Our project team is composed of researchers with a high diversity of disciplinary backgrounds from the social, natural and technical sciences; thus, finding a common language is an obligatory ongoing task, from proposal development to implementing research, analyzing results and finding conclusions. Balsamo and Mitcham (2010, p. 270) indicate that in interdisciplinary research settings, it is “important to acknowledge that everyone has something to contribute and to learn”. They identify five ethical habits or what we prefer to call virtues, for interdisciplinary research teams: (1) generosity, to acknowledge each other’s work; (2) confidence in the importance of each other’s contributions; (3) humility, to recognize the partiality of one’s knowledge; (4) flexibility, to change one’s perspective; and (5) integrity, to exercise responsible participation to build trust. Acknowledging and seeking to practise these virtues both implicitly and explicitly helps to build a common understanding as we found in our project team. The team initially assembled based on the members’ openness towards interdisciplinary collaboration to apply for funding. In the implementation of the different project steps over the past two years, we rarely encountered any friction. This was partly due to the rather self-contained nature of the different sub-projects. A large part, however, can also be attributed to trusting and acknowledging each other's work and expertise in dealing with the sub-projects and the concomitant confidence that all members of the team could contribute as well as would contribute significantly to the success of the venturesome project. During the core data collection, which was inherently mixed-methods, with both the quantitative and the qualitative data collected concurrently, there was a mutual curiosity about how to deal with sensitive questions in other fields: in technology-focused fields of research which usually dealt with questions of measurability and conversely, in person-focused fields of research which were often concerned with the vulnerability of individuals.

Also, all team members belong to academic institutions, which provides some structural trust regarding a joint mindset of academic integrity and critical (self-)reflexivity. Still, even a highly interdisciplinary and reflexive research team needs to pay attention to possible disciplinary blind spots. In our case, additional knowledge particularly about technical issues in augmented-reality research, which we lack, has been provided by individual members of our interdisciplinary SAB and their consultations.

Looking back, we even went beyond interdisciplinary work in the narrow sense when we teamed up with the ethics advisor from our project’s SAB to dive deeply into ethical issues on the occasion of writing this article. The interest was mutual, highly productive and led to a heightened appreciation not only of the above-mentioned expertise needed for the conducting of the empirical sub-projects but also of the experiential knowledge of how to navigate between ethical demands, institutional needs and limitations and the practical challenges of empirical research.

4.2 Ethical aspects of data processing and management

Data ethics are important because “data always involve persons” (Markham, 2018, p. 7). Further, Markham et al. (2018, p. 2) stress that in dealing with data, “the challenge of locating responsibility and accountability is exacerbated by the difficulty of determining with any clarity the relationship between action and consequence as well as between data and persons”. Thus, it is the researchers’ responsibility to collect only the data that are needed and to sort, select and analyse data in an ethical way, which is always a highly selective process based on expert knowledge and past experiences reported by scientific literature (Dewsbury, 2010). Particularly in MMR, this “controlled complexity” (Boase and Humphreys, 2018, p. 154) is needed to reduce potential harm, as MMR leads to an “increased chance of participants being identified as data collected via different methods are combined” (Preissle et al., 2015, p. 154). However, this demand implies that researchers know before data collection and analysis what the data represent and what information value the data have. In explorative methodological endeavours, in which researchers work with new sensors and methods and produce new data sets, it might not be evident before data collection just what data are most relevant, potentially causing an ethical dilemma with the demand to only collect data that are needed. In our project, too, we will be able to judge only retrospectively which collected data were ultimately needed for the success of the MMR development project. For the sake of controlled complexity, in the process of developing the final research design for the quasi-experiment, we already decided against also using mobile electroencephalography (EEG, Gaurav and Kumar, 2018), which we had suggested in combination with mobile eye-tracking in the funding proposal. After the start of the project, it became clear that the technical, methodological and also ethical complexity of applying EEG technology and, most of all, the resulting mixed-methods design would exceed the necessary degree of thoroughness we would be able to provide with our small team.

Furthermore, in various disciplines as well as within interdisciplinary contexts there is a growing demand for a framework to better understand the implications of ethics in human–technology-related research. For example, in the Human–Data-Interaction (HDI) research community, criteria such as legibility, agency and negotiability are formulated. These aim to make data and methods understandable, usable for people and set it into context with norms and legal restrictions. Furthermore, a distinction must be made between people directly working with data and people who are affected by the consequences of data usage. Data itself are abstracted in data directly measured (primary), derivatives thereof (secondary) and information derived for communication and dissemination (tertiary). In the tertiary case, also the way in which data are implemented and visualized impacts interpretation and decision-making. As the interaction with data and with technology, in general, is specific for human individuals, it is necessary to allow for different views on data and to allow these different views to be considered valid (Victorelli et al., 2020).

Feedback of involved users and parties may help to further shape the interaction between humans and technology. In the data collection, the qualitative parts of the MMR design, as well as the debriefing, gave our participants space to voice their experiences and ideas regarding using mobile technologies. In the upcoming participatory phase of the project, the scenario-building (Sections 3.3 and 4.3), feedback will also be collected in a structured way about desirable futures, allowing for a more refined view on AR, mobile media consumption and possible effects on future urban and park planning. However, we need to understand and consider that the properties of data-measurement set-ups, infrastructure and data interfaces have implications not only for how we measure and analyse but also for how we think about data sets. Literacy in data and related methods is a prerequisite for interpreting data correctly (i.e. in the frame of what the data can potentially be used for) and avoiding erroneous data usage. The current developments show increasing automation of data interpretation and usage (e.g. due to automated learning algorithms). This pre-interpretation of the course also impacts the attitude of users to data. To deal ethically with data, we need to consider the whole data life cycle including collection, processing and interpretation, identifying the involved persons interacting with the data directly and also considering the persons indirectly involved (Victorelli et al., 2020).

4.3 Societal implications of developing technologies and instruments

In methods-development projects, not only do the societal implications of results produced by developed methods need to be taken into account, but so do those of the methods and instruments developed and combined by researchers, and of the research process itself. While methods development aims to advance scientific research methods, the resulting methods for tracking and understanding human practices in increasingly technologically sophisticated ways might subsequently be applied by parties outside of academia. This might include scenarios where society benefits from technological development and where it is misused, e.g. for state surveillance, threats to autonomy or civil rights, or the violation of privacy. For example, within attention-economic settings, actors capitalizing on increased attention paid to their (media-)products could misuse research results from eye-tracking concerning the affective engagement with locative media consumption in public spaces. Actors who have, for instance, previously benefited from portraying the parks as spaces of fear (e.g. political actors or boulevard media) could further benefit from the knowledge of which forms of media content draw the most attention. We assume that the decision against collecting EEG data for a combination with eye-tracking and interview data led to a considerable reduction of this risk in our project. We recommend research teams working on innovative methods to think ahead and take into account possible concerns (beyond the obvious) related to potential misuse of methods and/or research results after the successful completion and – often open-access – publication of a project.

Transdisciplinarity – or whom to engage?

Public participation is a central pillar in the concept of responsible research and innovation (Stilgoe et al., 2014; Burget et al., 2017; Genus and Sterling, 2018), aimed at enhancing science’s reflexivity, inclusiveness and responsiveness to society by an early anticipation of impacts (Schuijff and Dijkstra, 2020). In MMR too, scholars are increasingly acknowledging “the importance of engaging with the views and perspectives” (Torrance, 2012, p. 115) of other parties involved. A central query in public engagement and co-creation literature is the question of whom to engage (Rowe and Frewer, 2000; Reed, 2008; Voorberg et al., 2014), and thus implicitly also whom to exclude. The answer differs based on the form of participation. While participant selection criteria (e.g. representativity of a certain population) are often legally mandatory in formally restricted participatory procedures applied at the public-administration level, informal participation, as often applied in the social sciences, is more adaptable to the respective specific context. Due to this relative freedom and adaptability of methods, the question of whom (not) to engage in participatory research processes, thus, gains weight in transdisciplinary research. In our project’s setting, we need to closely evaluate who qualifies as a stakeholder for a discussion of societal implications and the futures of AR use in public spaces. Our view of intended stakeholders of public spaces may significantly differ from those envisaged by political actors or city councils. Besides, when crafting socio-technical futures and concomitant regulations, usually rather elitist groups of society take part (i.e. experts), rendering many others as effectively not having futures (Sand, 2019). Special efforts, thus, need to go into including marginalized groups and individuals who may be hindered e.g. by time and resource constraints or a lack of knowledge about the opportunity to participate. Still, the aspirations of transdisciplinarity need ultimately to be weighed against feasibility and the ensuing question of “who is to gain” within the limits of an individual research project.

While we were originally planning on and looking forward to bringing together participants from various backgrounds for face-to-face workshops, this ongoing phase of the project has been heavily affected by a ban on participatory face-to-face events due to the COVID-19 pandemic. Ultimately, we decided to substitute the face-to-face workshops with an online Delphi-scenario process, an adaptation of the classic Delphi approach combined with scenario planning (Nowack et al., 2011; Wright et al., 2013). While the Delphi-scenario method usually focuses on experts in the narrow sense, we aim to include experts with a high diversity of professional backgrounds such as civil society, technology, scientists, politics and administration, media and tourism/recreation. The main selection criterion here is having expertise in areas relevant to the research question of developing scenarios for urban park planning and usage within the digital transformation. A collection of feedback from a broader audience is planned by applying digital tools such as storyboards/walks. While the necessary shift to online formats may benefit broader participation due to increased accessibility and wide distribution for instance in social media, it concomitantly entails a bias towards individuals who have internet access and skills.

5. Discussion and conclusion: Doing ethics under conditions of complexity and uncertainty

It can be concluded that doing ethics in mixed-methods development proves as challenging as in other contexts where researchers are faced with complexity and uncertainty. In the process of mixed-methods development that integrates new technologies, research ethics – as it is currently practised – remains a trade-off between aspiration and reality. Simultaneously, the need to engage with ethical challenges when researching with and on technologies is even more urgent. As Kudina and Verbeek (2019, p. 297) point out, “[i]f ethics is about the question of ‘how to act’ and ‘how to live,’ and technologies help to shape our actions and the ways we live our lives, then technologies are ‘actively’ taking part in ethics”. It seems, therefore, all the more important for researchers to openly share their experiences in doing ethics when engaging in methods development.

Unfortunately, existing funding schemes and an increasingly competitive academic system rush the implementation of projects, data collection, and analysis. Particularly in mixed-methods-development contexts, which always entail a particularly complex, novel research process, a lot of time and effort goes into the planning and preparation of processes. Still, as we have shown here, there is a need to also include meticulous consideration of ethical aspects, their discussion in the team and with advisors, and possibly revision from the start of the planning through the project implementation up to the storage of data and a sharing of metadata. Indeed, methods development would benefit from ethics committees that can have a full appreciation for the kind of work that can be undertaken and are themselves reflexive in their work. In particular, such a committee would need to be “willing to enter into a dialogue with applicants and be willing to listen to what applicants say” (Jennings, 2012, p. 92). Our project highlights the limitations of prospective ethics review in the context of mixed-methods development. When the concerns are new and they cannot be sufficiently articulated for standard, anticipatory or prospective research-ethics review or when the concerns emerge in the course of the research (which ethics committees routinely do not follow up on), they are unlikely to receive sufficient attention or guidance or they will simply not be captured or recognized in the formalized processes of review. Here, current concepts of ethics review may need adjustment and committee membership training to recognize challenges brought about by methods innovation in research. The notion of “retrospective review” (Dawson et al., 2019, para. 1) could help explore lessons learned and develop alternative frameworks for ethics review. Even more importantly, we appreciated the continuous dialogue on ethics with our ethics advisor colleague and found it very helpful for our mutual learning curves. We would like to highly recommend to both empirically working researchers and ethics committee members to embark on such a continued exchange beyond the one-time ethics approval whenever possible.

The commitment in the field of ethics shown by research communities such as the AoIR’s is all the more important and research ethics committees would only benefit from stronger participation in those discussions that take place on the ground of interdisciplinary empirical research.

Moreover, in the face of rising methodological, and thus ethical demands for young researchers, students also need to be introduced to and involved in ethical discussions early on. Hence, applied ethics, as well as reflections on “doing ethics” in empirical research, should be a mandatory part of teaching curricula at least at the PhD and master’s level – regardless of the discipline or methodological orientation (Prinzing et al., 2020).

With the ethical insights into our ongoing research project presented in this article, we hope to have given a vivid example and an impetus for an intensified ethical discussion in the field of MMR. After all, we as researchers can only benefit from “fostering more candor in such endeavours” (Preissle et al., 2015, p. 153) in our aspiration to contribute to the benefit of society.

Figures

Figure 1.

Video still: Participant looking at a digital trigger that contains a review of the park’s kiosk as recorded by eye-tracking world camera. Text field: Related text excerpt from participant’s subsequent retrospective think-aloud, translated from German