In my opinion, the TOS… Situating personal data literacy interventions

Abstract

Purpose

It has been demonstrated that AI-powered, data-driven tools’ usage is not universal, but deeply linked to socio-cultural contexts. The purpose of this paper is to display the need of adopting situated lenses, relating to specific personal and professional learning about data protection and privacy.

Design/methodology/approach

The authors introduce the results of a case study based on a large educational intervention at a fully online university. The views of the participants from degrees representing different knowledge areas and contexts of technology adoption (work, education and leisure) were explored after engaging in the analysis of the terms and conditions of use about privacy and data usage. After consultation, 27 course instructors (CIs) integrated the activity and worked with 823 students (702 of whom were complete and correct for analytical purposes).

Findings

The results of this study indicated that the intervention increased privacy-conscious online behaviour among most participants. Results were more contradictory when looking at the tools’ daily usage, with overall positive considerations around the tools being mostly needed or “indispensable”.

Research limitations/implications

Though appliable only to the authors’ case study and not generalisable, the authors’ results show both the complexity of privacy views and the presence of forms of renunciation in the trade-off between data protection and the need of using a specific software into a personal and professional context.

Practical implications

This study provides an example of teaching and learning activities that supports the development of data literacy, with a focus on data privacy. Therefore, beyond the research findings, any educator can build over the authors’ proposal to produce materials and interventions aimed at developing awareness on data privacy issues.

Social implications

Developing awareness, understanding and skills relating to data privacy is crucial to live in a society where digital technologies are used in any area of our personal and professional life. Well-informed citizens will be able to obscure, resist or claim for their rights whenever a violation of their privacy takes place. Also, they will be able to support (through adoption) better quality apps and platforms, instead of passively accepting what is evident or easy to use.

Originality/value

The authors specifically spot how students and educators, as part of a specific learning and cultural ecosystem, need tailored opportunities to keep on reflecting on their degrees of freedom and their possibilities to act regarding evolving data systems and their alternatives.

Citation

Raffaghelli, J.E., Romero Carbonell, M. and Romeu-Fontanillas, T. (2024), "In my opinion, the TOS… Situating personal data literacy interventions", Information and Learning Sciences, Vol. 125 No. 5/6, pp. 367-388. https://doi.org/10.1108/ILS-06-2023-0086

Introduction

Until recently, the debate on digital technologies in education and in human life overall centred on the marvels made possible by their application, almost ignoring the problems and drawbacks associated with all innovations – their dark aspects. The hidden costs of digital technologies have become evident nowadays. Several studied problems are: the digital divide and epistemic injustices in the infosphere to the “multitasking myth” in the first web (Adam, 2020; Hargittai, 2003); nudge-induced information overload to the limitations of the prosocial web as the source of “post-truth” (Peters, 2017) and the erosion of privacy as a value (Marín et al., 2021; Prinsloo et al., 2022); the over-exposition to the screen in terms of psychophysical health (Burr et al., 2020); and last but not least, the ecological impact of data centres and computer hardware with the recent developments of artificial intelligence (Selwyn, 2023). Educational policies and practices based on the distribution of devices, teacher training and the adoption of digital tools and apps frequently failed to achieve relevant educational aims (Sancho-Gil et al., 2020). The narratives of innovation as “educational success” have also been deconstructed by several studies pointing out at the subtle interests aimed at the marketisation of digital technologies by big technological companies (Perrotta et al., 2020; Saura et al., 2021; Williamson et al., 2022). Such a strand of research focuses on the strategies of platformisation and datafication. These neologisms are used to label the strategies of data extraction and algorithmic manipulation embedded in platforms offering “take out of the shelve” educational solutions to generate “smart” learning environments. The smart component can be connected with discourses of personalisation, effectiveness and learning achievement in short periods of time at a low cost (van Dijck et al., 2018; Williamson, 2017). According to these studies, it is necessary to rethink the role of educational technologies through participatory, contextualised interventions that deal complexity as an inherent dimension of socio-technical systems (Raffaghelli and Sangrà, 2023b). Rethinking the digital on the light of complexity entails considering technology in the respect of their human dignity and heterogeneity along transformations that narrate a specific historic and cultural space. Some have conceptualised this later approach as a “postdigital” perspective (Macgilchrist, 2021). Such a perspective acknowledges the limitations of digital systems and rejects the prevalent position on their perfection (Knox, 2019).

In the above-mentioned context, the problem of data practices that are not transparent and might interfere with the user’s privacy and agency has become central. If data is collected massively and in intrusive ways, leading to forms of manipulation or, even worse, neglecting learners’ diverse identities (Prinsloo, 2020), several levels of intervention are necessary to deal with the problem. The educational response should cover interventions supporting learners and educators understanding of the conundrums of data privacy, inter alia, their ability to explore the tools’ affordances; to see the space for negotiation about the data collected, processed and shared throughout the technological systems; and finally, to claim their rights to protect their privacy and agency.

Nevertheless, there is a dearth of research on data protection and privacy (Marín and Tur, 2023) and several studies highlight the unpreparedness of educators to deal with the topic (Marín et al., 2021; Stewart and Lyons, 2021). Also, the systems coordinating educators’ professional development and faculty development to deal with the topic might be in deficit (Raffaghelli and Stewart, 2020).

This paper presents the findings of a case study on a large educational intervention at a fully online university. Through our case, we aimed at exploring how prior knowledge may influence the perspective on data privacy. We hypothesised that different professional expectations might imply diversified software usage and relevance, exposing the participants to different forms of literacy. Connected to this, we explored what we called the “renunciation effect”: the “trade-off” between increased awareness about the risks for data protection embedded in a tool and the need to keep on adopting it. This effect might be diversified across disciplines which in our study were represented by the students enrolment in different UOC’s degrees. Finally, we also engaged the course instructors (CI) in a reflection about the implications for future design and organisation to deploy further interventions. We hence analysed and compared the CIs reflections about each specific group.

Background

Data practices, personal data protection and agency: the problem of renunciation

Traditional approaches to safeguarding personal data while interacting on the Internet have considered a variety of tools, including technological solutions, the use of terms and conditions and institutional data security provisions. The user has the option of exposing her data on public networks or cloud-based services (virtual private networks or VPNs) to avoid data capture by browsers (such as privacy-aware search engines like DuckDuckGo) or by encrypting her data (as is the case with instant messaging systems). Concern for the exposure of users to data breaches, forgeries or identity thefts has spurred the development of technologies to protect data privacy (Kearns and Roth, 2020).

This type of technical effort had legal foundations set up well before the beginning of the digital age. Since 1950, when the European Convention on Human Rights (ECHR) established the right to respect for private and family life, home and correspondence in Article 8, data protection has been related to human rights. During the 70s, several EU countries showed their concern for regulating access to and usability of personal data, particularly private and sensible data coming from health, financial, public and civic life, including education. Nonetheless, private big-tech companies have evolved quickly, challenging data protection. They operated in “grey zones”, which were regulated only after the technologies circulated and were massively used. As Floridi points out, the “infosphere” generated by the massive usage of digital technologies and participation through the internet cannot be considered equivalent to the Westphalian republics and territories within which the legislation attempts to cover their citizens (Floridi, 2014, 2020). Indeed, social media and apps originated in Silicon Valley (Facebook, Google, Amazon) or China (Tik Tok), taking the form of connecting to digital platforms and taking great advantage of data collection and processing as part of their business models (van Dijck et al., 2018). A common approach was the monetisation of data profiling users’ behaviour across several interactions on platforms to customise and personalise marketing strategies. In this sense, there were massive violations of the right to privacy using personal and even sensible data. Not only were data breaches the cause of revealing key information provided to platforms, such as telephone numbers or addresses. Particularly, the data usage was systematic and allowed the companies to intervene to manipulate users’ consumption habits, with huge impacts on health, social and economic levels (Zuboff, 2015). Other consequences were the critical biases and discrimination in which opened to a discussion on algorithmic injustice (Eubanks, 2018; Noble, 2018).

In this context, the legal intervention attempted to provide a comprehensive regulatory base, as in the case of the General Data Protection Regulation (European Commission, 2016). This regulation considers the conditions under which data could be collected, the identification of responsible persons for data treatment, the simplification of information about data traced and used and the right to request data and occasional cancellation. However, the respect for data protection had further consequences, as expressed by Floridi in his analysis of several episodes about the state or public body suing private big tech companies. The philosopher called this a “sort of de facto digital corporate sovereignty” (Floridi, 2020, p. 372), a form of hegemonic position as monopolies of the digital. The individual moves from being “a voter-consumer” to become a “follower-user” (ibidem).

These reflections provide the basis for considering the concept of “renunciation”, as an individual reaction to a context of de facto digital sovereignty. As the literature shows, the strong need for certain technologies generates a position of passive acceptance. Under such conditions, the usage of a technology that violates privacy does not stop, despite the intuition of risk or potential harm. Let us take into consideration how the phenomenon worked during the COVID-19 pandemic. The increase in digital platform usage at that moment was critical and deemed necessary as a tool to avoid the complete blockage of education. The “free offer” of commercial products such as Google Classroom or Amazon Cloud Services were considered a solution, and its social impact was critically considered only later (Perrotta et al., 2020; Williamson et al., 2020, 2022). Moreover, as the educators expressed the need for a quick response during the pandemic, they abandoned privacy (theirs and that of their students), feeding corporate sovereignty. Later, the “ethical” debate about educational technologies was also criticized. Indeed, data and AI ethics were deemed a sort of palliative to the conflicts raised by data monetisation and the several products offered to educators as part of the business models (Green, 2021). While data becomes a good that entails profit for big tech companies that operate in elusive ways, in front of renunciation as a widespread response by educators and learners overall, both technological and regulatory bases might become insufficient, we argue. At individual level, renunciation generates inertia and indifference (Kuhn, 2023; Pangrazio and Selwyn, 2019). At a social level, the institutions tend to rush behind perennial adjustments to the regulations and technological solutions needed to repair or protect from situations caused by past or existing practices connected to technological innovations (Floridi, 2023).

Educational response: can data literacy overcome renunciation?

The need to integrate a thoughtful, critical debate of the ramifications of data-driven practices in datafied societies as part of the educational response is already consolidated by both theoretical and empirical research in education (D’Ignazio and Bhargava, 2015; Knaus, 2020; Pangrazio and Sefton-Green, 2022). Nonetheless, it appears that few learners have the knowledge to actively participate in such debates, and many internet users are oblivious to the potential ramifications of releasing personal data online (Bowler et al., 2017; Kuhn, 2023; Pangrazio, 2018). Also, educators are mostly unaware or overwhelmed by the complexity imposed by datafication, being puzzled at the time of designing and deploying pedagogical practices aimed at cultivating data literacies (Stewart and Lyons, 2021; Raffaghelli, 2022). In this context, personal data protection and privacy have been integrated into several digital literacy frameworks (e.g. the European framework for Digital Competence, (Vuorikari et al., 2022). The ability to understand the digital footprint, or the personal information exposed on the web, has been spotted as a key by media literacy studies (Manca et al., 2021). Privacy as part of a critical digital approach is already well focused on as a problem by educators (Gouseti et al., 2021).

However, as it has been highlighted by Prinsloo et al. (2022), consistently with Floridi’s perspective on de facto sovereignty, “viewing privacy as simply a technological problem to be ‘solved’ (…) disregards the role of historical and current structural arrangements in maintaining or creating inequalities and power relations” (p. 878). As we emphasised in our prior paragraph, the terms and conditions dealing with data privacy embedded in the several tools adopted require careful consideration as a matter of public debate and engagement. If the debate about the literacies needed to live in a datafied world, rapidly evolving towards the development of an industry of AI whose primary source is digital data, stops at the level of developing skills, the needed citizens’ political response to datafication will not arrive. Rather, adherence to the system and reproduction will be the result (Raffaghelli, 2023). Big and open data literacy and technical approaches to dealing with data, mostly based on the evolution of data science, relate to techno-solutionism. In fact, the educational response to increasing data-intensive practices was initially focused on developing data science skills as part of data literacy approaches, with the hypothesis that the more students become aware and able to develop data practices, the more they will have an active choice as agents in the datafied societies Raffaghelli and Stewart, 2020). The scholarly research has indeed contested the idea that education per se could expand the subjects’ agency; overall, there was a plea to revisit data literacy frameworks to move towards critical data literacies (Raffaghelli, 2023). Firstly, it was highlighted the need to cultivate awareness about data capture and the transparency of terms and conditions by several educational and social media tools integrated into the classroom (Pangrazio and Selwyn, 2020; Stewart, 2020). Secondly, there was an effort to conceptualise a number of approaches to uncovering social injustice and algorithmic bias (Hobbs, 2020; Kuhn, 2021; Sander, 2020). Thirdly, creative approaches emphasised the need engaging in activism about data injustices relating to race or gender, particularly at representational levels (Bhargava et al., 2015; Lee et al., 2022). Data sovereignty was less easy to deal with, but there were collectives claiming to embrace alternative technologies and discussing diversified pathways to step aside from data monetisation and the domination of Big Tech companies in education (European Parliament and Levi, 2022; Ricaurte, 2019). Such a relational idea of data, though, is also difficult to connect to the students’ daily lives, and they might remain at the level of abstraction, far away from the immediate learners’ needs and understandings (Bowler et al., 2017; Kuhn, 2023; Pangrazio and Cardozo-Gaibisso, 2021). In a continuum, agency (having a choice, understanding it and taking decisions made on that choice) could be placed in the opposite side of renunciation (having a choice and neither understanding nor taking it into account). Concrete actions within situated educational settings might help explore the implications of datafication. The existing conditions, technologies’ affordances and terms of use, by the same instruments adopted in daily personal and professional life can indeed be a very concrete starting point. This operation might entail the users’ clear understanding of their degrees of freedom to protect themselves, as part of a personal data literacy (Pangrazio and Selwyn, 2019). Privacy and personal data implications, though, could be connected to broader data practices and debates. This critical perspective on data in the personal and professional situated contexts of practice might at least spot renunciation and set the basis to develop agency around data.

Methodological approach

This study is based on a case-study approach. The case study is a holistic approach that attempts to explore a problem and the connected solutions in a contextualised setting (Stake, 1994). Case studies encompass naturalistic observation that is respectful of the participants’ and educators’ learning and professional needs. Moreover, a case can set the basis for pursuing design-based research aimed at testing an educational solution through several iterations (O’Neill, 2016). Our research was based on a mixed-methods approach for data collection and analysis, but it should be considered idiographic in its approach: the aim was to collect rich evidence on the participants’ experiences and opinions to tailor future educational practises connected to data literacy.

Our educational intervention aimed at raising awareness on data tracking, privacy and ethics in the context of digital competence. The team was exploring the problem in the context of an institutional programme to develop digital competence in the context of a cross-sectional (across several degrees) programme at the university (Guitert et al., 2023). The university is indeed a fully online university. Aligning with the principles formulated by Manuel Castells, the Universitat Oberta de Catalunya (Open University of Catalonia, UOC from now on) approach is not only to use the digital medium to deliver the curriculum but also to reflect on it as a catalyser of the “knowledge and information society” (Raffaghelli and Sangrà, 2023a, p. 361).

Though our intervention sees “itself as digital (…) fundamentally involved with technology” (Knox, 2019, p. 360) it embraces the “postdigital” perspective in a “(re)turn to core educational concerns in a context of a wider society already entangled with, and constituted by, pervasive digital technologies” (ibid., p. 361). We aim to explore, hence, the participants’ reactions to such digital entanglements, searching to support a reflection leading to agency, where these are not to be considered a student’s individual ability but rather a relational implication of responsible engagement and participation (Fawns, 2019). A relevant assumption for us, according to prior research (Raffaghelli, 2022) was that the areas of knowledge or disciplinary fields that characterise a person’s interests, and presumably their prior and current learning activity, provide lenses to build a situated relationship with technology. Renunciation, namely, the disregard of the tool’s terms of service (TOS) quality in the way it uses personal data in favour of technology engagement, can be modulated by situated experiences and dispositions towards data.

With this in mind, we explored the following research questions throughout our educational intervention:

RQ1.

Can the activity spot forms of renunciation in the students’ contextual perspectives (coming from the areas of knowledge they are engaged with) about the tools used in personal and professional life?

RQ2.

Can the activity, as an awareness-raising approach, promote more responsible and agentic engagement with technologies, overcoming renunciation?

Instruments and procedures: the activity did you read the terms of service?

The learning activity revised the TOS of the students-selected digital tools. We built on Stewart’s Open Page project (Stewart, 2020), where she creatively co-reviewed with her undergraduate students the TOS of a collection of digital tools used in education. Building upon these bases, the activity could help students envisage strategies to avoid, hack, or denounce problematic terms of service, enabling them to participate as citizens in public debates on the dangers, damages and abuses of modern technologies. Nevertheless, considering the reviewed literature, our students might show signs of renunciation around relevant technologies in their life. The activity was deployed in three phases, visible in Figure 1. These were:

1. Students choose a popular digital tool first. They explore basic information based on specific criteria, provided by the CI. The criteria are information regarding monitored data, the temporality of the conditions demanded by users to use the digital tool, copyright and user rights on generated content, updated references to national or international rules and positive scenarios of usage. In this phase, students should focus individually on the instruments and their effectiveness.

2. Students complete an online analysis survey, focusing on the selected digital tool evaluation. The digital tool ranks each question from 1 (unclear or abusive TOS) to 5 (extremely clear and respectful TOS). At the end of the analysis and having answered all questions, an automatic score from 0 to 85 supports a final “quality verdict”. Students can use the scale to create comparisons and reflect using the score.

3. Students collaborate to evaluate one tool, chosen by the group and basing on the individual analyses. Students can support their work by creating infographics. After the collaborative debate, the tool’s score (b) can be translated into “badges” to signify the quality of the TOS for the digital tool. Gold, Silver, Bronze, Tin and Scrap are the scores and badges. Appendix 1 lists the Badges’ description.

Participants

All students enrolled in the digital competence course at UOC were invited to participate in this activity. This is a required introductory course for all UOC students, and it serves as a prerequisite for any career at the mentioned university. The course has a 25-year history, and data literacy has been an integral element of its recent evolution (Guitert et al., 2023). Students from multiple degrees work together to develop a technical, creative, collaborative, inclusive and critical perspective on the use of ICTs, which is one of the most intriguing aspects of this course. The present activity was offered on a voluntary basis because it was a pilot., considering it a base for a future strategy within the digital competence course. Twenty-seven CI offered the activity in total. This configured 27 different groups, composed by students from different UOC careers converging at the Digital Competence course. We collected 823 responses from students, of which 702 were complete and accurate for analysis.

Data collection and analysis

Table 1 displays the connection between the research questions, the instruments adopted, the data collection and relating analysis carried out.

The dataset analysed adopting Tableau and R as software. The dataset and several interactive visualisations have been published to Tableau Public (Raffaghelli et al., 2021).

Findings

RQ1 – digital tools’ relevance and terms of service quality across knowledge area (renunciation effect)

Participants’ profiles.

UOC students have traditionally been characterised as “lifelong learners” (Guitert et al., 2007), i.e. individuals who return to school to obtain qualifications pertinent to their adult lives and projects. As reported in Appendix 2, there is a significant cohort of young “worker students” (aged 25–34, 36.4%) in all academic disciplines, though they are relatively more prevalent in law and psychology. There is also an interesting, relevant group of “seniors” pursuing likely their second opportunities in life, as in the case of computer science, followed by business administration and economics (25, 43 and 53 to be placed in the category of students over 35 years old, respectively); marketing also has a relatively high number of senior students (4 over 13). In addition, a relevant group reports having more than 15 years of professional experience, with the most experienced workers belonging to computer science (26 of 46 with more than 10 years in the category), business administration (52 of 136), economics (73 of 144), Design (12 of 25) and marketing (8 of 13). Less experienced are pupils from the fields of education (only 18 out of 71), psychology (32 out of 110) and the humanities (11 out of 29). As for gender, more females participated in the activity overall (59.2%), although they were distributed in accordance with traditional gendered divisions (most male participants came from computer science, 42 of 46 within the category; and most female participants came from psychology, 99 of 110, and education, 61 of 71). In general, it can be presumed that younger and less experienced students were female.

The digital tools’ choice and relevance.

The students’ analysed 122 digital tools, including apps, websites and software. At a first glance (Data visualisations at Tableau), there was a high prevalence of Google Products (Gsuite, Google Maps, Google Scholar, etc., 201/703 choices), WhatsApp (94/703), Instagram (56/703), Netflix (48/703), Twitter (36/703) and Facebook (22/703), upon other types of more specific tools, where the rest of the choices were situated (other 246 choices). Tellingly, there were few choices for apps or software specifically devoted to the Internet of Things (IoT, only chosen in the case of the wearables for sports practise) and no choices for specific AI programmes (from home assistants to data science programmes). To understand the distribution of preferences according to the tools’ adoption, we classified them by purposes of usage. Figure 2 introduces a heatmap with the classification of preferred tools by degree and the percentages of choice by area of digital activity. As we can observe, across all disciplinary fields, the students preferred social networking (142 out of 703) and communication and collaboration tools (167 and 133 out of 703). However, this preference is distributed unevenly since the economics/labour relations and marketing students overwhelmingly prefer communication tools (46 of 149 and 4 of 13, more than the 30% within the category). They are followed by business administration and computer science for communication tools (between 26% and 30%). The design students demonstrate a clear preference for collaboration tools (8 of 25, more than 30%).

In education and the humanities, there is a clear preference for social networking tools (26%–30%), which also applies to marketing, communication and business administration. There are cases in which the preferences are more distributed, like in the case of law (despite declaring a greater preference for collaboration, communication and social networking, they also display some interest in entertainment, creativity tools and information management tools).

Overall, we can assume that the selected digital tools aim at collaborative work and participation on the web, particularly in the case of social networking. There was little focus on individual productivity, entertainment, eCommerce, learning, health and sports.

Additionally, the students graded the information that was available and its relevance to their personal lives. The information was based on a score that the students provided after reviewing the TOS in accordance with the exercise instructions and using a four-item checklist (see Figure 1, “Tool Selection”). The scores’ relevance was based on specific scores (the physical and social activity; the professional objectives; and the opportunities provided for lifelong learning). Table 2 introduces the students’ “check” of available information, which led to the composition of the final score of available information. As can be observed, most students considered that there was reasonable information on the tool selected: 678 out of 703 students pointed out that they found reviews on the tool, 694/703 that there was an information page and 657/703 that there were use cases illustrating the tool. It is to be highlighted that there was less agreement with the existence of Open-Source documentation on the tools selected (430/703). Table 3 introduces the scores given by the students according to the available information and relevance, by degree.

Given the violation of conditions required to proceed with a parametric inferential test, the Kruskal–Wallis non-parametric test was adopted. No significant differences were found between the degrees about the relevance score assigned (chi-squared = 16.677, df = 10, p-value < 0.01). Instead, the students grouped by degree considered the available information differently, with a significant difference between them (chi-squared = 29.214, df = 10, p-value < 0.001). The post-hoc analysis (Wilcoxon Multiple Pairwise Comparisons) highlighted that the students from computer science scored the tools differently than those of communication, law and marketing (<0.05) and particularly than those of psychology (p-value < 0.01). Also, economics scored differently regarding communication, law and marketing (p-value < 0.05). However, the most relevant difference was found between the scores given by psychology students and those of economics (p-value < 0.001), with the latter being less benevolent with the score provided to the tools selected than the former.

A non-parametric Kendall correlation test between the relevance and the available information as perceived by the student (N = 703) yielded a slightly significant result (tau = 0.07, at the cut-off level of 0.05). The students found that the information was acceptable (maximum score 4, median 4, in a 1–4 scale) and tended to express a mild to high relevance of the tool selected (maximum score of 3, 3 to 4, median 2 to 3, in a 1–4 scale).

Terms of service quality: Renunciation.

It is worth recalling that the TOS quality was based on the tool’s available information. As shown in Figure 1, numerous analytical categories determined the final score. Data tracking was established around Cookie analysis, informed consent instructions, right to be forgotten explanations and no profiling after data collection. This last piece was related to third-party cession, recommender systems, nudge generation and data ethics principles. The TOS’s temporality was based on the user’s freedom to amend the contract, including withdrawing from it, and the user’s content licences. GDPR compliance was examined. Finally, key cases (good or negative) should be searched through news and articles about the tool’s usage, performance and societal impact. Students scored information, TOS timeliness, regulations, crucial cases and positive cases. Thus, in Table 4, we synthesise important category evaluations to calculate the digital tools’ final score.

Given the violation of conditions required to proceed with a parametric inferential test, the Kruskal-Wallis non-parametric test was adopted. No significant difference between degrees was found for the information on data tracking, the temporality of TOS or compliance with EU regulations. Instead, a significant difference was found for the availability of information on critical cases (N = 703, chi-squared = 10.636, df = 10, p-value < 0.05) and a highly significant difference was also present for the information about the availability of information on positive social impact after using a selected tool. As for the first result, the post-hoc analysis yielded the following: law students scored differently than their pairs of business administration, communication, design economics and psychology p < 0.05); Marketing students scored differently than those in the design degree (p. 05); and this last group, communication students and economics students, scored differently than those in computer science (p < 0.05). It appears that the legal knowledge with which the law students are more familiar could provide further substance to their positions about the tools analysed. Interestingly, most students selected collaboration and communication tools, as well as social networking platforms. In this sense, we could expect that law and computer science students have knowledge of the legal concerns around such tools as well as the digital infrastructures and algorithmic manipulations adopted. Instead, for communication, design, marketing and business administration, the sides of productivity and creativity might be more relevant. Finally, the students in education had a different opinion than those in communication and design (p < 0.05). In this case, the students in education might consider the tools as key spaces of work, whereas communication and design could take into account the appearance, aesthetics, productivity instruments and so on. Coming to the post-hoc analysis for the positive social impact of the tools adopted, we observe that there was an extremely relevant difference between the students of computer science and communication, education, economics and psychology (p < 0.001). However, also the students of education displayed extreme differences from their peers in the humanities and law (p < 0.001). Moreover, communication scored differently regarding humanities, law and marketing (p < 0.01); and psychology, with law and marketing (p < 0.01). Tellingly, in knowledge areas such as education or communication, collaboration, content sharing and communication are a matter of study and are supported by positive narratives, also in scholarly research. As already expressed, in computer science, data science and algorithms are a matter of analysis, so there might be an enthusiastic approach to any digital tool selected. The difference between psychology, law and marketing is relevant since their diversified, positive views relate to collaboration and communication tools, as well as information management and entertainment. In this case, what can be seen as a problem for one discipline (digital addiction, loss of privacy, etc.) for another can be seen as an instrument of reaching audiences and achieving results (as it might be for marketing).

Despite these differences in the perception of tools both for critical reasons and for their positive social impact, the final score, which wrapped the students’ opinions on the several tools analysed, did not yield any significant difference between areas of knowledge.

A non-parametric Spearman correlation test between the final score and the available information as well as the relevance (N = 703) was carried out. The score and the available information were positively correlated in a significant way (N = 703, rho = 0.14, p < 0.001), highlighting a relationship between the accessed information and the judgement on the tool. This could be expected. Instead, less obvious was the correlation between the relevance assigned to the tools and the score given (rho = 0,20, p < 0.001). The significant result displayed a positive relationship between the students’ appreciation of a tool for its relevance in their daily or professional lives and the TOS quality assigned. Given the non-significant result between degrees for the score (or the fact that the students tended to assign a similar score no matter their areas of knowledge), we assume that the relationships explored between available information, relevance and score applied to most students. We could cautiously pinpoint the renunciation effect here: the students are prone to see that the information available is enough and the score is acceptable when the tools are relevant for their lives.

Renegotiation about the terms of service quality: Badges.

The badges categorised TOS quality scores and were issued individually after collaborative discussions on one single tool. Figure 3. displays the final situation. Tin badges predominate overall. As expected, computer science students were more likely to condemn the tools, with almost 60% of their badges falling between tin and scrap. Humanities, law and marketing students were also critical (over 60% of badges were scrap or tin). Education students were more benevolent, placing almost 60% of their badges between “bronze”, “silver” and “gold”. Communication, psychology, business administration and economics follow (approximately 50% bronze, silver and gold). Business administration and design (almost 20% of gold badges) showed enthusiasm for the tools. Psychology students found a specific type of tool of very low quality (almost 15%), while all other students placed fewer than 10% of tools in this group, showing confidence in the tools adopted. Design students found no “scrap” tools. Assigning a badge as the ultimate “seal of quality” effectively triggers students’ reflection, causing a more cautious approach that retraces their earlier knowledge and discipline. In any case, the relevant differences across disciplines observed in the first phase remained constant also in this last phase. The Pearson chi-squared test shows a minor significance across degree-grouped student selections (x-squared = 54.89, df = 40, p < 0.05). Interestingly, students from fields that “know from the inside” how platforms work (Computer Science, which understands algorithmic procedures and data modelling; Law, which may be more aware of issues related to regulations and the big tech industry; Marketing, which understands recommender systems and nudges to capture participants’ attention) tend to attribute less value than other students. They may value the tools less if they dislike how they work or if they do not achieve the aims (performing well, complying with current regulations, drawing participants’ attention).

It is also relevant to point out that the students from psychology might be concerned about the cognitive and emotional effects of platforms on mental health, and hence, they were very negative with one group of tools with respect to the rest of the applications analysed. Or, in the case of the humanities, there is a sort of reluctance towards some of the automation promised by the tools. Instead, the students who adopt the tools for creative purposes (particularly design and education) might consider them so relevant in their practise that they would overlook potential effects to embrace their affordances. In all the cases considered, hence, the effect of prior information (computer science, law and marketing) could be hypothesised as a factor modulating the quality analysis; and renunciation (the trade-off between the negative and positive effects of using a tool, in the case of education and design) becomes apparent. All in all, in the subsequent phase of badge issuing after an interdisciplinary exchange, it appears that the students consolidate their critical opinions on the tools.

RQ2 – course instructors’ opinion on the activity’s impact

As shown in Table 5, the CIs thought the experience was relevant (9 of 27 thought it fully relevant and appropriate), despite issues with timing, task difficulty and student participation (not all participated). 5 of 27 suggested improving course coordinator coaching, design and activity time.

In the following, we comment and compare the CI’s perspectives on the groups led. The CI in 18 groups of 27 considered that their students were interested but showed no clear intention to engage with the problem of TOS. We assume that their analysis of the problem of TOS and platforms relates to a single, guided activity rather than a more active engagement in the future. Three groups (3), though, were reported as interested and intending to transfer to their future professional and personal activities the understandings gained along the way. Also, we must consider the difficulties experienced by two groups and the fact that four groups provided no feedback or even did not participate enough to conclude the activity. Additionally, we notice that the three groups with the highest levels of engagement received higher mean scores from the CI, whereas the score tended to decline for the scarcely engaged students (no feedback, no participation). The CI considered indeed that there were different levels of engagement and participation by the students (Kruskal–Wallis chi-squared = 26, df = 4, p < 0.0001).

Though this situation requires much deeper exploration in terms of its meaning for the participants and the CIs, we assume that there was a clear difference between a smaller group of more engaged students, prone to becoming active in their engagement with problems connected to data protection and privacy. As for the majority of students, they will not persist in their attempt to understand privacy. Also, most students would position themselves between the overwhelmed and the indifferent approaches to privacy. For a smaller group, the activity was just not considered appropriate for their learning.

Discussion and conclusions

In this case study, we analysed the students’ reactions to the problems associated with data privacy. We focused on the need of disclosing the rather obscure TOS of digital tools that could be relevant for the students’ personal or professional lives. Building on a widespread debate on the situatedness effects of data-intensive technologies, we explored two research questions in this regard.

Concerning the first question (Can the activity spot forms of renunciation in the students’ contextual perspectives (coming from the areas of knowledge they are engaged with) about the tools used in personal and professional life?) This exercise revealed that the students’ awareness of the TOS’s quality varied according to their engagement in specific undergraduate degrees. The response varied from critical and cautious to a sort of excessive optimism after their experience evaluating the TOS in terms of safety, provision of information on data usage, fairness and ethics. Dealing with students who preferred an online university might explain why they were interested in collaboration, communication and social networking tools. Intriguingly, most viewed the information about the tools as generally adequate. Though, there was less consensus regarding the tools’ documentation within open-source sites, which highlights the opacity upon which the platforms supporting apps are built. Therefore, many of them took quality “for granted”, even when the business model lacked complete transparency. In this regard, the essential relationship the students establish with many of the tools they use daily, appears nearly impossible to criticise or request additional information. This aligns with existing research that suggests that the technological framework induces individuals to rely on adopted technology in order to achieve a state of being “present” and “productive”, to the extent that they may embrace automation and relinquish forms of creativity and self-expression. In Selwyn et al.’s (2023, p. 9) terms, “digital automation is clearly linked with political power that derives from having the authority to coordinate individuals and their interactions”; therefore, the acceptance of extractive automation implies renunciation. This effect was convergent with the explorations made by Pangrazio and Selwyn in schools with teens (2020) or in educators as professionals (Raffaghelli, 2022): feelings of detachment or overwhelm take the lead. In our study, there were nuances about this phenomenon, though. computer science, law and humanities students displayed a cautious and more critical opinion on the TOS quality compared to education and design students. The former groups were able, considering their own knowledge about data-intensive technologies, to embrace a different position from those that are probably more compelled by contextual professional situations to adopt it. A good example in the field of education is the case of Google Classroom’s introduction into schools by policymakers during the pandemics, which overwhelmed educators and classrooms with a feeling of no possible alternatives (Saura et al., 2021).

The point is: can we really generate educational interventions that have an impact on renunciation? Or can we just raise awareness while we keep exploring the trade-off together? Our intervention just spotted differences and opened a space for reflection. Moreover, the collaborative space yielded little change about the initial positions, though our research method (countering the digital badges) just scratched the surface of the complex interactions held in some of the groups. Our findings support the idea that the more we are constrained by cultural or organisational contexts, the more the effect of a data literacy intervention will be to raise awareness and generate some consensus on potential harms. Nevertheless, the second effect, Floridi (2020) highlighted as de facto sovereignty (Floridi, 2020) or the capacity to build alternative digital ecosystems, requires sustained interventions and projects over time.

Convergently, the second question displayed the difficulty of addressing genuine change about renunciation. We asked whether the activity, as an awareness-raising approach, promoted more responsible and agentic engagement with technologies, overcoming renunciation. Our analysis of the responses of 27 CIs to the collaborative workgroups and final discussions showed that no matter the interdisciplinary configuration of the students’ groups, some were very passive, some extremely active. Only a few students would move forward towards action. More frequently, the students’ positions related to acknowledging the problem of data protection and privacy but not seeing any possible future engagement. Therefore, the fact of becoming aware might not have any direct effect on renunciation or directly produce renegotiation. Moreover, most CIs were enthusiastic about the relevance of the activity, but some raised concerns about the complexity of its implementation. This is another convergent element that reminds us that working over relational questions and problems takes time and effort by both teachers and learners. The use of platforms is frequently imposed at institutional or community levels, and regional or local policies might address the creation of alternative digital infrastructures. Getting to know the TOS is a limited action, so understanding at least the right to claim alternative spaces should be part of a renewed approach to personal data literacy that, as Prinsloo highlights, goes far beyond the technical (Prinsloo, 2022). However, in her activist research, Ricaurte has shown that mathplaining and data colonisation can be resisted:

[…] citizen initiatives in search of justice create alternative frameworks that make evident the inequality of power structures, the discourses about datafication and our digital lives, and the need for reflection on the diversity of contexts in which data epistemologies drive multiple forms of exclusion (Ricaurte, 2019, p. 361)

Awareness-raising is, doubtless, the first step in our educational endeavour to develop critical audiences. Our exercise was probably powerful as an initial pathway to disentangle the many complex relationships behind the use of technologies in our contemporary society. Nevertheless, it was limited in implementing the creative approaches claimed by Ricaurte, or in its duration. We contend hence that “developing a digital competence” cannot just be a prescription based on international frameworks introducing ethics, harmful effects (digital overexposure, cyberbullying), platform policies, business models (data monetisation and institutional monopolisation) and the like. If we explore existing frameworks of digital competence, becoming aware of the digital imprint or practising data detox for digital wellbeing has been a matter of concern at least for the last ten years. But the mere existence and usage to raise awareness does not suffice to activate citizens to embrace alternative pathways against data-intensive practices. If citizenship is tied to political agency and participation, the programmes on digital competence should avoid “naive” or “technical” (sort of “whitewashed”) approaches. Dealing with digital tools has local implications at the level of the economy, society and cultural identity. This is only based on contextual debates, moving from the local to the global level and back (European Parliament and Levi, 2022), but it requires educational spaces we need to re-imagine. Students’ exploration and deeper understanding of the tools they use is a starting point that must be pursued along the career pathways.

Interdisciplinary work could be another relevant element (Verständig, 2021). Through the collaborative badge issuing within groups belonging to the same degree, our activity demonstrated that the different disciplinary perspectives enrich. Though this exercise needs to be coordinated and have follow-up to dig into different data perspectives and dispositions (Raffaghelli, 2022), this should not be understood as just “reaching standards” that might be higher in technological or scientific pathways regarding humanities and social sciences”. Epistemic diversity and representation appear to be crucial to face datafication as a complex, multilayered problem. Also, working with professionals outside academia or creating groups where younger and older students collaborate could become possible ways of embracing complexity through diversity.

With no intention of universal takeaways, our study scratched the surface of a key problem: understanding the situatedness of educational interventions to develop postdigital literacies. We invite educators in other cultural contexts to lead and document their experiences. This might be the only way to detach from too general, “global” assumptions on how human beings will live with data-intensive technologies, particularly AI, in the near future.

Figures

Figure 1.

Workflow of the learning activity “Did you read the TOS”?

Figure 2.

Heatmap: Classification of preferred tools per knowledge area – degree

Figure 3.

Badges assigned to the quality of TOS by degree

Figure A1.

Badges definition

Figure A2.

Heatmap: gender, age and experience by course

Table 1.

Data collection and analysis

RQ	Instruments	Aim of analysis	Type of analysis
RQ1 Renunciation across contexts of knowledge	Survey for the individual TOS’ analysis Badges	Identifying: Students’ profiles (gender, age, professional experience by disciplinary field) Students’ perceived relevance about the selected tools per areas of digital activity and discipline Students’ evaluation of the TOS’ quality Analysing: Differences in students’ positionings across knowledge areas and Renunciation	Quantitative synthesis Bivariate descriptive statistics Correlation between the tool’s assigned relevance and the TOS assessed quality Inferential testing on the difference of relevance and perceived quality per knowledge area
RQ2 Potential impact of the activity	The final instructors’ written report on the activity, based on the students’ response and discussion forum	Exploring the CI’s opinion on the activity: educational aim, design, orchestration and students’ engagement (in terms of participation and orientation to act in the future)	Qualitative content and sentiment analysis

Source: Table by Authors

Table 2.

Tools’ available information per degree

Reviews	Information page	Use cases	Open-source documentation	Total
678	694	657	430	703
96,44%	98,72%	93,46%	61,17%	100%

Source: Table by Authors

Table 3.

Tool’s relevance and overall score on the available information by degree

	Relevance			Available information
	Min	Med	Max	Min	Med	Max
Business Administration	0.5	2.0	3.8	0.0	4.0	4.0
Computer Science	0.3	2.0	3.5	2.0	4.0	4.0
Communication	1.0	2.0	3.3	3.0	4.0	4.0
Design	1.3	2.0	3.5	2.0	4.0	4.0
Economics/Labour Relations	0.5	2.0	3.8	2.0	4.0	4.0
Education	1.0	2.0	3.5	2.0	4.0	4.0
Humanities	0.5	2.0	3.5	2.0	4.0	4.0
Law	0.5	2.0	4.0	2.0	4.0	4.0
Marketing	1.8	3.0	3.8	3.0	4.0	4.0
Psychology	0.5	2.0	4.0	2.0	4.0	4.0
Grand Total	0.3	2.0	4.0	0.0	4.0	4.0

Source: Table by Authors

Table 4.

TOS quality areas by degree

	Info on data tracking		Temporality of TOS		Regulations informed		Available info on critical cases		Available info on positive social impact cases		Tools’ overall score
	Mean	SD	Mean	SD	Mean	Mean	SD	SD	Mean	SD	Mean	SD
Business Administration	3.15	1.31	3.53	1.02	3.96	3.15	1.31	1.27	2.75	1.83	57.53	15.95
Computer Science	3.09	1.11	3.25	1.03	3.60	3.09	1.11	1.34	2.11	2.06	52.91	14.65
Communication	3.26	1.02	3.10	1.25	3.76	3.26	1.02	1.50	3.24	1.97	56.57	15.18
Design	3.06	1.46	3.67	0.66	4.30	3.06	1.46	0.76	3.24	1.74	59.12	12.25
Economics/ Labour Relations	2.93	1.06	3.42	0.96	4.03	2.93	1.06	1.15	2.74	1.65	55.57	13.67
Education	3.17	1.00	3.58	0.88	3.70	3.17	1.00	1.59	2.34	1.93	57.56	13.14
Humanities	2.92	1.23	3.34	1.04	3.79	2.92	1.23	1.29	2.41	1.88	53.17	14.80
Law	2.86	0.93	3.34	0.84	3.65	2.86	0.93	1.45	2.26	1.84	52.53	13.09
Marketing	2.93	0.96	3.59	0.70	3.92	2.93	0.96	0.93	1.92	1.61	54.46	11.03
Psychology	2.88	1.23	3.24	1.09	3.92	2.88	1.23	1.40	2.82	1.88	53.96	17.39
Grand Total	3.01	1.15	3.41	0.98	3.88	3.01	1.15	1.32	2.61	1.85	55.38	14.79

Source: Table by Authors

Table 5.

CI’s opinion on the activity

Educational aims, design and orchestration	Freq	%	Students’ engagement	Freq	%
Enthusiastic	12	44.44	Enthusiastic	3	11.11
Relevant and appropriate activity	3	11.11	Significant interest, intention to transfer learning
Crucial activity	9	33.33
Moderately enthusiastic	6	22.22	Moderately enthusiastic	18	66.67
Relevant, improve coaching	1	3.70	Significant interest, partial intention to transfer learning
Relevant, improve design	2	7.41
Relevant, improve timing	3	11.11
Cautious	9	33.32	Cautious	6	22.22
Improve coaching	1	3.70	Focus on the difficulties	2	7.41
Improve design	4	14.81	Little feedback	3	11.11
Improve timing	4	14.81	Low participation	1	3.70
Total	27	100.00	Total	27	100.00