Digital team coaching for workplace communication: longitudinal evaluation of recipients’ perceptions

Abstract

Purpose

The purpose of this study is to describe the implementation of a digital-based team coaching intervention aimed at improving team communication in the workplace through social network visualization. The study examined recipients’ perceptions of the intervention at two time points and assessed the temporal stability of various factors, including the intervention’s integrity, design, transferability, acceptance and the usability of the adopted visualization tool. The moderating role of digital usability was also evaluated.

Design/methodology/approach

Four team coaching sessions were delivered to 62 participants from seven teams across three departments within a large public health-care organization in Northern Italy. Perceptions of the intervention dimensions were collected after the second and fourth sessions.

Findings

Results indicated that, at both time points, recipients appreciated the intervention’s integrity and usability more than its design, transferability and acceptance. Furthermore, no significant changes in recipients’ perceptions were observed over time. The transferability of the intervention was significantly associated with its acceptance, but only when the usability of the digital tool was high.

Research limitations/implications

The study enriches existing literature on digital interventions in group communication by focusing on process dimensions like recipients’ perceptions of various aspects and the implementation process. Furthermore, the study underscores the potential of integrating specific techniques such as sociomapping and coaching within health-care organizations, encouraging more research and development in these areas.

Practical implications

The study emphasizes the critical role of usability and integrity in digital-based team coaching interventions, suggesting that high-quality, user-friendly tools not only lead to initial effectiveness but also sustain positive impacts over time, while also increasing transferability and acceptance.

Originality/value

The present study uniquely deploys a longitudinal approach to examine recipients’ perceptions of a digital-based intervention that combines social network visualization and team coaching to enhance team communication.

Citation

Giusino, D., De Angelis, M., Kubík, R., Axtell, C. and Pietrantoni, L. (2023), "Digital team coaching for workplace communication: longitudinal evaluation of recipients’ perceptions", Team Performance Management, Vol. 29 No. 7/8, pp. 257-277. https://doi.org/10.1108/TPM-11-2022-0077

1. Introduction

Teamwork and team communication are critical factors for successful team performance in the workplace and vital outcomes of team-level workplace interventions, as they significantly contribute to team effectiveness (Salas et al., 2018). Effective communication can be characterized by four primary attributes: clarity, timing, relevance and frequency (Franc et al., 2019). By focusing on these aspects, team-level workplace interventions, such as team coaching leveraging social network analysis and visualization, can offer valuable strategies for its optimization, foster teamwork and communication, ultimately promoting team effectiveness and organizational success (Bahbouh, 2012; Bahbouh and Lasker, 2014; Bahbouh and Willis, 2022).

Health-care, in particular, exemplifies a domain where teamwork and communication are considered critical for patient safety and team performance (Rosen et al., 2018; Shoukat et al., 2022). Health-care is a complex, demanding and diverse field requiring interdisciplinary collaboration. Team-based work plays a pivotal role in making informed decisions that draw upon a wide range of expertise (Barnes and Hollenbeck, 2009) and performing tasks requiring multiple individuals’ specialized skills (Mathieu et al., 2017). Past research highlights the importance of communication as a critical determinant of team effectiveness in health-care settings (Ervin et al., 2018; Fowler et al., 2021; Hopkinson et al., 2021; Molleman et al., 2010). Fostering effective communication within health-care teams is thus crucial not only for team success but also for overall organizational performance.

Digital technologies have expanded the scope of interventions, giving rise to digital-based team coaching interventions in the workplace. Digital workplace interventions can be defined as structured, planned and science-based actions aiming to promote desirable work outcomes by exploiting the potential offered by digital technologies. It may be adaptations of traditional, in-person interventions facilitated through online teleconferencing platforms or designed exclusively for computer or smartphone applications (Baños et al., 2022). Although research has shown that digital interventions can effectively promote desirable workplace outcomes (Phillips et al., 2019), and despite promising preliminary evidence regarding the effectiveness of digital-based team coaching interventions in various industries (Bernardová, 2012; Franc et al., 2019; Tetour, 2019), much of the existing literature has primarily focused on individual-level implementation and evaluation of digital workplace interventions. This is especially true regarding digital-based team coaching interventions based on social networks and sociometric analysis (Bahbouh, 2012; Bahbouh and Lasker, 2014; Bahbouh and Willis, 2022). This leaves a gap in our understanding of the potential benefits and challenges associated with team-level digital-based team coaching interventions, presenting an opportunity for further investigation and development in this emerging field.

Recently, the main frameworks for the evaluation of both the process and effects of workplace interventions (Nielsen and Abildgaard, 2013; Nielsen and Randall, 2013) have underlined the importance of taking into consideration the perceptions of the workplace actors involved in interventions, including the recipients of the intervention itself. These models argue that recipients’ perceptions should be integral to workplace interventions’ evaluation as they are vital mechanisms for their effectiveness. Recipients’ perceptions constitute underlying psychological aspects that may explain workers’ behavioral reactions to the intervention activities and, as such, may facilitate or hinder the effectiveness of the intervention itself, thus contributing to its success or failure. Particularly, Nielsen and Randall (2013, p. 608) stated that “an important part of […] evaluation should be the measurement of change in employees’ knowledge of the intervention, their expectations that the intervention can bring about changes.”

In light of the above, this study represents the first attempt to monitor aspects of the process, in particular, the perceptions of the recipients over time, of a digital-based team coaching intervention designed to enhance team communication through social network visualization and team coaching techniques in the health-care sector. The aim is to test whether this monitoring can provide an explanatory framework for the observed results and act as a catalyst for future outcome evaluation studies. This intervention was implemented in a large public health-care organization in Northern Italy as part of a broader project focused on evaluating organizational interventions (De Angelis et al., 2020). The study had three primary objectives. The first objective was to assess recipients’ perceptions of dimensions that are relevant to digital team interventions according to previous literature (Broetje et al., 2022; Holton et al., 2000; Martin et al., 2020; Vuori et al., 2012; Yelon et al., 2004; Zhou et al., 2019), specifically the usability, the transferability, the integrity, the training design and the acceptance of the intervention. The second objective was to evaluate the temporal stability of recipients’ perceptions, explicitly examining whether and how these perceptions evolved during the intervention implementation. The third objective was to investigate the role of the usability of the digital tool as a moderator in the relationship between perceptions of the intervention and its overall acceptance.

2. Theoretical background

2.1 From traditional face-to-face to digital-based team-level interventions

Traditional face-to-face team interventions in health-care have generally yielded positive results over recent decades, particularly those aimed at enhancing team communication. Numerous studies have demonstrated the efficacy of such interventions in improving various aspects of health-care delivery (Hung et al., 2020; Kilpatrick et al., 2020; McCulloch et al., 2011; Prewett et al., 2013; Sacks et al., 2015), suggesting that even brief, well-structured interventions can have a significant impact on team communication and overall performance in health-care settings. In other words, traditional in-person team communication interventions have demonstrated considerable success in fostering team communication within the health-care sector. These interventions, ranging from intensive team programs to brief targeted sessions, have been linked to improvements in staff attitudes, teamwork quality, technical performance, health-care efficiency and patient outcomes. As the health-care landscape continues to evolve, it is essential to build upon these findings and explore innovative ways to enhance team communication and performance further (Larson and DeChurch, 2020).

Digital-based interventions have become powerful tools for enhancing team communication across various industries. By leveraging cutting-edge technology, these interventions facilitate more effective, efficient and adaptable communication strategies, which are critical for team success. Relative to traditional face-to-face interventions, digital workplace interventions generally come with peculiar challenges, including, for example, users’ engagement and adherence, ethics, privacy and data protection (Baños et al., 2022; Phillips et al., 2019). On the contrary, digital interventions for teamwork can offer unique advantages over traditional face-to-face interventions, such as increased geographical and temporal accessibility, cost-effectiveness, personalization and attractiveness (ibid). Thus, digital interventions constitute a promising avenue for the future of workplace interventions. In this context, several studies have explored the impact of digital-based interventions on team communication in different sectors, such as military aviation (Bernardová, 2012), private enterprises (Franc et al., 2019), the hospitality industry (Tetour, 2019) and undergraduate business education (Willox et al., 2023). Taken together, these studies suggest that digital-based team communication interventions can enhance information sharing and situational awareness, improve clarity, timing, relevance and frequency of communication among team members, promote greater collaboration, problem-solving and overall productivity, and streamline communication processes, reduce misunderstandings and enhance overall team performance.

The increasing adoption of e-health practices (Eikey et al., 2015), along with the surge in remote teamwork following the COVID-19 pandemic (Newman and Ford, 2021), has led to a rise in online communication within the health-care sector. This shift underscores the need to explore the potential of technology in facilitating health-care team interventions, explicitly focusing on digital-based approaches to improve team communication. There is a notable gap in the current research landscape regarding digital-based interventions targeting team communication in health-care settings.

2.2 Team coaching interventions based on social network analysis and visualization

Innovative team coaching techniques and tools are revolutionizing the way organizations foster collaboration, communication and performance among their teams. In The Team Coaching Casebook, Clutterbuck et al. (2022) provided a comprehensive analysis of the power and impact of team coaching in organizational settings, emphasizing the importance of a customized approach based on individual team dynamics and challenges. Techniques assessing team strengths and weaknesses provide coaches with invaluable insights into areas for development. Team coaching tools facilitate creativity and problem-solving by engaging team members in hands-on, collaborative activities or navigating interpersonal conflicts and promoting understanding among team members. Furthermore, digital technologies are being integrated into team coaching practices, enabling interactive experiences that foster team bonding and enhance learning. These innovative approaches not only drive team performance but also contribute to creating a culture of continuous learning, adaptability and organizational change capability (Supriharyanti and Sukoco, 2023).

In this framework, team communication interventions grounded in social network analytical theory and methods (Wasserman and Faust, 1994) leverage digital tools for collecting, processing and visualizing complex relational data. In Bahbouh’s (2012) seminal handbook, the theory and technique of an innovative digital-based approach to improving team communication by leveraging sociometric analysis, social network visualization and team coaching were presented. Sociometric analysis and social network visualization help identify group communication patterns, isolated members and subgroups, and pinpoint the influencers. For example, when teams and their managers understand who from the team feels isolated or there are smaller cliques within the group, they can take actions to work on it. However, these approaches focus on the as-is situation. Using valid and reliable team communication measures, such as clarity, timing, relevance and frequency of communication with each team member, these digital tools use algorithms to create graphical representations, known as “sociomaps,” depicting the current and desired communication structures within the team (Rozehnalová, 2013). The structure of each intervention session comprises five sequential steps:

1. data collection;

2. team sociomap presentation;

3. team coaching;

4. creation of action plans and commitments; and

5. review of action plans and commitments.

The intervention is based on team coaching workshops, defined as direct interactions with a team, intended to help members make coordinated and task-appropriate use of their collective resources in accomplishing the team’s work (Clutterbuck et al., 2022). In these team interventions, the sessions and team coaching activities are guided by the digital visualization of sociomaps to stimulate team reflexivity and self-awareness. The insights from the visualization then help formulate individual and team action plans that lead to the desired and effective communication state within the team.

Therefore, the sociomaps help not only understand the current dynamics in the group but also provide visualization of the desired situation and enable monitoring of the group dynamics over time. The visual clarity of sociomaps visualization (3D color-coded map of group dynamics) makes the trends and patterns visually apparent and more actionable than traditional sociograms. Still, sociomapping, sociometric analysis or social network visualization can bring benefits to the group mainly when used with interpretation and potentially with team coaching sessions (face-to-face or online). This approach aims to foster better communication, collaboration and understanding among team members in various organizational settings. By combining digital software-based social network analysis and team coaching strategies, Bahbouh (2012) addressed communication gaps and inefficiencies within teams, ultimately enhancing overall team performance and satisfaction. This intervention not only highlights the importance of understanding the underlying structure of team communication networks but also emphasizes the role of tailored coaching in addressing individual and collective needs.

Later, Zakharchyn and Kosmyna (2015) confirmed the benefits of using sociometric analysis techniques for organizations aiming to optimize employee behavior and overall team performance. Despite such preliminary findings, the present contribution aims to advance the evidence related to this type of approach to improving team communication via social network analysis and team coaching by monitoring aspects related to the implementation of the intervention, that is, the recipients’ perceptions of the intervention itself and of the actors involved (e.g. the facilitator or the coach).

Back in 2009, Baron and Morin (2009) underscored the importance of coaching relationships in leadership coaching, emphasizing that the quality of these relationships is a critical factor in achieving desired outcomes. Applying the significance of coaching relationships to team coaching, similar principles can be observed. In the context of team coaching, the quality of relationships between the coach and the team members, as well as among team members themselves, plays a crucial role in the success of the intervention. By establishing an environment where team members feel comfortable sharing their thoughts, concerns and feedback, the coach can facilitate meaningful discussions and encourage collective problem-solving. In team coaching, the role of the coach is to nurture and support the development of both individual and collective competencies, aiming to improve overall team performance.

2.3 Perceptions of the dimensions of the team coaching intervention

According to recent models for evaluating workplace interventions (Nielsen and Abildgaard, 2013; Nielsen and Randall, 2013) as well as empirical studies in the same research strand (von Thiele Schwarz et al., 2021), it is crucial to understand and address the perceptions by recipients of the dimensions of workplace interventions. Intervention recipients’ perceptions refer to their attitudes toward the intervention and its various aspects, such as its dimensions, qualities, features, elements, components and ingredients. These aspects may encompass content, structure, facilitators, design, relevance, usefulness, objectives and more. Persson et al. (2012) suggested that each worker understands their work environment uniquely. Individuals interpret cues from the environment, developing a shared understanding based on common experiences and conditions. This concept can also be applied to workplace interventions. When workers participate in an intervention together, they may develop a collective perception of the intervention, influenced by cues from one another. In particular, in team interventions, these shared perceptions may shape their views on the positive or negative aspects of the action or initiative. By considering individual and team perceptions, researchers and practitioners can better understand and explain the overall impact of workplace interventions. Specifically, recipients’ perceptions should be an integral part of workplace interventions’ evaluation because they are key mechanisms for the effectiveness of the intervention. Thus, an essential part in evaluating workplace interventions should be measuring change in employees’ perceptions of the intervention, and their expectations that the intervention can bring about changes (Nielsen and Randall, 2012).

Numerous studies (Broetje et al., 2022; Holton et al., 2000; Martin et al., 2020; Vuori et al., 2012; Yelon et al., 2004; Zhou et al., 2019) have underscored the importance of various factors associated with the implementation of interventions, including their transferability, integrity, design, acceptance and usability. These elements are pivotal in comprehending the underlying reasons for an intervention’s effectiveness and its potential for broader application. Specifically, a thorough understanding of these aspects allows for a more nuanced analysis than merely assessing the effectiveness of an intervention pre- and post-implementation. It enables an examination of the components that may have been instrumental during the intervention, leading to a shift in perception among the participants.

Transferability can be defined as the extent to which intervention recipients think that the knowledge and skills that they learn during interventions are transferrable to the real-world workplace setting (Yelon et al., 2004). Yelon et al. (2004) emphasized the importance of transferability in the context of interventions, particularly in relation to their effectiveness and long-term impact. A successful intervention should not only facilitate learning and improvement within the context of the program but also enable participants to transfer these gains to their daily work. The integrity of an intervention refers to the extent to which the facilitator’s behavior was positive, rewarding and relevant to the recipients’ participation (Vuori et al., 2012). When interventions are implemented with integrity, the intended benefits are more likely to be realized as participants receive the full range of intended support, guidance and resources. Design can be defined as the degree to which intervention recipients perceive that the intervention has been designed and delivered to give them the ability to transfer learning to the job and that intervention instructions match job requirements. Holton et al. (2000) stressed the significance of well-designed training interventions for ensuring effectiveness and impact.

When considering digital-based team intervention tools, it is essential to recognize the role of acceptance of digital tools in determining the success of these interventions. It can be defined as the recipients’ experience that the intervention met their expectations and needs. Specifically referring to an internet-based team development tool deployed among nurses, Broetje et al. (2022) argued that understanding the acceptance and other recipients’ attitudes toward digital team interventions is critical to their successful implementation, uptake, adoption and use. Thus, these authors highlight the importance of investigating recipients’ perceptions and acceptance, specifically when evaluating digital-based team-level interventions.

Finally, potential factors worth examining include the usability of the digital tool. Usability can be understood in terms of the quality of the recipients’ experience with the technological platform where the digital interventions take place, entailing dimensions such as aesthetics, feedback, interactivity, functionality and other design elements (Zhou et al., 2019). Zhou et al. (2019) conducted a study to evaluate the usability of a mobile health social network analysis tool designed to enhance communication and collaboration among health-care professionals. The digital tool aimed to facilitate information sharing and teamwork in health-care settings, ultimately improving patient outcomes. The authors found that the tool was well-received by health-care professionals and was efficient in helping users identify communication patterns and collaborate more effectively with their colleagues. The usability of the platform used in a team-based intervention is crucial for ensuring its recipients’ better acceptance of the intervention. When a platform is highly usable, users are more likely to engage with the intervention, increasing adoption and compliance (Cruz Zapata et al., 2015; Kumar and Mohite, 2018). First, a user-friendly platform minimizes frustration and barriers to use, allowing participants to focus on the intervention’s content and objectives. Second, efficient and effective platforms enable users to achieve their goals within the intervention more easily. Third, when participants enjoy using the platform and find it beneficial, they are more likely to share their positive experiences with their colleagues, creating a ripple effect that can further enhance the acceptance of the intervention within the organization. However, little is known about how the usability of digital platforms interacts with other dimensions of digital-based team communication interventions and its potential role as a moderator.

3. The design and implementation of the team coaching intervention

This study is part of the European project H-WORK, funded by the EU-H2020 research and innovation framework (De Angelis et al., 2020). The project aims to design, implement and validate multilevel workplace interventions. The study was explicitly approved by the Bioethics Committee of the Alma Mater Studiorum University of Bologna (Protocol no. 0185076) and adhered to ethical requirements. Data collection occurred between March 2021 and January 2022. To maintain anonymity, participants created a personal code for each completed survey. The researchers’ contact information was shared with participants to address any questions or concerns.

The intervention implementation followed a workplace needs assessment and stepwise framework for team interventions (Lacerenza et al., 2018), ensuring organization–intervention fit (Andersen et al., 2021). Team-level communication, interactions and information exchanges were identified as areas needing improvement. A steering committee was established, as recommended by Nielsen et al. (2013), which included the health and safety manager, workers safety representative, directors of involved hospital departments and nursing manager.

Recipients were recruited through voluntary subscriptions to the team coaching intervention course, with information provided by their health and safety manager. Inclusion criteria required participants in each intervention edition to be members of the same team. Teams were usually colocated teams that only attended the digital team intervention. As health-care workers, they do not usually work in a remote mode. The interventions were delivered in remote format, on a videoconferencing platform, because of COVID-19-related social distancing public health and safety measures during the implementation period.

The intervention consisted of four team sessions, with two-month intervals between sessions. The first session was 3 h long, while the remaining sessions were 2 h each. Sessions were led by two trained professionals, one as the main facilitator and the other as an assistant. The first session introduced participants to the intervention framework and key concepts, such as team communication and effectiveness. Using visualized sociomaps, a team discussion on current and desired communication helped formulate improvement strategies. The second session discussed team communication and developing action plans for desired changes. The third session provided feedback and evaluated progress while also enhancing meta-communication skills. Participants shared thoughts on behaviors or work situations to improve communication effectiveness within the team and individual feedback on colleagues’ communication styles. The fourth and final session was a debriefing based on team coaching principles. Team communication measures were collected at each session to generate updated sociomaps, allowing for comparisons and enriching discussions.

Figure 1 displays an example of sociomaps used in the implementation of the intervention for the same team. The sociomap visually represents the interconnectedness of team members based on their communication about work-related topics. The positions of individuals on the map indicate their existing or desired communication patterns. For instance, communication frequency is represented by the proximity of team members on the map: the closer they are, the more frequent their interactions are or are intended to be. Each individual’s height and color on the sociomap signify their average communication intensity within the team. A higher elevation, marked by red, does not inherently indicate a positive characteristic, just as a lower height, denoted by blue, does not necessarily suggest a negative quality. However, individuals in red may experience communication overload, while those in blue might be insufficiently engaged with the team. Ultimately, the interpretation relies on each person’s role within the team and their perception of their position on the sociomap.

4. Methods

4.1 Recipients of the intervention

The intervention was delivered to seven teams across three departments (Department of Emergency, Department of Medicine and Department of Neuroscience) within a large public health-care organization in Northern Italy. Three teams belonged to the Department of Emergency (Team 2, Team 4 and Team 5; 26 individual participants, 41.9%), two teams were from the Department of Medicine (Team 1 and Team 3; 16 individual participants, 25.8%) and two teams were from the Department of Neuroscience (Team 6 and Team 7; 20 individual participants, 32.3%). Each team comprised 6–12 members: Team 1 had 7 members (11.3%), Team 2 had 10 members (16.1%), Team 3 had 9 members (14.5%), Team 4 had 6 members (9.7%), Team 5 had 10 members (16.1%), Team 6 had 8 members (12.9%) and Team 7 had 12 members (19.4%).

In total, 62 participants attended at least one of the four team coaching sessions. Specifically, 4 participants (6.5%) completed one session, 12 participants (19.4%) completed two sessions, 16 participants (25.8%) completed three sessions and 30 participants (48.4%) completed all four scheduled sessions. Although the entire team agreed to participate in the intervention, not all members completed all sessions.

Archival data from organizational records regarding participants’ sociodemographic and job-related information were provided by the contact persons from the targeted organization during the design phase. Twenty-seven participants were nurses (43.5%), 13 were physiotherapists (21%), 8 were health-care assistants (12.9%), 5 were doctors (8.1%), 4 were speech therapists (6.5%), 3 were health-care technicians (4.8%), 1 was an educator (1.6%) and 1 was an ambulance driver (1.6%). Three participants (4.8%) were identified as head nurses and 1 participant (1.6%) as a head doctor, while 58 participants (93.5%) held no leadership roles. Organizational tenure ranged from 0 to 35 years (M = 15.15, SD = 10.49), and ages ranged from 29 to 65 years (M = 46.9, SD = 9.44). Forty-six participants were female (74.2%) and 16 were male (25.8%).

4.2 Measures

Measures were collected one week after the second session (T1) and one week after the fourth and last session (T2). Items referenced the team using “we” (Chan, 1998) and were administered in Italian. Five dimensions of the intervention were measured as follows:

Usability. Six items adapted from Zhou et al. (2019) were used to assess the usability of the digital tool. Example items include, “The digital tool was easy to use” and “Overall, I am satisfied with this digital tool.” Response options ranged from “1 = strongly disagree” to “5 = strongly agree.” Cronbach’s α was 0.90 at T1 and 0.76 at T2.

Transferability. Three items from Yelon et al. (2004) measured the perception of the transferability of the intervention. An example item is, “The skills we developed during the team intervention will help us in our work.” Response options ranged from “1 = strongly disagree” to “5 = strongly agree.” Cronbach’s α was 0.84 at T1 and 0.90 at T2.

Integrity. Six items adapted from Vuori et al. (2012) assessed the perceptions of the intervention’s integrity. Example items include, “Did the facilitators make you feel like your participation was valued?” and “Did you find group discussions useful?” Response options ranged from “1 = not at all” to “5 = all the time.” Cronbach’s α was 0.83 at T1 and 0.82 at T2.

Design. Four items from Holton et al. (2000) measured the perception of the intervention’s design. An example item is, “The activities and exercises helped us apply learning on the job.” Response options ranged from “1 = strongly disagree” to “5 = strongly agree.” Cronbach’s α was 0.76 at T1 and 0.90 at T2.

Acceptance. Three items from Martin et al. (2020) assessed the acceptance of the intervention. An example item is, “I would recommend the team intervention to others in a similar situation.” Response options ranged from “1 = to a very low extent” to “5 = to a very high extent.” Cronbach’s α was 0.88 at T1 and 0.78 at T2.

Sociodemographic and job-related information was collected through the same questionnaire. At the end, respondents were asked to create a unique ID code to maintain anonymity while allowing tracking of individuals’ answers across different data collection time points.

4.3 Data analysis

The statistical package for the social sciences (SPSS) Statistics software version 25 was used to perform statistical analysis. Frequencies were run to gather information about the sample. Descriptives were run to calculate skewness and kurtosis to test the assumption of normality of difference scores between the observations of continuous variables aimed to be compared. Following George and Mallery (2010), if values of skewness or kurtosis were between −2 and +2, the distribution was assumed to be normal. The parametric paired-samples t-test, assuming normal data distribution, was conducted to compute mean scores and investigate statistically significant within-subjects within-time differences between variables whose difference scores were normally distributed. The nonparametric Wilcoxon signed-rank test, which does not assume normal data distribution, was conducted to compute mean scores and investigate significant within-subjects within-time differences between variables whose difference scores were not normally distributed. Similarly, a repeated-measures t-test was performed to investigate significant within-subjects differences across time in variables with normally distributed difference scores, whereas the Wilcoxon test was conducted to explore within-subjects differences across time in variables with not normally distributed difference scores. The paired-samples t-test and Wilcoxon test were preferred to repeated-measures analysis of variance and Friedman test, respectively, because they are considered to have less error risks when two observations are compared instead of more. The use of the Wilcoxon test was not generalized to both normally and not normally distributed variables as nonparametric tests ensure less statistical power when applied to normal data, whereas the precise identification of the actual existence of statistically significant differences was one main goal of this study. However, during researchers’ exploration of the data set, results did not change substantially when applying paired-samples/repeated-measures t-test to not normal data nor when applying the Wilcoxon test to normally distributed data. The average measure intraclass correlation coefficient (ICC; Shrout and Fleiss, 1979) was calculated as an index of inter-rater reliability to assess the level of agreement among team members in their subjective evaluations at both data collection time points. Due to one team having only one participant providing valid data, ICC could be computed for a sample of six teams. Finally, correlation analysis was performed before moderation analysis, which was conducted using the PROCESS macro for SPSS.

5. Results

5.1 Recipients’ perceptions of the team coaching intervention dimensions

At T1, 33 recipients (72.7% females, 36.4% nurses, M_age = 46.3, M_tenure = 15.4) completed the questionnaire, yielding a 53% response rate. At T2, 29 recipients (72.4% females, 37.9% nurses, M_age = 49.5, M_tenure = 16.4) completed the questionnaire, yielding a 46% response rate. At T1, inter-rater reliability was statistically significant at p < 0.05 for four teams out of six, such as Team 1 (ICC = 0.99, n = 4), Team 5 (ICC = 0.93, n = 6), Team 6 (ICC = 0.87, n = 3) and Team 7 (ICC = 0.98, n = 8), thus indicating a high level of consistency of evaluations within the teams after the second intervention session. Similarly, at T2, inter-rater reliability was statistically significant at p < 0.05 for five teams out of six, such as Team 1 (ICC = 0.97, n = 4), Team 3 (ICC = 0.85, n = 5), Team 5 (ICC = 0.81, n = 5), Team 6 (ICC = 0.98, n = 5) and Team 7 (ICC = 98, n = 6), thus indicating a high level of consistency of evaluations across the teams after the fourth and last intervention session.

Table 1 displays the results from paired-sample t-tests at T1. Significant differences were observed between integrity (M = 3.97, SD = 0.56) and design (M = 3.70, SD = 0.53); t(32) = 2.65, p = 0.012. In addition, a significant difference was found between integrity and transferability (M = 3.65, SD = 0.55); t(32) = 3.12, p = 0.004. Finally, a significant difference was noted between integrity and acceptance (M = 3.61, SD = 0.69); t(32) = 3.43, p = 0.002. The Wilcoxon signed-rank test for usability and acceptance, where the difference score was not normally distributed, revealed no statistically significant difference between their mean scores (Z = −1.782, p = 0.075). These findings suggest that, at T1, recipients appreciated the intervention’s integrity significantly more than its design, transferability and acceptance.

Table 2 presents the results from paired-samples t-tests at T2. Significant differences were found between usability (M = 4.03, SD = 0.58) and design (M = 3.70, SD = 0.70); t(28) = 2.16, p = 0.039. In addition, a significant difference was observed between usability and acceptance (M = 3.51, SD = 0.65); t(28) = 3.85, p = 0.001. Other significant differences were identified between integrity and design; t(28) = 2.63, p = 0.014, and between integrity and acceptance; t(28) = 3.92, p = 0.001. Finally, significant differences were found between acceptance and both transferability [t(28) = 3.15, p = 0.004] and design [t(28) = 2.16, p = 0.039]. The Wilcoxon signed-rank test for integrity and transferability, where the difference score was not normally distributed, indicated a statistically significant difference between their mean scores (Z = −2.166, p = 0.03). These findings suggest that, at T2, recipients appreciated the intervention’s usability and integrity significantly more than its design, acceptance and transferability.

5.2 Temporal stability of recipients’ perceptions

Nineteen recipients (78.9% females, 47.4% nurses, M_age = 49, M_tenure = 17.9) completed the questionnaire at both T1 and T2. The repeated-measures t-test revealed no statistically significant effects of time on examined perceptions with normally distributed difference scores, such as design [t(18) = 0.84, p = 0.407] and integrity [t(18) = 0.63, p = 0.535]. Similarly, the Wilcoxon signed-rank test indicated no statistically significant effects of time on transferability (Z = −0.709, p = 0.478), acceptance (Z = −1.803, p = 0.071) and usability (Z = −0.400, p = 0.689). These findings suggest that recipients’ perceptions of the team coaching intervention dimensions remained consistent over time throughout the implementation process.

5.3 Usability of the digital tool as a moderator

We conducted a moderation analysis to investigate whether the usability of the digital tool moderated the relationship between the perceptions of the team coaching intervention dimensions (i.e. integrity, design and transferability) and the overall acceptance of the digital intervention at the end of the intervention, meaning after the fourth and last session.

Table 3 presents the mean scores and the intercorrelations of recipients’ perceptions at T1 and T2, both at individual and team levels of analysis with aggregated data. Individual-level correlation analysis shows positive and statistically significant (either p < 0.05 or p < 0.01) associations among almost all the recipients’ perceptions of the intervention dimensions both at T1 and T2, with Pearson’s r ranging from 0.41 to 0.78. Only the perception of the usability of the digital tool does not appear to be associated with the other perceptions of the intervention dimensions. Some main differences can be observed in the team-level correlation analysis, where integrity at T1 does not appear to be associated with other perceptions of the intervention dimensions, and usability at T2 appears to be negatively associated with transferability (r = −0.75, p < 0.05) and (r = −0.85, p < 0.05) acceptance.

Then, moderation results revealed that only the relationship between transferability and acceptance was moderated by usability (n = 19). Particularly, there was a significant interaction effect of transferability and usability (B = 0.47, p < 0.05) on acceptance. Figure 2 shows that the transferability of the intervention was significantly associated with the acceptance of the intervention, but only when the usability of the digital tool was high [R² = 0.50, F(3,15) = 6.21, p < 0.01]. This finding suggests that the individual perception of the ease of use and effectiveness of the digital platform play a crucial role in facilitating the successful implementation of interventions, ensuring that participants can apply the learned skills in their work environment, ultimately leading to greater acceptance of the intervention.

6. Discussion

This study investigated specific process dimensions of a digital-based intervention that was implemented in a hospital with the aim of fostering communication among team members. The study contributes to the literature on team communication interventions by providing insights into the relevance of monitoring aspects of the implementation, and particularly, the recipients’ perceptions of the intervention and of the actors involved, such as transferability, design, integrity, acceptance and usability of the intervention. Although there is a growing literature on digital interventions for group communication, there is a lack of research on how the implementation process affects the relevance and acceptance of these interventions. These process measures are crucial for understanding why a given digital intervention may be relevant in a specific working environment and how to ensure that the digital intervention can be perceived as useful and that its contents can be transferred to the workplace. Moreover, in this case, digital intervention relies on specific intervention techniques such as sociomapping and coaching, thus providing interesting insights into the further development of these techniques as intervention mechanisms in health-care organizations. The importance of these observations is amplified given the current changes within hospital environments, notably the consolidation of health-care processes across different departments and their digital transformation, alongside the consequences of the COVID-19 pandemic on health-care personnel and facilities in both rural and urban regions (Knop et al., 2021).

In this perspective, we examined recipients’ perceptions of a digital team coaching intervention at two time points (T1 and T2) and assessed the temporal stability of these perceptions. At T1, 33 recipients participated, while 29 participated at T2, with 53% and 46% response rates, respectively. The results indicated that at both time points, recipients appreciated the intervention’s integrity and usability more than its design, transferability and acceptance. Furthermore, no significant changes in recipients’ perceptions were observed over time.

The first aspect to discuss relates to the preference of team coaching intervention participants for usability and integrity. From this perspective, the scarcity of medical professionals, increasingly burdened with more complex and managerial responsibilities, has spurred the development of a broader organizational strategy. This strategy delegated traditional tasks to other team members, such as nurses and health-care assistants (Knop et al., 2021). Although the roles of general practitioners and nurses are converging in some areas, the success of this multiactor approach hinges on several factors, including effective communication among team members (Mohr et al., 2011). The facilitators may have also been crucial in nurturing a group dynamic that encouraged team members to discuss communication patterns openly. Essentially, the ability to openly discuss these aspects in sessions led by a supportive coach, who can foster an open and stigma-free atmosphere, and through a user-friendly digital solution, may shed light on the reasons behind these preferences. Reflecting and contemplating better communication through graphical maps could help clarify role definitions, streamline task delegation and ultimately lead to better patient outcomes. These aspects align with previous studies demonstrating the importance of a training facilitator’s skills in promoting an open atmosphere. The quality of questions, attention to detail and linking to practical examples are all pivotal for participant engagement and learning outcomes (Wavre and Kuknor, 2023). Defining learning curves (e.g. current and desired communication and action plans) through graphical outputs, which can be reviewed and discussed in groups, might have been instrumental in leveraging peers’ and facilitators’ feedback to create a supportive learning environment. In the present study, such elements linked to the facilitator’s skills and the type of atmosphere the participants perceived during the coaching sessions are among the most relevant online. Future studies that want to investigate the impact of online training coaching sessions should continue to include these process aspects in their evaluations.

As a result, aspects such as design, transferability and acceptance, although positively evaluated, become secondary in importance. For instance, elements of the intervention’s design may have been seen as less important because it often focuses on aesthetic aspects, which might be perceived as secondary to the actual content and function of the intervention. While a visually appealing design can enhance user experience, it might not have been considered as crucial as the core components of the intervention. Recipients might have encountered difficulties seeing how the intervention could be generalized or adapted to their specific situations, which could have led them to value transferability less than other aspects. Furthermore, acceptance might not have been highly valued because recipients may have focused more on the immediate experience of engaging with the intervention rather than considering their overall acceptance. Alternatively, recipients may have had mixed opinions about the intervention, which could have contributed to lower average scores for acceptance.

Another aspect to discuss concerns how the digital tool was perceived as highly usable, with a user-friendly and intuitive interface that greatly enhanced the user experience and reduced the learning curve and potential frustrations. Participants quickly became familiar with the digital tool, including how to read the maps and develop improvement actions consistent with the workplace. The efficient and effective engagement with the content is attributed to the digital tool, which increases satisfaction and motivation, ultimately resulting in better outcomes. The high usability of the digital tool is a crucial factor that should not be overlooked, as it can significantly affect the success of the intervention.

One potential explanation for the favorable perceptions of the team coaching intervention among recipients may be related to the uses of visualization tools, which have been identified as critical mechanisms for effective workplace interventions (Abildgaard and Nielsen, 2018; von Thiele Schwarz et al., 2017). Recipients may have appreciated the graphical indicators used by the digital tool, such as colors, heights and distances, which could aid in comprehending sociomaps and provide a visual representation of abstract concepts, such as communicating with colleagues. Furthermore, the digital nature of the team coaching intervention may have provided benefits or affordances that would not be as readily accessible in other forms of team interventions, such as group scores and visualization, as well as the ability for individuals to participate remotely if they are not physically present during the sessions.

Our moderation analysis indicated that the usability of the digital tool moderated the relationship between a vital team coaching intervention dimension (i.e. transferability) and the overall acceptance of the digital intervention. This result suggests that when the digital tool is easy to use and efficient, participants are more likely to perceive the intervention as transferable to their work environment, leading to greater acceptance. One possible explanation is that a user-friendly digital tool enhances the participants’ experience, allowing them to focus on the content and applicability of the intervention rather than being distracted or frustrated by technical difficulties. Put differently, the innovative element, namely, the ability to visualize the geographic distribution of communication among team members, and to decipher the patterns and volume of information shared within the team through colors, heights and distances on the geographic plane, can, if easily comprehended, facilitate a guided group discussion on a topic as intricate and elusive as communication between colleagues and superiors. The digital tool enables a more effective visualization of fairly complex information, allowing participants to grasp the content more thoroughly and, consequently, contemplate strategies for transferring actions to the workplace to address any identified deficiencies. This increased focus on the intervention’s content may help participants better understand and apply the learned skills in their work setting, ultimately improving their perception of the intervention’s transferability. This result highlights the importance of investing in developing and improving digital tools that are both user-friendly and effective in facilitating the successful transfer of learned skills to the workplace. By ensuring that digital tools are easy to use and support the intervention’s objectives, professionals can lead to greater acceptance and satisfaction among participants. This, in turn, can contribute to improved team dynamics, increased productivity and enhanced well-being in the workplace.

The findings of this study have several implications. From a theoretical point of view, it contributes to the existing literature on digital interventions for group communication by shedding light on the importance of process dimensions, such as recipients’ perceptions of the intervention and its various components (e.g. transferability, design, integrity, acceptance and usability), and the implementation process. It emphasizes the need for further research on the effects of implementation processes on the relevance and acceptance of digital interventions. In addition, it highlights the potential of combining specific intervention techniques, like sociomapping and coaching, as intervention mechanisms in health-care organizations, thus encouraging further development and research on these techniques. From a practical perspective, the results highlight the importance of prioritizing the usability and integrity of a digital-based team coaching intervention. Participants perceived these aspects more favorably than design, transferability and acceptance. Therefore, developers and facilitators of such interventions should focus on creating user-friendly digital tools and maintaining consistency throughout the intervention process to ensure its effectiveness. Second, the temporal stability of recipients’ perceptions suggests that once a team coaching intervention is well received, its positive impact can be sustained over time. This underlines the value of investing in developing high-quality interventions that meet the needs and expectations of participants from the outset. Finally, the study reveals a significant association between transferability and acceptance of the intervention when the usability of the digital tool is high. This implies that when a digital tool is easy to use, participants are more likely to perceive the intervention as applicable to their work context and be open to adopting the changes it proposes. Consequently, designers and practitioners may prioritize the development of user-friendly digital tools to enhance the likelihood of successful implementation and adoption of team communication interventions.

We recognize the study’s limitations, including the small sample size, the lack of a control group and the reliance on self-reported evaluations. The study focuses solely on recipients’ perceptions of the intervention without measuring its effects on team communication patterns. Thus, while the findings offer valuable insights into the perceived qualities and dimensions of the intervention, they do not provide direct evidence of its impact on the intended outcomes. Another limitation is that the generalizability of the results is limited to the specific context of digital interventions aimed at improving team communication in health-care settings. Therefore, caution should be taken when applying the findings to other workplaces or organizations.

Despite these limitations, the study provides valuable recommendations for managers and practitioners seeking to implement digital-based workplace interventions to improve team communication. The study also contributes to filling a gap in the literature by exploring the potential of a digital-based intervention combined with an online coach involved with the goal of promoting better communication among team members, which represents a novel approach compared to traditional in-person interventions. Moreover, it sets the stage for future research on the implementation processes of digital interventions and their effects on recipients’ perceptions and the interventions’ relevance and acceptance.

Figures

Figure 1.

Example of a “sociomap” of the same team in different sessions

Figure 2.

Usability of the digital tool as moderator

Table 1.

Paired-samples t-test of recipients’ perceptions of intervention at T1

Paired recipients’ perceptions (T1)	M	SD	95% CI
			Lower	Upper	t	df	p
Integrity – Usability	0.192	0.70	−0.056	0.440	1.578	32	0.124
Integrity – Design	0.273	0.59	0.063	0.482	2.648	32	0.012
Integrity – Transferability	0.323	0.59	0.112	0.534	3.121	32	0.004
Integrity – Acceptance	0.359	0.60	0.145	0.572	3.427	32	0.002
Usability – Design	0.081	0.80	−0.204	0.366	0.578	32	0.567
Usability – Transferability	0.131	0.81	−0.155	0.417	0.935	32	0.357
Design – Transferability	0.051	0.45	−0.108	0.209	0.650	32	0.520
Design – Acceptance	0.086	0.66	−0.148	0.320	0.747	32	0.461
Transferability – Acceptance	0.035	0.55	−0.158	0.229	0.372	32	0.712

Notes:

M = mean; SD = standard deviation; CI = confidence interval; df = degrees of freedom

Source: Table by authors

Table 2.

Paired-samples t-test of recipients’ perceptions of intervention at T2

Paired recipients’ perceptions (T2)	M	SD	95% CI
			Lower	Upper	t	df	p
Usability – Integrity	0.037	0.65	−0.212	0.285	0.303	28	0.764
Usability – Transferability	0.251	0.69	−0.013	0.514	1.949	28	0.061
Usability – Design	0.328	0.82	0.017	0.639	2.161	28	0.039
Usability – Acceptance	0.526	0.74	0.246	0.806	3.852	28	0.001
Integrity – Design	0.291	0.60	0.065	0.518	2.631	28	0.014
Integrity – Acceptance	0.490	0.67	0.234	0.745	3.921	28	0.001
Transferability – Design	0.078	0.44	−0.088	0.244	0.957	28	0.347
Transferability – Acceptance	0.276	0.47	0.096	0.455	3.147	28	0.004
Design – Acceptance	0.198	0.49	0.011	0.386	2.165	28	0.039

Notes:

M = mean; SD = standard deviation; CI = confidence interval; df = degrees of freedom.

Source: Table by authors

Table 3.

Correlations among perceptions of intervention dimension at T1 and T2

Recipients’ perceptions	MT1	SDT1	MT2	SDT2	1	2	3	4	5
Individual level
1. Usability	3.78	0.61	4.03	0.58	–	0.27	0.17	0.07	0.18
2. Integrity	3.97	0.56	4.00	0.59	0.28	–	0.64*	0.58*	0.42**
3. Transferability	3.65	0.55	3.78	0.58	0.02	0.42*	–	0.78**	0.71**
4. Design	3.70	0.53	3.70	0.70	0.01	0.41*	0.65*	–	0.73*
5. Acceptance	3.61	0.69	3.51	0.65	0.16	0.55**	0.63**	0.44*	–
Team level
1. Usability	3.79	0.27	4.00	0.18	–	−0.52	−0.75*	−0.70	−0.85*
2. Integrity	3.98	0.38	3.99	0.32	0.07	–	0.87*	0.84*	0.64
3. Transferability	3.70	0.28	3.82	0.34	−0.02	0.70	–	0.88*	0.86*
4. Design	3.78	0.26	3.76	0.50	0.10	0.41	0.85*	–	0.80*
5. Acceptance	3.66	0.34	3.60	0.38	−0.09	0.71	0.84*	0.64	–

Notes:

Correlations at T1 are reported in the lower semi-diagonal, whereas correlations at T2 are reported in the higher semi-diagonal; M = mean, SD = standard deviation, *p < 0.05 and **p < 0.01

Source: Table by authors