Social impact and performance measurement systems in an Italian social enterprise: a participatory action research project

Abstract

Purpose

The purpose of this paper is to investigate the multidimensional nature of social and nonprofit organisations' accountability and performance measurement systems (PMSs). It further considers how these systems help in defining outcome performance indicators downward to beneficiaries

Design/methodology/approach

The paper discusses participatory action research (PAR) within an Italian social enterprise. In order to increase dialogue, participation and engagement, the researchers adopted focus groups as a preferred method of investigation and conducted a broad documental analysis from July 2016 to March 2018. The paper discusses the gathered data in light of the social impact value chain as well as the multiple-constituency approach.

Findings

The findings support the idea that social and nonprofit organisations lack the expertise and resources to evaluate outcomes and impact; however, through PAR, the organisation defined their desired outcomes and ascertained which internal output measures were most likely to be correlated with these outcomes. Moreover, the findings highlight that nonprofits develop outcome measurements less frequently because they have more control over their immediate activities and outputs.

Practical implications

This research suggests the need to reinforce lateral and downward accountability based on mission and mission-based activities in order to make the performance management system of social and nonprofit organisation linked to the organisational strategies.

Originality/value

This paper innovates methodologically in two directions: 1) it adopts action research as a qualitative method, allowing the researcher to generate solutions to collectively-identified problems and 2) the paper's arguments are strongly supported by rich empirical exploration that occurred over a period of 20 months in an Italian social enterprise.

Citation

Costa, E. and Andreaus, M. (2021), "Social impact and performance measurement systems in an Italian social enterprise: a participatory action research project", Journal of Public Budgeting, Accounting & Financial Management, Vol. 33 No. 3, pp. 289-313. https://doi.org/10.1108/JPBAFM-02-2020-0012

1. Introduction

The nonprofit sector has grown rapidly in many developed countries over the last decade, playing a central role in providing public services (Grossi et al., 2017, Haigh et al., 2015). Within this sector, there has been emerging growth in so-called “social enterprises”, organisations seeking to achieve social missions through the use of market mechanisms (Ebrahim et al., 2014; Mair and Martì, 2006; Costa and Pesci, 2016). Social enterprises combine aspects of typical nonprofit organisations (the social value orientation) and of typical for-profit companies (the financial efficiency); however, their primary purpose is to deliver social value to the beneficiaries of their social mission, and the financial performance became a means toward the social purposes (Ebrahim and Rangan, 2014). In essence, they are able to combine an orientation towards social values and a desire to gain economic value (as a means to an end; Dees, 1998). In this sense, the blended nature of both mission and margin allow to consider social enterprises as hybrid organisations that reflects the propensity of social enterprises to blend traditionally for-profit practices with traditionally nonprofit practices (Ebrahim et al., 2014; Grossi et al., 2017; Haigh et al., 2015).

Despite growth in the number of social enterprises and their increasing academic recognition, there remains much to be understood in terms of their institutional mechanisms, governance rules and accountability functioning (Ebrahim et al., 2014; Grossi et al., 2017; Costa et al., 2014).

Indeed, traditional accounting theories focus on the investor-owned enterprise perspective according to which organisations pursue the production of goods and services in order to maximise economic value for shareholders (Palmer and Vinten, 1998). Such a perspective encounters limitations when dealing with social enterprises (Hofmann and McSwain, 2013). Two primary shortcomings exist: first, the bottom line for nonprofit organisations is not based on the maximisation of shareholders' economic value for shareholders; rather, it is more broad and complex because it encompasses the creation of “social value” for the community as a whole. Second, social enterprises are characterised by a different stakeholder profile because their governance structure includes multiple stakeholders, and their activities benefit a broad range of stakeholders.

It thus becomes difficult to obtain a meaningful picture of social enterprise performance compared to pure public or private entities (Grossi et al., 2017). Focussing only on economic and financial indicators fails to offer a comprehensive evaluation of nonprofit organisational performance.

While for-profit organisations summarise their economic and financial performances in financial statements because shareholders consider profit to be the company's mission; in contrast, social enterprises recognise no direct correlation between increments of achievement in the organisation's mission and its financial performance (Moore, 2003). To reflect the dual nature of social enterprises with both financial and social value, these organisations have begun to experiment with certain accounting practices measuring not only economic performance but also social results (Bagnoli and Megali, 2011; Manetti, 2014; Nicholls, 2009). Difficulties aligning these measurement systems sometimes arise because these two types of value creation are intrinsically connected rather than in direct opposition in a zero sum equation (Emerson, 2003).

This difference has created a need for a more complex, multi-directional and multi-stakeholder performance measurement system (PMS) (Grossi et al., 2017; Christensen and Ebrahim, 2006; Najam, 1996), the actualisation of which remains a major obstacle. Multiple PMSs also deserve public recognition. Decision makers – such as governments and institutional departments – have begun considering the idea of granting special treatment to social enterprises when they are unable to adequately account for their activities (Van Dooren, 2017). Moreover, financiers and investors require impact data because their decisions in favour of a given impact investment are partly based on the social impact created by social enterprises (Ebrahim, 2009, 2019).

With these considerations in mind, what should a PMS for a social enterprise look like in order to accomplish the goal of measuring social impact? As Costa and Pesci (2016) outline, a universal reply to this question does not exist. Social enterprises differ in size, degree of formality, form, sector, geographic scope, rationale for operation, stakeholders and other circumstances; therefore, it is difficult to normatively support a standardised and universal metric for measuring social enterprise performance (see also Palmer and Vinten, 1998).

Recently the debate on PMS and social enterprise accountability has been propelled by the “theory-driven evaluation” method (Rogers, 2007), according to which organisations observe how different programmes and initiatives cause intended or observed outcomes and impacts (Ebrahim and Rangan, 2010; Epstein and McFarlan, 2011; Ebrahim et al., 2014). This method follows the impact value chain (Clark et al., 2004), which is a “logic chain of results” in which organisational inputs (e.g. money, staff time, capital assets, etc.) are used to support activities and services (e.g. health services, schooling, job training, etc.). These activities ultimately result in the delivery of outputs to a target beneficiary population (i.e. results that a social enterprise and a nonprofit organisation can measure or assess directly). The identified output can lead to different effects and changes in beneficiaries' attitudes, behaviours, knowledge, skills and/or status, i.e. the outcome of the social enterprise's activity. Short-term benefits and changes then can foster a societal impact on the broader society in the long term (Ebrahim and Rangan, 2010; Epstein and McFarlan, 2011; Clark et al., 2004; Ebrahim et al., 2014; Costa and Pesci, 2016).

Within this debate, Costa and Pesci (2016) have contributed to the theoretical domain of the impact value chain by highlighting that within social enterprises' complex PMS, no universal measures can be defined without engaging different stakeholders. Costa and Pesci's study (2016) elaborates on Ebrahim and Rangan (2014)' research by supporting the decision to measure social impact in terms of the most proper metric for specific needs. However, there is a need for more empirical exploration in this direction (Grossi et al., 2017; Haigh et al., 2015; Ebrahim et al., 2014; Mair and Martì, 2006) in order to increase our knowledge regarding accountability and PMS for social and nonprofit organisations (Gray et al., 2001).

In order to fill this gap in the literature, this paper presents a case study of an Italian social enterprise that adopted a participatory process to design and implement a PMS indicator in order to measure the social value to beneficiaries. The case study presented focuses on a multi-stakeholder social cooperative (Defourny and Nyssen, 2017) aimed at addressing a societal problem – the care-assistance of elderly people.

By merging Ebrahim and Rangan (2010)' impact value chain and the participatory approach adopted in the multiple-constituency theory proposed by Costa and Pesci (2016), the paper follows a participatory action research (PAR) approach (Chiu, 2003). As such, the researchers were directly engaged with the social enterprise in order to identify performance measurement problems, co-create solutions with selected stakeholders in different focus groups and implement outcome indicators. The PAR was performed over a period of 20 months from June 2016 to March 2018.

The remainder of this paper describes this research in greater detail. The following section explains the current debate on accountability and PMS in social and nonprofit organisations. Section 3 then introduces the research methodology by outlining the PAR approach. Section 4 deals with the vast empirical material of this case study and the construction of the PMS in the Italian social enterprise before drawing final conclusions.

2. Accountability and PMS in social and nonprofit organisations

2.1 The accountability dilemma: “for what” and “to whom” questions

Accounting information is important both in the nonprofit and for-profit sectors, but in the nonprofit context, it assumes a key role because donors want to be kept informed regarding usage of their financial contributions. This demand for accounting information follows two primary theoretical foundations (Hofmann and McSwain, 2013); first, an information asymmetry emerges between the organisation (nonprofit or for-profit) and its stakeholders, and second, an agency problem arises between the property and the managers (both nonprofit and for-profit organisations).

Social and nonprofit organisations often adopt the conventional accounting framework as a primary reference for their accountability, despite the fact that this framework design reflects a for-profit context. Many authors have criticised this accounting approach due to its inability to consider certain social aspects related to modern western life (inconsistencies, injustices, invisibilities and inequalities); consequently, it lacks the capacity to produce social changes (Gray, 2002). Critical theorists argue that organisations have to share accounting information with all the stakeholders engaged with the organisation's activities, not just the shareholders (Gray et al., 1996). The same accountability aimed at a wide range of stakeholders has to extend beyond financial aspects to social impacts. When it comes to social and nonprofit organisations, various studies have pointed out the limitations of conventional accounting and its narrow focus on financial measurements (Reheul et al., 2014).

Of course social and nonprofit organisations have to consider their economic and financial performance, just as pure private-owned organisations do; at the same time, they must also consider their mission achievement and social impact (Grossi et al., 2017; Costa and Pesci, 2016; Haigh et al., 2015; Andreaus and Costa, 2014; Ramus and Vaccaro, 2017; Bryan, 2019). An orientation towards creating social value does not mean that social enterprises should ignore strategies to generate earnings and cash flow. On the contrary, they must constantly create economic value in order to survive over time and ensure that they will be able to continue with their mission. In other words, while business organisations consider wealth creation to be a way of measuring value creation, social enterprises consider wealth creation as a means to an end, enabling survival over time (Dees, 1998). As such, social enterprises can be considered “double bottom line” organisations (Costa and Pesci, 2016; Chetkovich and Frumkin, 2003) grappling with the tension between mission and margin: they have to maintain sustainability from a financial perspective while simultaneously serving a socially-oriented mission. Otherwise, social enterprises could be exposed to mission drift, that is, to abandon social concerns in favour of only market-oriented activities (Ramus and Vaccaro, 2017).

In social and nonprofit organisations, both financial and non-financial dimensions are important and strictly interrelated; both must be considered in their PMSs (Grossi et al., 2017). Within the nonprofit sector, financial resources are meaningless if they are not employed to achieve a social mission, yet without efficient employment of financial resources, it is impossible to achieve this mission (Epstein and McFarlan, 2011; Ramus and Vaccaro, 2017; Bryan, 2019).

This context therefore requires an urgent and broad reflection on the question of accountability (Ebrahim, 2019; Grossi et al., 2017; Haigh et al., 2015), which still remains a complex and multifaceted concept requiring further investigation, especially with reference to social enterprises (Edwards and Hulme, 1996; Christensen and Ebrahim, 2006; Costa and Pesci, 2016) and nonprofit organisations (Gray et al., 2001; Costa et al., 2011, 2014). Indeed, accountability for social enterprises and nonprofit organisations cannot solely be connected to economic and financial aspects; it also has to include stakeholder relationships (Andreaus and Costa, 2014; Ebrahim and Rangan, 2010).

When dealing with social and nonprofit organisations, researchers broadly have discussed the accountability framework along five main questions: who should be held accountable, to whom, for what, how and with what consequences. Two of these enquiries (Ebrahim et al., 2014) continue to attract the attention of many scholars: accountability to whom (e.g. Ebrahim, 2003) and accountability for what (e.g. Quarter et al., 2009; Andreaus and Costa, 2014). The “accountability to whom” question deals primarily with stakeholders affected by social and nonprofit organisations, namely donors, funders, beneficiaries, workers, volunteers and the members themselves.

Several studies state that social and nonprofit organisations are accountable to multiple actors: 1) to patrons, 2) to clients and 3) to themselves as organisations (Ebrahim, 2009, 2019; Najam, 1996; Edwards and Hulme, 1996). Nonprofit organisations' accountability to the patron, or upwards accountability (Najam, 1996), usually refers to relationships with donors, foundations and governments; this form of accountability relates to how nonprofits have spent the funds received for designated purposes. Accountability to clients, or downwards accountability, involves the relationships with people for whom nonprofit organisations provide services. Finally, accountability to themselves refers to the internal accountability of nonprofits (Ebrahim, 2003). Due to the presence of a wide range of stakeholders, it has become critical for organisations to identify priorities in discharging accountability.

By addressing the “accountability for what” agenda, scholars have delved into issues, categories and drivers by identifying and describing the core area(s) to which the social and nonprofit organisations should be accountable. Financial accountability often is considered to be of primary relevance. However, for some key stakeholders, it is less important in light of the fact that financial aspects are not the primary focus of such organisations. For this reason, other information may play a more important role, particularly data related to the organisation's key mission (Connolly and Hyndman, 2013; Bryan, 2019). Andreaus and Costa (2014) have proposed an integrated accountability framework (IAM) that includes three main areas of disclosure: 1) the economic and financial dimension or the capability to be economically sustainable in the long term; 2) the mission-related dimension, or the organisation's raison d'être and 3) the social-related dimension, or the relationship with stakeholders. These different areas have to be analysed together in order to address the integrated accountability challenges of nonprofit organisations. The IAM acknowledges the complexity of the economic-financial dimension of accountability in the nonprofit sector: creating economic value is absolutely essential to ensuring long-term survival for the organisation pursuing its mission. At the same time, the model also includes the mission-related dimension due to the urgency of evaluating the mission performance of nonprofits, a driver for accountability (Andreaus and Costa, 2014).

2.2 PMS in nonprofit and social enterprises

Many organisations measure performance and results by resorting to a PMS, allowing them to achieve multiple aims (internal planning, decision-making, control and external accountability issues). If well-designed, these measures can provide information about both financial performance and non-financial aspects. Using a variety of tools (performance appraisals, benchmarking, evaluation techniques, balanced scorecard tools, etc.), it is possible to measure financial aspects while also assessing the organisation's mission achievement. Within social and nonprofit organisations, it remains difficult to measure organisational performance due to the ambiguities that characterise this sector (e.g. measuring the achievement of diverging aims and targets) (Grossi et al., 2017). The complexity of social and nonprofit organisations' operations makes it difficult to measure their performance, particularly given that financial and non-financial issues coexist in this ambiguous environment (Grossi et al., 2019).

In the past few years, a growing number of academics have turned their attention to the management control systems and performance measurements of social and nonprofit organisations. However, according to Ebrahim and Rangan (2010), few studies have looked at this topic within the nonprofit domain specifically. Over the past ten years, there has been an attempt to propose multiple and diverse PMSs for social and nonprofit contexts, and it is possible to divide these initiatives along two different lines. On one hand, scholars recommend an adapted balanced scorecard framework to include social and environmental impacts (Simons, 1995; Kaplan and Norton, 2001; Moore, 2003; Somers, 2005; Bull, 2007). On the other, there are contingency models regarding the relationship between social impact and management control (Ebrahim and Rangan, 2010; Bagnoli and Megali, 2011). Table 1 provides a more detailed review of these studies.

The first branch of studies involves an adaptation of the traditional balanced scorecard, as reported by Kaplan and Norton (1996). The authors improved upon the traditional balanced scorecard in order to build a valid tool for social and nonprofit organisations by including both financial and non-financial aspects (Kaplan and Norton, 2001). Unfortunately, this contribution suffers from incompleteness and an inability to account for the differences between for-profit and nonprofit sectors (Moore, 2003; Somers, 2005).

Moore (2003) tried to fill this gap by building an alternative framework called the public value scorecard (Arena et al., 2015a). This model does not focus on a “competitive strategy” but considers the “public value strategy”, which is made up of three strictly connected elements: social mission, legitimacy and support and organisational capabilities (Moore, 2003). In Moore's opinion, these elements have to be captured in a “strategic triangle” in order to obtain a sustainable strategy for creating value. The public value scorecard helps managers to evaluate all relevant aspects before implementing a strategy in the nonprofit context (Arena et al., 2015).

Somers (2005) also sought to adapt the balanced scorecard to social nonprofit contexts by identifying three important changes in the structure of the traditional management control tool. This model includes a new perspective beyond the financial one, an extension of the financial perspective to consider long-term sustainability that also replaces the consumer's perspective with a wider stakeholder view (Somers, 2005). In line with Somers' thinking, Bull (2007) has proposed balanced scorecard adjustments by integrating multicriteria systems that can be adopted easily in the multi-stakeholder context characterising social and nonprofit organisations.

The other research branch refers to contingency models, which are mainly based on the peculiarities of social and nonprofit organisations (Arena et al., 2015). Ebrahim and Rangan (2010) developed a contingency framework for measuring social performance and analysing management control systems within the social sector. They argue that the dual objectives of social enterprises (social mission and profit/surplus) can be misaligned or in contrast, adding that the lack of standardised methods of assessing social performance makes comparison difficult (Ebrahim et al., 2014). The contingency framework put forth by Ebrahim and Rangan is a matrix that considers a logical chain of results within which organisational activities and inputs lead to a wide range of outputs, outcomes and social impacts (Ebrahim and Rangan, 2010). The authors created a basic logic model that includes inputs (what goes in), activities (what happens), outputs (what resultsimmediate) and outcomes (what resultsmedium/long term). Inputs include funds, equipment and supplies, while activities involve basic needs/services delivered, capacity building, construction of infrastructure, policy dialogues and workshops. Meanwhile, outputs and outcomes refer to the immediate results (roads built and policy papers written) and medium-long term results (e.g. improved living and increased income). As Ebrahim and Rangan (2014) highlight, organisational inputs, activities and outputs are much easier to measure than outcomes and impact, but it is ultimately the latter that indicates progress toward the social mission Bagnoli and Megali (2011) provide another important contribution to contingency theory by suggesting a three-level accountability process including a social effectiveness dimension (the ability to achieve social goals), an institutional dimension (respecting legal and self-imposed rules) and an economic dimension (evaluating performance measurement and long-term sustainability). Moreover, the Impact Centre Erasmus (ICE), based in the Netherlands, developed a useful guide entitled, “The Impact Path: The Entrepreneur's Manual to Impact Measurement Growth” (Avance, 2020). This guide provides step-by-step instructions enabling social enterprises to improve their PMSs and social impact measurements. It also includes concrete and detailed examples as well as a list of possible indicators. As initiatives like this one show, assessing social performance is more difficult and complex than evaluating financial performance. Financial measures refer to well-established accounting and market measurement tools, while social performance measures are not uniform. Organisations lack a common and unique method of measurement because the results involve different activities that often cannot be measured with common standards (Ebrahim et al., 2014). As such, Costa and Pesci (2016) recently have criticised the “one-size-fits-all approach” to social impact accounting in social and nonprofit organisations. Based on Ebrahim and Rangan (2010), the authors have proposed a five-step multi-constituency framework according to which organisations should define multiple and varied metrics based on the varying needs of different stakeholders, with a strong focus on stakeholder engagement.

As the literature demonstrates, many scholars developed different PMS frameworks for social and nonprofit organisations. As Grossi et al. (2017) underscore, these different frameworks make the following fundamental assertions:

1. The PMS will be multidimensional because of the large variety of purposes embodied by social and nonprofit organisations.

2. Because they are loosely coupled, PMSs are not designed for the implementation of organisational strategies.

3. Clear and measurable targets will be used in moderation because of the measurement challenges inherent to non-financial targets (see Van Dooren, 2017, p. 390 on this issue).

4. Performance indicators serve as the basis for monetary staff rewards only moderately.

5. Hybrid, social and nonprofit organisations adopt performance indicators with a medium intensity.

6. Performance indicators generally are used for external purposes rather than internal steering.

Despite the breadth of these findings, there remains a need for empirical exploration in this direction (Grossi et al., 2017; Haigh et al., 2015; Ebrahim et al., 2014; Costa and Pesci, 2016) in order to increase knowledge regarding accountability and PMSs for social and nonprofit organisations (Gray et al., 2001). In order to fill this gap in the literature and to provide an in-depth empirical investigation, this paper presents the development of a contingency framework of PMS based on Ebrahim and Rangan (2010) impact value chain and the multi-constituency approach of Costa and Pesci (2016). This paper aims to investigate the multidimensional nature of PMS and its link to outcome performance indicators downward to beneficiaries.

3. Methodology and method

3.1 Participatory action research (PAR)

This study employs PAR (Chiu, 2003; Mackenzie et al., 2012) because it brings about organisational change by actively involving employees in the formulation of solutions to address the identified problems. The social enterprise under investigation contacted the research team in order to identify and solve its difficulties together, designing and implementing a PMS that incorporates outcome and impact indicators. Therefore, the research team proposed that they adopt PAR in order to create dialogue and induce critical thinking among participants, generating awareness of the possible solutions that could be put into practice.

By adopting PAR, the research team and the social enterprise's employees worked together to co-generate knowledge through ongoing communicative processes and the joint implementation of findings. PAR adopts focus groups as the preferred method of analysis. In focus groups, members are involved and engaged throughout the research stages, from problem definition to data analysis and knowledge transfer. In order to be effective, the development of focus groups within PAR requires the following: a long period before concluding the study (roughly six months); a small number of participants (approximately between five and eight); a system of multiple decision makers that actively involves all group members; and finally, all group members interacting in pairs.

The PAR consists of three different stages (Chiu, 2003):

1. Stage 1 (problem identification): during this stage, the group's activities primarily aim to facilitate the recognition of interests while exploring participants' emergent experiences and opinions. Also important at this stage is negotiation with stakeholders and construction of a relationship between the organisation and its community.

2. Stage 2 (solution generation): in stage 2, the group members should make explicit all the possible solutions to the identified problems and analyse the possible gaps that may arise when implementing the proposed actions.

3. Stage 3 (implementation and evaluation of the final solutions): this final stage follows a (variable) period of time during which the proposed solutions are carried out in the field. At this time, some secondary problems may arise, and they should be managed effectively. Sometimes targeted focus groups facilitate reflection on the intervention program and on the overall effectiveness of the project.

3.2 PAR in action in an Italian social enterprise

The PAR has been developed in collaboration with a social enterprise based in north-eastern Italy, with an average annual turnover of € 15 million. The social enterprise had 270 members and 670 workers in the period between 2016 and 2018. This NPO was created in 2014 after the merger of two social cooperatives providing different services: care to elderly people and transportation for disabled people. Currently, the social enterprise is formally registered in Italy as a social cooperative (Defourny and Nyssen, 2017) because it deals with urgent societal problems – i.e. the care-assistance of elderly people – and because the homecare staff workers, professionals and volunteers are all invited to become members. The multi-stakeholder social cooperative participates in various activities: homecare, residential and semi-residential services, remote rescue systems, remote control systems and a family “help-desk”. The homecare service has grown over the years in order to encompass the needs of different stakeholders. For example, the social enterprise today offers homecare and palliative services to elderly people with dementia. The social enterprise under investigation shares responsibility with the public government, which has entrusted service delivery to the social organisation through public contributions. Because of the recent global financial crisis, resource constraints increased (Hyndman and McKillop, 2018), and the public government began scrutinising the social enterprise in order to understand how they promote social impacts for different stakeholders. This situation has prompted increasing calls for transparency and performance indicators, perhaps in an effort to create the illusion of control (Van Dooren, 2017).

In response to these calls, a PAR was developed in order to assess new performance indicators.

Building on previous research regarding NPO accountability and action research (Costa et al., 2011) while applying PAR (Chiu, 2003), this research follows three different steps. In stage one, the available documentation (e.g. internal documents, previous surveys and publications) provided the information and indicators regarding the input, activities and output of the social cooperative. These indicators have been distributed among these different dimensions with the help of the 12 individual semi-structured interviews. Table 2 provides the results of these collective processes. The first stage made both the research team and the social cooperative aware that the organisation had a large amount of unstructured data focussing on the results of the activities carried out (output), rather than on the middle- and long-term effects (outcome and impact). In stage two, five focus groups were developed with internal primary stakeholders in order to build new outcome indicators. These outcome indicators were complemented with other internal documents in order to co-construct and co-design indicators that could be implemented due to data availability. In essence, we sought out low-hanging fruit, indicators that could be implemented with a minimum amount of effort. Finally, in the third stage, the results of the focus groups were shared with organisation personnel in order to discuss the implementation of the newly co-constructed outcome indicators, to collect feedback, to monitor the implementation status and to identify future avenues of work. Table 2 details the PAR stages.

The adoption of PAR presents various limitations (Mackenzie et al., 2012), and the research team encountered the following criticalities. First, PAR is time and resource intensive because it requires the development of close working relationships with participants. In addition to the face-to-face meetings and interviews shown in Table 2, the research team had weekly meetings via e-mail or telephone with the CEO, HR director and administration office representative. Second, PAR requires sensitivity to the relationship between “insiders” (the participants, in this case the social enterprise workers) and “outsiders” (the project research team). Therefore, in order to create a close relationship with the employees and to foster an openness to engage in the focus groups, we preferred conducting 12 individual semi-structured interviews in stage 1 with all potential participants in the focus group developed in stages 2 and 3.

4. Empirical evidence: the design and implementation of PMS in an Italian social enterprise

4.1 Multiple stakeholder approach to input, activities and output for homecare assistance services

Following on Ebrahim and Rangan (2010), the PAR first aimed to understand how the social impact value chain could be applied to the social enterprise under investigation. As highlighted by the previous literature, when organisational complexities increase, managers feel the need to deal with the situation by developing more bureaucratic procedures, such as indicators, metrics and performance targets (Van Dooren, 2017; Vakkuri, 2010). The social enterprise under investigation was increasing in complexity due to a changing environment, fuelling a sense of urgency to move towards performance indicator use. The expected results of the re-organisation and re-definition from the overall PMS included a simplification of these complexities via performance indicators that would nourish this desire for simplicity (Van Dooren, 2017).

During the first phase of the PAR, we explored a large amount of data within the social enterprise (i.e. annual report, staff management report, management system re-examination report, statute and management reports) in order to apply the social impact value chain to this context. Before creating and rebuilding ex-novo a PMS, we decided to position the different set of indicators already present in the organisation within the input, activity, output and outcome/impact chain. Moreover, in order to consider the viewpoints of different stakeholders, the analysis adopted a multi-constituency approach (Costa and Pesci, 2016). Table 3 shows the results of this activity and presents different indicators for various stakeholders for the input, activity and output dimensions.

Input is defined as the knowledge, equipment and financial resources used to support the activities or processes of service delivery (GECES-Sub-group on Impact Measurement, 2014; EVPA European Venture Philanthropy Association, 2013). Within the social enterprise's existing documents and reports, we found several input indicators related to different stakeholders (i.e. the workers/employees, the members, the beneficiaries and the institutional stakeholders). In this stakeholder category, the social enterprise monitors the number of people engaged in the provision of the homecare service (both employees and members), the financial resources adopted and the buildings in which the homecare assistance is delivered.

All of these resources (mainly human and financial) led to a set of activities that could be defined as the set of initiatives undertaken by the social enterprise in order to create value for beneficiaries, the elderly (GECES-Sub-group on Impact Measurement, 2014). The social enterprise carries out many activities, as detailed and described in narrative-style reports. The orientation towards homecare beneficiaries is apparent throughout the various material analysed, with attention to describing and carrying out a set of activities. By analysing the activities carried out, it is also possible to consider the output of the social enterprise, which reflects initiative results (Ebrahim and Rangan, 2010).

In sum, the first stage of analysis revealed that assessing social performance and impact represents a key challenge in the Italian social enterprise under investigation. Indeed, as also highlighted in previous studies (Ebrahim et al., 2014), the social enterprise under investigation over-emphasises short-term, quantitative outputs and upward accountability to donors (Edwards and Hulme, 1996; Christensen and Ebrahim, 2006). While nonprofits often find it easy to develop and monitor economic and financial indicators (Epstein and McFarlan, 2011), they consider it very difficult, if not impossible, to quantify socio-economic, environmental and social effects (Mair and Martì, 2006). Therefore, the main problem identified in this first stage of investigation is an inability to utilise PMS in order to collect information about outcomes and impacts (Costa and Pesci, 2016) or to convey possible indicators for measuring the short- and long-term effects of these activities. The social enterprise has identified the need to take major steps in this direction and to develop useful and meaningful measures that capture their social impact and reflect the objectives pursued. The purpose of the second stage therefore was to engage all members of the focus group in order to design, propose and monitor possible outcome indicators.

4.2 The implementation of outcome indicators and the link to the organisational mission

In order to develop new outcome indicators and find possible solutions to the problems identified in the first stage, the social enterprise decided to better link the PMS system with the overall organisational mission (Grossi et al., 2017). During the focus group, it emerged that financial data and information should be supplemented by measures related to organisational mission achievement (Epstein and McFarlan, 2011; Christensen and Ebrahim, 2006). After discussing the social enterprise's mission during the first focus group, we utilised subsequent meetings to converse about the main activities of the homecare assistance and their short-term effects, i.e. the outcomes. The working group decided to focus on the construction of outcome indicators rather than impacts because of the inability to control all of the variables related to measuring impact (see Ebrahim and Rangan, 2010). Indeed, it would be difficult to measure the long-term effects of their activities in avoiding marginalisation effects; in fact, the social enterprise would have to manage macroeconomic variables beyond its direct control and often beyond the timeframe of its programs.

4.2.1 Discussing and redefining the mission and organisational strategies

Consistent with previous researchers (see Christensen and Ebrahim, 2006), we used the focus group to discuss the idea that upward accountability requirements to donors do not necessarily yield improved mission achievement. Therefore, the members decided to manage the tensions between upward accountability and mission. As an initial purpose, the focus group interacted with the definition of mission present in the social enterprise at that time, as presented in the statute. According to that document, the mission of this social enterprise is “to ensure the permanence of elderly people in their home and in their living environment, by delaying hospitalisation as much as possible and avoiding marginalisation” (Social Enterprise, Statute, p. 14).

When approaching this definition, the members of the focus group and the homecare coordinators expressed disagreement with the above-mentioned statement because “it does not include the work carried out by all the homecare assistants and workers in order to support the family caregiver” (Focus group, interview #7). Therefore, the first focus group developed a redefined mission able to convey the mission-based activities (Christensen and Ebrahim, 2006) that create value and impact for stakeholders downward to beneficiaries (Najam, 1996; Christensen and Ebrahim, 2006). The focus group proposed redefining the mission using a “bullet-point definition” in order to make explicit the different objectives and purposes of the social enterprise's homecare activities. Members of the focus group identified five mission-based activities as primary drivers of social outcome/impact creation: 1) how provision of homecare service is delivered; 2) homecare workers' level of education and professionalism; 3) the voluntary training of homecare workers; 4) the definition of the individual homecare program to beneficiaries and 5) family caregiver support (see also Table 4).

After redefining the mission and the five core mission-based activities able to lead outcome and impact, the focus group worked intensely with the aim of “building specific indicators focused on the effectiveness of the organisational mission, which is oriented to create social value to beneficiaries” (Focus group, interview #1).

4.2.2 Planning and delivering the homecare service to elderly people

The social enterprise's PMS use at the start of our research project was designed for external purposes. It basically sought to monitor and demonstrate to the donors (i.e. the public funders) the total number of hours delivered by employees, particularly 1) direct homecare workers (92% of the overall hours in 2016) and 2) managers and coordinators (8% in 2016). However, no information was available with regards to the effectiveness of their overall efforts. Therefore, during the second focus group, a need emerged to “make visible the value that the social cooperative provides to elderly people” (focus group, interview #3). Thus, the focus of the discussion moved from the overall amount of hours contractually delivered to the specific activities that the social enterprise designed when providing homecare service in order to “recognize the effort in designing and preparing the service, beyond the direct homecare service, thus making visible the overall number of hours dedicated to every single beneficiary” (focus group, interview #6). During the second and third focus groups, participants came up with the idea of identifying and calculating the average number of service hours by beneficiary. To do so, we would consider the social enterprise design and organise homecare assistance following the steps below.

1. First contact with the potential beneficiary. The coordinator manages the first contact with the public government or directly with the family. This first contact does not correspond to commencement of service with the family. For this reason, the work carried out in this step cannot be considered in the total number of hours per beneficiary.

2. First joint appointment between the coordinator, family, public government social worker and doctor at the beneficiary's home. This type of appointment is offered periodically in order to monitor the service; thus, these hours need to be included in the new indicator.

3. First visit with coaching. On the first day of homecare delivery, some coaching activities occur in order to engage multiple homecare workers with the same beneficiary. These hours also need to be included in the new indicator.

Table 5 indicates the final calculation of the new outcome indicators identified with reference to the homecare assistance service. The members of the focus group agreed upon the idea that, even if the coaching activities count for 0.5% of the overall amount of the homecare assistance hours devoted to beneficiaries, the PMS needs to include such information in order to make the stakeholders “aware of the way in which homecare activities are designed, planned an realized” (focus group, interview #1).

4.2.3 Professionalism of the employees/homecare workers

Members of the focus groups reflected on the idea of professionalism in two directions: 1) downward to the beneficiaries and 2) lateral to the staff and to the organisation itself (Najam, 1996). The idea to prioritise downward and lateral accountability can help support staff members in reconciling multiple accountabilities and in staying focused on the mission (Christensen and Ebrahim, 2006). Indeed, during the focus groups, the Vhomecare coordinators highlighted the need to “be responsible with other colleagues in order to be responsible with the beneficiaries” (focus group, interview #4). As such, they focused on the idea of being professional and fair “within the organisation, with other colleagues, in order to be timely, precise and accurate both in developing the job and in communicating with all other colleagues” (focus group, interview #2).

The analysis thus supported the idea that “an organisational environment that empowers and supports staff members can lead to improved accountability on all levels while better ensuring mission achievement” (Christensen and Ebrahim, 2006, p. 206).

In order to design the PMS with outcome indicators, the members of the focus groups proposed various indicators for professionalism, with the following results.

1. Professionalism downward to the beneficiaries:

   • Number of homecare workers by seniority and qualifications

   • Amount of non-compliance among beneficiary and homecare workers

   • Timely communication with the beneficiary

2. Professionalism lateral to the staff and the organisation itself:

   • Timely communication with the social enterprise and with other employees

3. o missed card punch

4. o tardiness over 15 minutes

5. o leaves and permissions

   • Flexibility of homecare workers in terms of planning their day.

Table 6 represents the number and percentage of homecare workers by seniority and qualification. Focus group members defined seniority as “direct experience with elderly people for more than 24 months” (focus group, interview #1).

A total of 94% of homecare workers demonstrate both seniority and high qualification, and therefore, the social enterprise decided to continuously monitor this data because it captures the quality of the service they provide to beneficiaries. As member #1 explained during the focus group, “For our social enterprise, outcome indicators represent the opportunity to reflect on the quality of the homecare service, rather than quantity of activities carried out […] To date, the focus of our accounting system was based on how many hours we deploy to beneficiaries, but today it's time to turn to quality effort and outcome measurements”.

Along with the quality information related to seniority and qualifications, the social enterprise would monitor the amount of non-compliance, which was approximately 60 per year in 2016, with an average of five per month. The organisation also sought to monitor timely communication, with timely defined to mean within 24 h of the event.

Regarding lateral accountability and internal professionalism toward staff and the overall organisation, it should be noted that this time period saw implementation of an indicator related to homecare workers' flexibility. Flexibility and availability are relevant aspects in homecare service because possible unforeseen events can occur. Occasionally, homecare workers are asked to provide services that would not be within their schedule. From the data collected in the first quarter of 2017, the social enterprise became aware that, on average, their homecare workers are flexible more than four days in a month, allowing coverage due to unexpected events and consistently satisfying beneficiaries' needs.

4.2.4 Training of the homecare worker

The focus group worked on the idea of implementing new and reliable indicators to monitor the training of homecare workers. As interview #3 of the focus group highlighted, “Training is a pillar of the social cooperative's activities due to the changing environment the homecare workers face in their day-by-day work […] The physical, health and mental status of the beneficiaries can continuously change, and homecare workers need to adapt to new challenges”.

In greater detail, the focus groups discussed the role of homecare workers' voluntary training. Their willingness to participate in such voluntary activities is regarded as a professional attitude both internally to the organisation and externally to beneficiaries.

As highlighted in Table 7, almost 40% of the total training courses hours offered by the social enterprise to employees in 2017 (both admin staff and homecare workers) were voluntary. However, the analysis in terms of course number shows that 16 out of 19 were mandatory training courses in 2017. These courses were devoted primarily to homecare workers in order to help them manage the increasing number of beneficiaries affected by dementia. Members of the focus groups highlighted that the social cooperative is struggling to offer as many mandatory courses as possible to “make some knowledge and some skills compulsory” (focus group, interview #1); at the same time, the organisation seeks to increase the homecare social workers' awareness of voluntary training activities.

4.2.5 Individual homecare program

The individual homecare program (labelled PAI) is the official document containing the agreement between the family/beneficiary, the public government and the social enterprise related to individual homecare activity. It contains detailed information about three domains: 1) the list of specific homecare activities that the homecare workers have to deliver, 2) daily, weekly and monthly planning for the duration of the contract and 3) the homecare workers' roles and engagement. The initial definition of the PAI brings together various stakeholders' needs and perspectives. Following this definition, members of the focus groups highlight the importance of PAI revisions and adjustments. Indeed, according to participants, “When the PAI is revised and adjusted, it means that the social enterprise is adapting and tailoring the service to their beneficiaries, thus increasing the social value of their activities” (focus group, interview #3). It may not be possible to find a causal relationship between the number of PAI revisions and the increased social impact created because of the multitude of factors that can affect improved conditions for beneficiaries (Ebrahim and Rangan, 2014; Costa and Pesci, 2016). However, it is arguable that the number of PAI revisions is an “indicator of the attention to the customisation of the homecare service” (focus group, interview #4).

Table 8 shows that the number of PAI revisions and adjustments of the PAI averages around 1.4 times during the lifetime of the PAI (approximately 3.2 years). However, focussing on contracts with duration of 3 years or more shows that the number of PAI changes and adaptations increases to almost 5, thus enlarging the managerial complexities of the homecare service.

4.2.6 Family caregiver support

Participants considered family caregiver support to be a “key driver of the revised mission of the social enterprise” (focus group, member #1). To date, the social enterprise has not monitored, evaluated or managed the support activities they implement with the family caregiver; however, the focus groups indicated that they conduct three main activities with the family caregiver:

1. Human and psychological assistance. The social enterprise provides various forms of human and psychological assistance to the family caregiver who often co-designs and co-implements the homecare service activity with the professional homecare worker. Most of this assistance relates to phone calls and phone contacts as well as meetings at the social enterprise's offices or beneficiary's home.

2. Knowledge transfer. With strong professionalism, the homecare workers involve the family caregiver in delivering the service in order to grant the family independence in taking care of their elderly relative. The workers provide knowledge related to hygiene, caring and mobilisation. Though knowledge transfer is not required in the formal PAI designed with the public government, the social cooperative considers it important to the success of the homecare service.

3. Specific training activities. The social enterprise under investigation is planning specific training initiatives for family caregivers.

For all of these family caregiver support activities, the social cooperative, with the help of the researchers, has designed specific questionnaires to be delivered in the implementation phase of the PAR in order to monitor these activities.

4.3 Fine-tuning and implementation of the PMS

By implementing the PAR and five focus groups, the social enterprise under investigation was able to build new and refined outcome indicators for assessing accountability mechanisms, mainly lateral to homecare workers and downward to beneficiaries (Christensen and Ebrahim, 2006; Najam, 1996). Table 9 shows the implementation status of the fourteen indicators identified during stage two of PAR; these indicators were discussed during the last focus group in March, 2018.

The organisation has fully implemented eight indicators primarily related to 1) planning and delivering the homecare service, 2) ensuring the professionalism of the homecare workers, 3) training the homecare workers and 4) delivering the individual homecare program. Our questionnaires to families have yet to be delivered, but they will assess the quality of the family caregiver support system. By analysing the implemented indicators, we gain an understanding of the empowerment resulting from collective decision making among staff members; this process allows for felt responsibility, guiding accountability practices in social and nonprofit organisations (Christensen and Ebrahim, 2006). Moreover, as highlighted during focus group #4, being accountable and responsible internally allows homecare workers to focus on doing their jobs well; this process includes multiple accountabilities to achieve mission and mission-oriented activities for beneficiaries.

The implemented indicators remain much weaker in downward relationships with beneficiaries and families of beneficiaries in terms of assessing the quality of the homecare service provided. Consistent with previous literature (Ebrahim and Rangan, 2010), it might be necessary to get a better handle on balancing different indicators for various stakeholders, especially beneficiaries. Ultimately, the real long-term goals of social and nonprofit organisations might extend well beyond the boundaries of the organisation itself.

5. Conclusions and contributions to the literature

This paper has adopted the value impact chain (Ebrahim and Rangan, 2010) as well as the multiple-constituency theory (Costa and Pesci, 2016) in order to investigate the multidimensional nature of nonprofit PMS and its link to outcome performance indicators downward to beneficiaries. In order to pursue this aim, the paper followed a PAR project (Chiu, 2003)within an Italian social enterprise. The researcher has been engaged with members of the organisation in order to create strong group empowerment and commitment. With the aim of increased dialogue, participation and engagement, the researcher adopted focus groups and documental analysis as preferred methods of investigation from July 2016 to March 2018.

The paper highlights that the identification of outcome and impact indicators remains difficult in terms of both accountability and PMS in the nonprofit sector (Grossi et al., 2017; Haigh et al., 2015; Ebrahim et al., 2014); this challenge stems from the multi-stakeholder nature of the organisations (Costa and Pesci, 2016). However, several conclusions and contributions to previous studies can be drawn from this paper. First, the paper encourages the idea that the impact value chain can create an organisational vocabulary (Clark et al., 2004) able to differentiate outputs from outcomes. Indeed, our findings reinforce concerns related to the possible inability of social enterprises to have specific expertise or resources to evaluate outcomes and impacts achieved. A lack of skills, training and competences may inhibit social enterprises path to social impact measurement; however, through the PAR, the social enterprise can begin to define the desired outcomes and ascertain which internal output measures are most likely to correlate with desired outcomes (Clark et al., 2004). Moreover, the adoption and implementation of this new vocabulary supported the idea that it was not feasible, or even desirable, for the social enterprise under investigation to develop metrics at all levels of the impact value chain (Ebrahim and Rangan, 2010). Indeed, the designed PMS omits impact metrics because they are typically beyond the organisation's control and roles. Therefore, the paper empirically supports the normative idea that outcome measurements are less commonly developed in nonprofit organisations, given that they have the most control over their immediate activities and outputs (Ebrahim and Rangan, 2010). Nonetheless, the impact value chain vocabulary helps them assess both output and outcome measurements.

Second, the empirical evidence collected in the paper and the solutions designed in the second stage of the PMS highlight the central role of organisational mission in supporting PMS assessment (Bryan, 2019; Grossi et al., 2017; Ebrahim and Rangan, 2010; Christensen and Ebrahim, 2006; Najam, 1996). Indeed, the findings indicate that the designed outcome measurements are effective when aligned to mission-based activities. Therefore, the social enterprise under investigation assesses PMS, which is effective for multiple stakeholders (i.e. homecare workers and beneficiaries). This tool is helpful because of its measurement systems and structures that support achieving organisational mission (Ebrahim and Rangan, 2010; Christensen and Ebrahim, 2006). Indeed, activities that provide opportunities for feedback from lateral and downward stakeholders (i.e. homecare workers and beneficiaries) served to broaden the social enterprise's accountabilities while also ensuring better mission achievement. The discussion around the mission and the mission-based activities was the starting point for the focus group in stage 2 in order to find solutions for outcome measurement and to improve PMS. Therefore, the paper supports the idea that lateral accountability can enhance PMS systems and downward accountability when the organisation is oriented toward mission-based activity (Christensen and Ebrahim, 2006). Moreover, the paper reinforces the idea that accountability and PMS oriented to the original mission help social and non-profit organisations to triggers a sense-making process that helps the employees of the organisation to rationalise and institutionalise their social awareness and maintain their commitment toward the organisational mission, thus preventing the mission drift (Ramus and Vaccaro, 2017).

Third, the paper reinforces the idea of a multiple-constituency approach to PMS and social impact measurement (Costa and Pesci, 2016; Grossi et al., 2017) because of the large variety of purposes and stakeholders in social and nonprofit organisations. The empirical data collected in the focus groups allowed for a better understanding of how multiple accountabilities (i.e. lateral to homecare workers and to the organisation itself and downward to the beneficiaries) to different stakeholders co-exist within the social enterprise's organisational life. It also provides insight into how these multiple-accountabilities jointly collaborate to meet the social and nonprofit organisation's mission. The paper thus argues against a standardised and universal metric for measuring the performance of social and nonprofit organisations (Costa and Pesci, 2016; Palmer and Vinten, 1998), and it does not support the adoption of monetary proxies to measure the impact of such organisations (Arena et al., 2015).

Fourth, the paper also contributes to hybrid organisations literature (Grossi et al., 2017; Ebrahim and Rangan, 2014; Haigh et al., 2015; Defourny and Nyssen, 2017) by introducing a case of a social enterprise which adopts blended PMS in order to consider its hybridity in terms of social value orientation and financial constraints. Indeed, as highlighted by previous studies, hybrid organisations include social enterprises and social cooperatives (Defourny and Nyssen, 2017; Grossi et al., 2017) and the paper presented one case of multiple-stakeholder social cooperative which combines the need to meet financial goals upward to the donors, with the non-financial measurement downward to the beneficiaries in order to measure the impact and furthermore the lateral accountability to promote a strategic approach to PMS. The multi-stakeholder social cooperative analysed in this paper contributes to the conceptualisation of hybrid organisations because it has been designed in a multidimensional way by combining three important dimensions: economic, social and mission-based achievements (Andreaus and Costa, 2014).

In conclusion, this paper has discussed the efforts of an Italian social enterprise to design a PMS not only focused on input and financial metrics (Van Dooren, 2017) but also incorporating outcome metrics. Indeed, in order to measure organisational success, the social enterprise decided to focus primarily on mission and mission achievement by defining lateral and downward multiple-accountabilities with the ability to demonstrate how the organisation is (or is not) achieving the outcome measurement. Therefore, the analysis supported the idea that “an organisational environment that empowers and supports staff members can lead to improved accountability on all levels while better ensuring mission achievement” (Christensen and Ebrahim, 2006, p. 206). Combined with financial performance measurements, these outcome measurements allow organisations to take a more informed view of their performance while better understanding how they affect the communities that they serve (Epstein and McFarlan, 2011).

As for managerial and practical implications, our findings imply that the PMS should be loosely coupled with organisational strategies (Grossi et al., 2017). Indeed, this paper suggests that the re-positioning of the mission and mission-based activities at the “core” of the social enterprise's PMS necessitates the creation of a culture of lateral and downward accountability. This approach will reinforce the defining of long-term internal strategies and the well-being of the workers while increasing awareness of the impacts on beneficiaries. Mission-based activities should be internally-oriented rather than focused on upward accountability and external scrutiny. Moreover, the paper helps social enterprises in defining new vocabulary by differentiating between three terms: output, outcome and impact. These terms should not be considered interchangeable. By recognising different indicators for each of them, social enterprises can successfully grow their social impact measurements. Finally, the paper suggests that social enterprise managers should adopt different indicators tailored for each output, outcome and impact dimension in order to better integrate them in day-by-day organisational activities.

Table 1

A review of studies on PMS in social and nonprofit organisations

Author/Year	Method	Main contributions
Simons (1995)	Levers of control framework (belief systems, boundary systems, diagnostic control systems, interactive control systems)	To use controls in order to implement and develop strategies To help managers identify opportunities and solve problems
Kaplan and Norton (1996)	Balanced scorecard (BSC): translating visions and strategies into aims and indicators. Considering financial metrics plus three additional perspectives: customer, internal process and learning and growth	To measure performance by creating links between daily actions carried out by organisations and their objectives To include intangible assets in the analysis (in addition to financial resources)
Kaplan and Norton (2001)	Adjustment of the original version of BSC	To create a useful tool for nonprofit organisations
Moore (2003)	Public value scorecard: adaptation of Kaplan and Norton's balanced scorecard with a focus on the “public value strategy” instead of competitive strategy. The public value scorecard considers a “strategic triangle,” including three key elements: social mission, legitimacy and support and organisational capabilities	To give managers the ability to consider all relevant aspects for the implementation of a strategy in the nonprofit context
Somers (2005)	Social enterprise balanced scorecard (SEBC): adaptation of the Kaplan and Norton balanced scorecard for social enterprises	To provide a tool for social organisations considering social goals above the financial perspective, the financial perspective with a focus on sustainability and a broadened customer perspective capturing a large number of stakeholder groups
Bull (2007)	Adaptation of Kaplan and Norton's balanced scorecard for social enterprises	To integrate social aspects in management control To provide an easily adaptable tool for the multi-stakeholder context characterising social organisations
Mook et al., (2007)	Expanded value added statement (EVAS): based on a traditional accounting statement but modified to include social and environmental items	To include the concept of intellectual capital in the social accounting model To report the social value created in addition to the economic value added
Ebrahim and Rangan (2010)	Contingency framework: a matrix referring to two characteristics of an organisation, the theory of change and the operational strategy, giving four broad types of results: niche, integrated, institutional and ecosystem	To measure social results and to examine the management control of organizations operating within the social sector
Bagnoli and Megali (2011)	Multidimensional control model: analysis of economic and financial performance, social effectiveness and institutional legitimacy	To provide a performance measurement system based on a multidimensional control model for all social enterprises

Table 2

The implementation of PAR in an Italian case

	Stage 1: Problem identification (Jul 2016–Jan 2017)	Stage 2: Solution generation (June–August 2017)	Stage 3: Implementation and evaluation (dec 2017–Mar 2018)
Documents analysed	- Annual financial statement - Staff management report - Management system re-examination report - Statute - Charter of services - Management report - Annual report on elderly day care centres for fiscal year 2015	- Internal reports from healthcare professionals - Internal planning for elderly people's individual plans - Spreadsheet system for managing the agenda of different healthcare workers
Interviews/focus groups	12 informal individual interviews were performed with the CEO, the president, the director and nine internal workers at the social cooperative	5 focus groups with 11 members, including the CEO, HR director, the director of personnel services, the administration office representative, two technical representatives and three homecare coordinators. Finally, two University of Trento researchers took part in the focus groups	1 focus group with 11 members (the same as stage 2)
Issue identified	The documental analysis, along with the internal interviews, helped the research team to understand that the social cooperative deals with a large volume of data and information which do not correspond to any strategic decisions. Moreover, by developing input/activities/output/outcome and impact value chain analysis, the research team highlighted the lack of outcome and impact indicators and the need to better allocate the current data available within the social impact value chain of Ebrahim and Rangan (2014)	The researchers thoroughly analysed what emerged up to the construction of a limited number of indicators and variables that, if introduced, allow the drastic reduction of documentation while also considering non-financial variables in the PMS	The prearranged indicators were presented to the focus group through an interactive presentation (during which doubts and perplexities emerged). This situation led to the correction of certain elements. The organisation should take the next step, the task of starting to monitor the proposed indicators, eliminating all unnecessary data. Then it would be desirable for the cooperative to include the variables among those already considered in management control. In this way, it could extend its analysis from financial variables only to the inclusion of social variables

Table 3

Input, activities and output indicators by stakeholder following a multiple-constituency approach

Stakeholder	Input	Activities	Output
Employees	INDICATORS -Total number of employees -Nationality -Gender -Type of contract -Type of position (full-time/part-time) -Staff turnover -Age -Seniority -Work environment -Work support services HIGHLIGHTS 350 total workers 145 working members 300 female workers 241 homecare workers 300 workers with fixed positions 203 part-time positions	INDICATORS -Working hours -Training hours -Workable hours by different activities HIGHLIGHTS 336,683 worked hours (84% of the total workable hours) 2,574 annual training hours 5.22 training hours per worker	INDICATORS -Average monthly salary and wages -Compensation by type of activity and employee position -Economic value of voluntary work -Number of promotions -Amount of compliance (total, managed, causes) -Number and type of non-compliance -Service supply process indicators -Employee professionalism indicators -Attention to welfare of worker indicators HIGHLIGHTS 93% of the workers have a salary between 500–1,400 € 2,015 volunteer hours per year 41,320 € economic value of volunteering 74 non-compliant
Members	INDICATORS -Total number of members -Social categories -Equity -Members' loans HIGHLIGHTS 218 total members 242,042 equity	INDICATORS -Membership fees -For those members who are also workers -Worked hours -Training hours -Workable hours by different activities	INDICATORS -Revenues by strategic area HIGHLIGHTS 6,642,212 total homecare revenue
Beneficiaries	INDICATORS -Total number of beneficiaries by age, gender, type of health-issue, self-sufficiency level, family situation, area -Beneficiaries monthly trends -Number of activated/cancelled contracts with beneficiaries HIGHLIGHTS −1,169 total beneficiaries −730 elderly women −120 independent/autonomous −491 partially independent/autonomous −415 not independent/autonomous	INDICATORS -Homecare activity description with detailed report by type of beneficiary and purpose HIGHLIGHTS -Daily homecare assistance -Soup kitchen -Beneficiaries -Hairdresser and hygiene -Beautician -Counselling -Health training -Recreational activities	INDICATORS -Duration of homecare assistance service -Main motivations for interruption of the homecare service -Beneficiaries' feedback -Level of satisfaction indicators -Annual average of beneficiaries HIGHLIGHTS −40% of the homecare contracts exceed 24 months −46% of the contracts expired because of the death of the beneficiary −82% of beneficiaries are satisfied or extremely satisfied
Institutional stakeholders	INDICATORS -Public contributions -Grants -Loans HIGHLIGHTS −335,200 € loans −7,557,246 €Public Government contributions		INDICATORS -Supplied hours by type of institutional stakeholders (Public Local Government, Public National Government, Private)

Table 4

Redefinition of mission and explication of the social enterprise's activities

Redefined mission	Activities
1. To ensure the permanence of elderly people in their home and in their living environment 2. To delay hospitalisation as much as possible 3. To avoid marginalisation 4. To support the family caregivers	1. Planning and delivering homecare services to elderly people 2. Ensuring professionalism of the employees/homecare workers 3. Training the homecare workers 4. Providing the individual homecare program 5. Giving family caregiver support

Table 5

Number of overall hours by beneficiary (first quarter 2017)*

	Jan 2017	Feb 2017	Mar 2017	Apr 2017	May 2017	Jun 2017	Total
Direct homecare assistance (H)	15.206	14.776	16.687	15.036	16.248	15.679	93.632
Joint appointments and coaching (H)	42	51	62	79	99	115	447
Total (H)	15.248	14.827	16.749	15.115	16.347	15.794	94.079
n. beneficiaries (B)	915	926	943	947	954	973	5.657
Monthly H/B	16.66	16.02	17.77	15.96	17.14	16.24	16,63
Weekly H/B	4,17	4,01	4,44	3,99	4,28	4,06	4,16

Note(s): * The data were collected during the second stage of the focus group in order to test the feasibility of collecting the information

Table 6

Number of homecare workers by seniority and qualification (updated July 2017)

Professionalism: Seniority and high qualification	Number	Percentage
Total homecare workers	251	100%
Total homecare workers with seniority (S)	208	83%
Total homecare workers with high qualifications (HQs)	150	60%
S and no HQ	81	32%
No S and No HQ high qualification	16	6%
S and HQ	235	94%

Table 7

Hours of activity training (mandatory and voluntary) in 2017

Activity training	Administrative staff	Homecare workers	Total	Percentage
Mandatory training	87.75	1,512	1599.75	62%
Voluntary training	242.75	742.5	985.25	38%
Total activity training	330.5	2254.5	2,585	100%
Training by workers	6.0	3.9
Voluntary training by workers	4.4	1.3

Table 8

PAI revisions and adjustments (first quarter 2017)

PAI revisions
Average duration of the homecare service (in years)	3.2
Number of PAI revisions during the duration of the contract	1.4
Average duration of the homecare service (in years) > 3	3.4
Number of PAI revisions during the duration of the contract >3	4.6

Table 9

The implementation of outcome indicators in the PMS

Activities		Outcome indicators	Status of implementation
Planning and delivering of the homecare service to elderly people		I1. Overall and average number of service hours by beneficiary (including contact, joint appointments and coaching activities)	Fully implemented
Ensuring professionalism of the homecare workers	Professionalism in terms of the quality of service provided to beneficiaries	I2. Number of homecare workers by seniority and qualification I3. Amount of non-compliance among beneficiaries and homecare workers I4. Timely communication with the beneficiary	Fully implemented
	Professionalism of the staff and the organisation itself	I5. Timely communication with the organisation and among staff missed card punch tardiness beyond 15 minutes leaves and permissions I6. Flexibility of the homecare workers in terms of planning theirs days	I5 and I6 have both been partially implemented. In more detail, the social enterprise has designed a specific spreadsheet in order to manually collect information to build I5; however, timely updating of this information has been lacking. The organisation is working with a software house to hasten data collection. I6 was manually implemented with a spreadsheet
Training homecare workers		I7. Availability of workers to participate in optional training I8. Use of role plays upon completion of training	I7 was implemented as an indicator. In order to increase the availability of homecare workers to attend the voluntary training, the social enterprise is trying to offer additional voluntary courses I8 has been implemented with one role play event in 2018, which helped to investigate the effectiveness of the training courses
Providing individual homecare program		I9. Inclusion of information concerning every rescheduling and/or service modification	I9 has been partially implemented
Giving family caregiver support	Human and psycological support	I10. List of activities in support of users' family members I11. Submit a questionnaire to measure the effectiveness of the provided support activities	I10 has been fully implemented, but I11 (the questionnaire) has not been submitted to families
	Skills transfer	I12. Submit a questionnaire to measure the capability of the worker to transfer skills the user's family members to understand	Not implemented
	Training activities	I13. Activate training activities for users' family members I14. Submit a questionnaire to monitor training activities	Not implemented