Monitoring, evaluation and impact: Making data work for communities

This project commenced in June 2014 and is ongoing.

The aim of the project and the partnership is to employ a collaborative, needs-led approach to building capacity within community data environments. We take an agnostic approach to tools and IT systems designed to support VCS organisation monitoring, evaluation and impact. This means that we do not seek to develop or promote any specific tool or method but instead engage with academic questions about what constitutes valid knowledge and legitimate testimony in order to broaden the ‘portfolio of evidence’ organisations and funders currently work with. 

We seek to raise questions of ethics, proportionality and dignity through asking whether the number and type of questions asked of people seeking support are appropriate to the encounter. Whilst we recognise and attempt to share existing expertise in the voluntary and community sector, we are also committed to building capacity and involving universities in the development of learning platforms that support positive social change.

In the next phase of this project, we will explore ways of working collaboratively with funders, commissioners and government to find creative and dignified ways of making data work for communities.

The key findings of our project are as follows:

• Through listening to voluntary and community sector (VCS) organisations describe the ways in which they work with data we found that the majority of information they collect is obtained from people who are seeking support but is intended for the organisations who fund those activities. The information requested by funders is intended to support their decision-making and ensure that funding from government and the money the public donates to charity is allocated and used in accountable, appropriate and transparent ways. The information also plays a part in the economic valuation of services, facilitating comparison of costs relative to outcomes achieved, otherwise known as ‘value for money’. 
• Within this context we discovered that funder preferences for certain forms of data can have profound consequences for the moments or ‘encounters’ in which people seek support. Unlike statutory services, contacts with VCS organisations serve unmet needs that may or may not be recognised by those experiencing them. The encounter or ‘moment of contact’ is therefore of tremendous significance as ‘seeking support’ becomes a form of ‘reaching out’ that could potentially form part of an experience of crisis or else could form an important site for raising awareness and informing. Either way, encounters or contacts are crucial and VCS organisations demonstrate tremendous expertise in their design and innovation of these engagement spaces.
• Our theoretical interest in ‘practices’ allowed us to look at these encounters, as described to us by volunteers and service managers, alongside requirements to meet funder requests for data. Thinking of ‘data as practice’ enabled us to consider the organisational and ethical consequences of requesting and producing data and enabled us to focus on the performance of data ‘in practice’. For example, funder preferences for numeric data become – ‘in practice’ - requests for people to complete questionnaires. This is an activity experienced by those seeking and offering support as ‘form-filling’. Since funders seek to target ‘eligible groups’ and specific levels of need, the information requested in such forms is often sensitive in nature and can be personally identifying. There are often multiple forms to complete especially where encounters and the spaces they take place in are funded by more than one funding body and each one requests that organisations use a questionnaire specific to their interests. Funders tend to act in silos and do not know what other funding streams underpin VCS activities and what data collection requirements are associated with them. Organisations reported how these conditions lead to a proliferation of data collection requirements which quickly become disproportionate to the encounter producing experiences that deter rather than encourage people from seeking support. VCS organisations reported feeling compelled to meet funder requests for data because without their support they could not continue to function. 
• In our work we argue that a performative view of how people work with data or ‘data practice’ provides a basis for recognising how ‘data burden’ occurs in community settings. It also provides a basis for thinking through and re-orienting requests for data around the encounters through which the needs of people reaching out for support are met with compassion and dignity. We make this argument in a position statement and forthcoming academic article in which, together with our research partners, we propose a ‘call for change’. Funder-recipient relationships mean that it is hard for VCS organisations to ‘say no’ to data collection requests but throughout this project we have heard organisations report that the alternative is a profound sense of ethical compromise experienced by service managers and volunteers who are told they must collect certain information.
• We have sought to address these issues by collectivising and anonymising VCS organisations' concerns and reporting them in a ‘position statement’ which we are seeking to disseminate and promote as part of this project.

Impact

Our position statement on these issues entitled ‘Monitoring, Evaluation and Impact: a call for change’ has been adopted by the Brighton and Hove Fairness Commission and features in their Final Report which will be publicly launched on 27 June 2016. It has also been included in Brighton and Hove’s Framework and Guidance for the Public Services (Social Value) Act 2012. We are actively seeking to continue discussion on community data burden with policy makers, funders, commissioners and VCS organisations at local, national and international levels.