Participation - finding out what difference it makes

About the research

Purpose of the research

We wanted to find out what is already known about the difference that service user and carer participation can make and how people have gone about investigating this. We were especially interested in how the difference was being measured.

When you ask a question such as ‘how do you find out whether participation is making a difference?’ you find yourself asking even more questions rather than providing one short answer. So, the guide is not so much about answers but about making better sense of the questions. To do this, we will present ‘nine big questions’ that the research suggests are important to ask when finding out whether participation has made a difference.

The research methods

In order to find out what is already known about this topic, we:

• searched for reviews of this topic (the work that other researchers have done to compile knowledge about evaluation of participation)
• read and digested 30 reviews in total
• reviewed twelve practice guides to evaluating participation, which we call toolkits
• sent a short questionnaire to 1,599 different social care organisations in England, Wales and Northern Ireland (we had 30 replies)
• issued a press release about the research (we received 12 responses)
• interviewed key people at ten practice sites where they had done or were doing evaluations of participation
• were steered by an Advisory group of eight service users and carers, facilitated by a service user–researcher.

Understanding the research findings

We discovered that much more has been written about how people participate than about how we find out what difference participation makes. Even so, some of the methods used to help people participate can also be used to help find out what difference it has made. We have collected all of this information into the ‘nine big questions’. We hope this will help both to understand the findings and to make practical use of them.

Crawford et al (2002) (R08) point out that the ultimate goal of service user participation should be the promotion of health, quality of life, or overall user satisfaction with services. However these outcomes are often difficult to measure, they can take a substantial amount of time to become evident, and the link with the participation of services users and carers can be difficult to prove. These can be barriers to evaluation (see big question 2). As a result, evaluations tend to use short term indicators.

Crawford (2003: p79) (R01) developed four categories of outcome, where these have been evaluated:

• increased satisfaction with services
• promotion of further user involvement initiatives
• improved management
• changes to service priorities.

Evaluations can be usefully considered as focusing on ‘voice’, ‘choice’, and ‘change’ (R09). Each of these constructs lends itself directly to an evaluatory question: ‘did they listen?’, ‘did I get what I wanted?’ or ‘did the service change?’ Our research suggested two kinds of benefit from participation – intrinsic and extrinsic. Intrinsic benefits come from the process of participation itself, such as improvements to self-esteem and changes in attitude. How much these intrinsic benefits are valued varies, with Truman (2005: p572 (R14) suggesting that ‘user involvement should not be seen as an end in itself but rather it is a means of enabling people to make choices and have control over their daily lives’. So the intrinsic value of service user participation might also have an impact on extrinsic changes; for example, increased self-confidence gained via the process of participation (intrinsic gains) might be necessary before people have confidence to campaign for specific changes to a service (extrinsic gains).

An example of an intrinsic benefit of participation is: ‘children have said that having a say is more important than getting what they want’ (R07).

An example of an extrinsic benefit of participation is: ‘new services were developed and costs of care were reduced’ (R09).

Open communication is a crucial part of the process of evaluation, but the power to provide a service (and stop providing it) makes it difficult to have an equal relationship between service users, carers and professionals. This is why the question of who does the finding out (who evaluates) is so important (see big question 5).

Although there are many different methods that can be used to find out what difference participation is making, we do not yet have enough evidence to know which method is best for which situation (see big question 6). Networks, whether service user and carer or professional, are important in providing strength and support, but it is also necessary to reach out to people who are not part of a network or group (‘seldom heard’ people).