SCIE Resource guide 07: Participation - finding out what difference it makes
Big question 2: What stops us from finding out whether participation makes a difference?
What are the barriers to evaluation?
Summary
There is more information about what stops people from participating, than what stops people from evaluating (R15). The findings point to differences in power between service users, carers, workers and organisations as a barrier to evaluation. Consider which people are likely to have the most power in the house extension example in What is evidence of success? and how that might exclude some people from defining ‘success’.
Evaluation takes time, commitment, skills, resources and systematic planning, and if any of these are not available it is likely to prevent the evaluation from happening or from being successful. Evaluations need to be planned from the beginning, and costed into any proposals.
Findings box 2
- Differences in power can be a barrier to honest evaluation (R19) (R20) and evaluating means being prepared to accept findings that might change the power balance and may be contrary to current policies (R17).
- Real or perceived fear of the costs of evaluating can be daunting, along with concerns about additional costs that might be indicated by the findings (R17).
- Timescales may be different for different groups – service users, professionals, organisations, researchers.
- The main focus of evaluation may be elsewhere, e.g. driven by the terms of the project’s funding, which may not have specified evaluation of service user and carer participation (Site 2) (Site 3) (Site 4).
- Poor motivation to get involved in evaluation, perhaps because of ill health or past experience of it not making a difference (R02).
- Attitudes of staff may be hostile or unsupportive to evaluations (R05)
- The culture in the organisation is hostile or not supportive to evaluating participation (R06).
- It is difficult to prove that that this change is due to that participation (R08).
- Practical matters can prevent thorough evaluations, such as lack of transport in rural areas (Branfield et al, 2006).
- Psychological issues, such as seeing participation as something that you ought to do, whatever the result, so why evaluate it, can be a barrier. If participation is a requirement or a right what is the point of evaluating it? (R02) (R21).
- The tick-box mentality (if the box is ticked it feels like it has been done).
- An insufficiently clear plan for participation and evaluation means that there is nothing tangible to measure.
Ideas box 2
What lies behind these quotes? How might they prevent you from finding out what difference participation has made, if they are not confronted?
- Nobody ever asks the paid workers if their views are ‘representative’ ... can you imagine asking that in the middle of a meeting?
- Things are slow to change, but it’s getting better.
- If you complain they say ‘don’t threaten me’.
- If you’re taken seriously it makes you feel good.
- I’m the only voice.
- Without good information we can’t make the choice which would suit us best.
- When there are too many pages I can’t be bothered, so I put it in the bin.
- It took me a year to get the gist of the meetings – then I could contribute, but by then it was time to leave.
- People don’t go to McDonald’s to cook their own burgers.
- That’s how it is in all organisations.
