Establishing outcomes to understand and measure impact

Building local outcomes for better care

Over the last decade, there has been a wealth of national, regional and local initiatives to deliver integrated care. Though challenging to deliver, integrated care is critical to achieve high quality and sustainable service now and in the future – not just in England, but across the UK, and in every major health and care system in the world. This section provides tips and tools for how to articulate the outcomes providers seek to deliver.

What do we mean by outcomes?

Outcomes are the benefits that are delivered as a result of a service. Outcomes can range from broad lifestyle goals (e.g. supporting people with chronic conditions to live active and fulfilling lives in their homes and communities) to specific quality of care outcomes (e.g. patient care in the right setting at the right time). Outcomes are fundamentally different from outputs which measure the levels of activity of a service or intervention (e.g. number of personalised care plans or levels of delayed transfer of care).

Why do we need whole-system outcomes?

Failure to plan and coordinate services with and around people’s needs leads to fragmentations in care and sub-optimal outcomes. Redesigning care around the delivery of outcomes requires a multidisciplinary approach across prevention, diagnosis, treatment and follow-up. This needs to be supported by an environment of continuous learning, improvement and innovation with ongoing and robust quality assessments.

Outcomes should achieve the following:

Link to improved experiences of care, as defined by people

Person-centred health and care should be anchored in what people want and need. In other words, people’s own narratives should be at the centre. The outcomes defined should reflect the philosophy, overall aims and mission of that narrative. The narrative for person-centred coordinated care and accompanying ‘I-statements’ developed by National Voices help define goals from a user-based perspective. See also the How to ... lead and manage better care.

Capture expected changes to people’s health and wellbeing

These can be direct care outcomes for people (such as improved health and wellbeing, ability to manage own chronic illness, independence, quality of life, ability to die in a place of their choosing) or indirect care measures – or proxy outcomes – such as reduced hospitalisations/nursing home placements and other avoidable utilisation of services.

Consider use of resources

Value for money is critical to the sustainability of health and care. As such, financial measures should be included alongside quality and user experience measures in an outcome framework, with a clear outline of when, how and what benefits will be realised.

What are whole-system outcomes?

Outcomes should include individual and whole-system definitions of what ‘good’ looks like. They should combine locally articulated goals with those set out in national frameworks and include measures related to quality of care, patient wellbeing and experience as well as financial sustainability. This is expanded upon in the diagram below.

 
View larger version

View larger version
 

Collaboration

Integrated, person-centred care can never be achieved by one agency working in isolation. Because of this, outcomes should be co-produced or agreed to by the range of stakeholders involved. This is easier said than done: partners across health and care often have different goals, lines of accountability, cultures and ways of working. It is important to identify and acknowledge differences and complexities, and work together to negotiate a set of measures which are anchored in a common purpose – to improve health and wellbeing for individuals and communities.

There is no doubt that the pace of work across traditionally separate systems to improve care experiences and outcomes across the country is accelerating. The task now is to develop ways of measuring the difference integrated care and support is making to people’s quality of life. This is far harder than measuring more traditional straight input or output measures. System leaders need to understand that they have the freedom to experiment, and be allowed the flexibility to try new approaches. These systems need to be developed locally, alongside those who use services, an approach far more relevant and owned than top-down approaches can ever be. If we can get this right, the prize is more and more people living the lives they want, supported by services that fully and properly reflect their ambitions and strengths.

Tony Hunter, Chief Executive, SCIE

Using a logic model

Logic modelling, or programme logic, is a tool to help establish outcomes and consider impact – i.e. articulating how a project or programme is intended to achieve its outcomes and impact. As a tool, logic modelling has been used widely to help shape interventions and related monitoring and evaluation activities, including new models of care in many vanguard areas.

A logic model is a diagrammatic representation of the key components of a programme and the way that actions are intended to lead to outcomes. It helps set out plans and thoughts about what inputs (resources and target groups) will deliver the interventions (projects or programmes), which in turn will produce a set of outputs, outcomes and impacts. This logic chain, and the links between its different elements can help programme leads and stakeholders test assumptions and ‘reality check’ whether the programme’s interventions are adequate to achieve its intended outcomes.

Logic modelling often takes place at an early stage of the process. However, it is also possible to use logic modelling at a later point to review the relevance and strength of existing programmes. Read more here about How logic models deliver care changes that work.

Logic model

Inputs

What resources are required for the intervention?

How many people will enter the intervention?

(Consider the inclusion / exclusion criteria)

Examples: Resources, people, technology, and locations.

Interventions

What activities are required to use the inputs and achieve the outputs?

What actions taken by providers to prevent or improve a social outcome, a medical disorder, a community/population health or a social situation?

Examples: New models of care, projects or programmes of activity.

Outputs

What are the expected ‘products’ of this process?

How many people have completed the intervention? (define completion)

Examples: Increase in activity such as percentage change of case reviews, increased capabilities or new capabilities such as greater data-sharing mechanisms.

Whole system outcomes

Outcomes are benefits that are delivered as a result of a service.

• Short-term outcomes
• Medium-term outcomes
• Long-term outcomes.

Supplemented by SMARTER measures (see Selecting the right measures)

• Measures
• Measures
• Measures

Impacts

What is expected to happen long after the intervention has finished?

Delivering person centred coordinated care through:

• Improving service user experience
• Achieving health and well being outcomes
• Using resources effectively

Five steps to build a strong logic model

• Be clear about the starting position. What is the target audience for the work, what is the baseline and context for the work?
• Establish the inputs required to deliver the work. What interventions are needed to achieve the outputs, and what will the outcome and impact be?
• Consider how inputs will resource the interventions that will deliver specified outputs, outcomes and impact.
• Consider what resources are available. Where possible, evidence any assumptions with reference to pilot initiatives or evaluations from similar work undertaken elsewhere.
• Engage with patients, service users and carers – and other stakeholders – to develop a robust logic model that reflects a shared vision of success.

Logic modelling in practice

Healthwatch has developed an outcomes and impact development tool to assist local organisations in understanding what outcomes and impacts they can achieve through the delivery of their functions. The table below shows the draft template and how the sections relate to the logic model presented above. It also shows how outcomes have been split into short-, medium- and long-term.

For more tips on how to use logic models, please see Midlands and Lancashire Commissioning Support Unit’s guide to using Logic Models. It provides key insights into the theory and principles, as well as practical use.

What do we want the OUTCOMES to be?

• Short-term outcome

  What could we do to achieve them? Inputs/ Interventions

  How would we know we have succeeded? Output

• Medium-term outcome

  What could we do to achieve them? Inputs/ Interventions

  How would we know we have succeeded? Output

• Long-term Impact

  How could we measure IMPACT?

  What could we do to achieve them? Measures

  How would we know we have succeeded? Output

To establish outcomes, remember the following:

• Build engagement through co-producing measures of quality across organisations.
• Articulate outcomes and measures that are relevant and meaningful for all, including: users; carers and communities; planners and commissioners; health and care providers; clinicians and care professionals; and national bodies.
• This is an evolutionary process. Measuring success is an ongoing process and not something that is ‘ticked off’ once.
• The power of citizens. For care to be coordinated around citizens, they need to be involved in the development of how that coordination takes place.

 
View larger version

View larger version
 

Engagement is not only a topic of academic interest; it has enormous practical significance. Put simply, organisations with more engaged clinicians and staff achieve better outcomes and experiences for the patients they serve.

The King’s Fund. Source: Leadership and engagement for improvement in the NHS, King’s Fund (2012)

In summary: top tips to establish whole-system outcomes

• Identify outcomes at an early stage of the process rather than retrospectively fitting them to a predetermined and funded set of activities or interventions.
• Identify stakeholders involved in delivering – or who are affected by – the programme of change. Engage with them to co-produce measures of success. Acknowledge that different partners have different needs, definitions of success, lines of accountability and traditions regarding information governance, data and ‘evidence’.
• Set out clearly the rationale for the programme. Map how investments (inputs) will be used (activity) to deliver results (outputs, outcomes and impact).
• Align local whole-system outcomes with national frameworks and other local outcomes (including those of individual providers and at commissioner level). Identify up front what is already collected in the system, what can be shared and in what form – across health, care and the voluntary sector. Existing lists of defined outcomes can be used as a ‘menu’ of possible options, but choices must be made locally.
• Clarify reporting arrangements, roles and accountabilities and agree a plan up front for how data is to be collected, analysed and used.