Evaluate the impact and implementation of care and health practice and policy
SCIE’s evaluation services help care and health commissioners, providers and policy makers to demonstrate how they are implementing policy and the impact it has on practice and outcomes.
Increased local accountability, growing numbers of self-funders, reduced budgets and high levels of public scrutiny mean resource allocation has never been more challenging. Robust evaluation can help care providers and commissioners to test the impact and cost-effectiveness of their activities and support decision making.
We very much welcome the honesty, clarity and accuracy of this evaluation of our Care Act implementation work. I would encourage other authorities to consider this type of approach to getting an informed external view of their service areas and believe SCIE provided this in an objective and professional way.Alan Lotinga, Service Director, Health and Wellbeing, Birmingham City Council
You can commission SCIE to provide:
- large-scale evaluations of local and national initiatives
- experimental, longitudinal (change over time) evaluations including trial approaches, such as quasi-experimental and randomised control trials
- formative and summative evaluation incorporating a realist approach to our work, we work in partnership with clients to ensure findings are used to inform and drive service development and improvement
- cooperative action research to develop our clients’ sustainable capacity to conduct their own evaluation
- multi-method evaluations incorporating qualitative and quantitative approaches, such as interviews, focus groups, literature reviews, document reviews, observation and survey work
- commissioning and managing independent evaluation contractors for clients. As well as directly conducting evaluations for our clients, we also provide a commissioning and evaluation management service to clients, which includes developing invitations to tenders (ITTs), assessing ITT responses, selecting contractors and drawing up and managing the resulting evaluation contracts.
- evaluation advice, scoping and design. We can support you to develop your own inhouse evaluation services.
- translation of evaluation findings into practical implementation tools and resources (see our Digital services offer)
- dissemination of evaluation findings across the care and health sectors (see our Communications services)
We provide evaluation services to a wide range of clients including local authorities, care providers, national agencies, government departments, foundation trusts, research agencies and universities. Some of our evaluation clients are outlined below.
Birmingham City Council - Care Act Programme Review – November 2015 Open
Following the initial implementation of the Care Act 2014, Birmingham City Council commissioned SCIE to undertake an external review of the approach taken by the Council in order to be Care Act compliant.
The aims of the review were to recommend the next steps required in order to meet the Care Act requirements and embed its principles around five key areas: assessment and eligibility determination, custodial settings, independent advocacy, personal budgets and transition from children’s to adults’ services.
The review identified whether the Council had put in place the appropriate tools to meet the Care Act duties and to enable staff to embed the Care Act principles in due time.
SCIE’s review, which was carried out over 6 weeks (spending four days onsite), involved:
- 10 interviews with a total of 25 people including Care Act Leads, key players in implementing the changes and operational managers
- 5 focus group attended by more than 35 front line staff
- reviewing almost 60 documents including forms, protocols, procedures, guidance, training material and leaflets.
The final report, which was provided as an action plan, identified:
- findings – strengths and areas of consideration
- traffic light analysis against Care Act 2014 and Regulation duties.
An overview of the findings and recommendations was presented to a forum of 50 people including the Adult Social Care Lead member, the Care Act Regional Lead, the Service Director for Business Change, the Service Director Health and Wellbeing (who commissioned the review) and two assistant directors with the remit for Adult Social Care.
The review is being used by the Council to identify priorities and next for the next six months in order to ensure Care Act compliance and improving outcomes for local people and staff in BCC.
We very much welcome the honesty, clarity and accuracy of this evaluation. The level of support and guidance given by SCIE throughout has been much appreciated. The process has been well managed and co-ordinated; and delivered in an expert manner. The evaluation was flexible, thorough and was undertaken with sensitivity to local pressures and circumstances. Recommendations made have proved to be a good basis for making improvements across our service. I would encourage other authorities to consider this type of approach to getting an informed external view of their service areas and believe SCIE provided this in an objective and professional wayAlan Lotinga, Service Director, Health and Wellbeing, Birmingham City Council
Department of Health, Government Equalities Office and the Department for Work and Pensions – Supporting carers in employment pilot Open
In April 2015, a consortium of Government departments commissioned SCIE to manage the delivery and evaluation of a two-year pilot programme, which aims to test what works to support carers to remain in and/or return to employment.
The Carers in Employment (CIE) programme explores the different ways local authorities can meet this challenge. Pilots are funded to explore the different ways this challenge can be addressed. To date, pilot activity falls into three broad areas: engagement and awareness with employers, information and advice to carers and assistive technology.
SCIE’s role involves managing the independent evaluation of CIE by the Institute of Employment Studies (IES). The aim of the evaluation is to find out what works best to support carers and to share learning as widely as possible. Interim report findings are due in September 2016, with the final evaluation of the pilots in July 2017.
SCIE is also coordinating and supporting the delivery of pilots which involves regular network learning events and the use of a remote learning hub and discussion forum.
Department of Health - Evaluation of Integrated Personal Commissioning (IPC) Open
The Department of Health has commissioned a summative evaluation of the Integrated Personal Commissioning (IPC) programme. The evaluation will be carried out by a consortium led by SQW, in partnership with Bryson Purdon Social Research (BPSR), Social Care Institute for Excellence (SCIE), the Social Policy Research Unit (SPRU) and the Centre for Health Economics (CHE) both at the University of York, and Mott MacDonald
Integrated Personal Commissioning (IPC) was launched in April 2015. It is a partnership between NHS England and the Local Government Association.
IPC is an approach to joining up health and social care, and other services where appropriate. The purpose is to enable people, with help from carers and families, to combine the resources available to them in order to control their care. This is achieved through personalised care planning and personal budgets. IPC also aims to support people to develop the skills and confidence needed to self-manage their care in partnership with carers, the voluntary, community and social enterprise (VCSE) sector, community capacity and peer support.
The programme is being tested out in 18 demonstrator areas in England where Local authorities and CCGs are working together with their providers and voluntary sector partners. The programme is planned to run until March 2018. It is focussed on four particular user groups:
- Children and young people with complex needs, including those eligible for education, health and care plans
- People with multiple long-term conditions, particularly older people with frailty
- People with learning disabilities with high support needs, including those who are in institutional settings or at risk of being placed in these settings
- People with significant mental health needs, such as those eligible for the Care Programme Approach (CPA), or those who use high levels of unplanned care.
The evaluation aims to identify key lessons about the effectiveness of IPC schemes, in particular how far improved outcomes occurred and what were the key drivers of any change. It will also consider how far the changes that do occur represent value for money. The evaluation will deploy a mixed-methods approach, including a light-touch process evaluation, impact and economic analysis and thematic case studies. Data will be collected from areas in collaboration with the local teams in each site.
SCIE’s role in the evaluation is to facilitate co-production with people who use services and carers in the process. The main element of this will be a co-production panel to help shape all parts of the evaluation from the initial scoping to assessing the results. SCIE will also be contributing to fieldwork research.
Further details about the evaluation are available from Graham Thom, SQW Managing Director.
Department of Health – Named Social Worker pilot project Open
The Department of Health commissioned the Innovation Unit in partnership with SCIE to deliver the Named Social Worker pilot project in six sites as part of efforts to strengthen the rights of service users and their families.
The sites will have the opportunity to develop the practice, devise how impact is measured and refine a NSW service model. Each site will be responsible for developing their plan, with assistance and coaching from the Innovation Unit and SCIE team. The six pilot sites are:
The pilot will assign a named social worker to adults with learning disabilities, autism or mental health conditions. The social worker will be the main contact for service users and their families, regardless of the setting they are being supported in, and will be able to challenge decisions about their care.
The project will be delivered in three phases and the findings from each phase will be launched in three separate reports made available to the sector. The project will conclude with the third and final report being launched in April 2017.
Department of Health (Northern Ireland) – evaluation of social care workforce regulation (September 2016) Open
The Department of Health (Northern Ireland) has commissioned SCIE to support their evaluation of the impact of social care workforce registration and the implementation of codes of conduct and practice standards.
With SCIE’s expert input, the Department of Health (Northern Ireland) has established the framework for the evaluation, including identifying benchmarks against which to measure impact, formative evaluation of the processes and longer term summative evaluation of the impact.
Since 2005, all professionally qualified social workers have had to register with the Northern Ireland Social Care Council (NISCC) in order to hold a social work post. Since 2010 Northern Ireland has been rolling out compulsory registration to the social care workforce on a phased basis and will be the first country in the UK to regulate unqualified social care workers when roll out to the final group of social care workers is complete in 2017. The Department of Health (Northern Ireland) is keen to learn about the impact of regulation on key stakeholders, including staff, service users and employers, through evaluation from the experience of the regulation of the social work and social care workforce.
Supported by experts from SCIE, the aim is to conduct a mixed-methods evaluation of the impact of workforce registration and the implementation of codes of conduct and practice standards. The primary objectives for this study are to examine how, and to what extent, registration and practice standards have:
- strengthened public protection and safety
- improved the quality of service provision
- excluded unsuitable persons from the workforce
- promoted the value of social care within an integrated system in Northern Ireland.
SCIE has been of great assistance in helping us develop an evaluation framework that utilises existing opportunities and resources that are available within the health and social care system already making this work both viable and affordable at a time of financial constraint. The design of the framework supports identifying benchmarks against which to measure impact, formative evaluation of the processes and longer term summative evaluation of the impact. The formative evaluation allows for learning to be identified and for improvements to be made in real time as opposed to waiting for a final summative evaluation. We believe we have a robust framework to evaluate the impact of registration of the social care workforce in Northern IrelandChristine Smyth, Deputy Chief Social Work Officer, Department of Health (Northern Ireland)
Dunhill Medical Trust and The Mid and East Antrim Agewell Partnership – Evaluation of Involving many to prescribe alternative care together pilot Open
With funding from the Dunhill Medical Trust (the Trust), SCIE and York Consulting (YCL) are currently supporting The Mid and East Antrim Agewell Partnership’s (MEAAP) action research evaluation of their Involving Many to Prescribe Alternative Care Together (IMPACTAgewell) pilot, which is also funded by the Trust.
Starting in July 2017 and ending in March 2020, SCIE and YCL are supporting MEAAP to undertake a three-year evaluation of their IMPACTAgewell pilot by providing:
- independent and expert quality assurance all aspects of MEAAP’s evaluation, from evaluation design, through fieldwork and data collection and onto analysis, reporting and dissemination
- expert input to the development of all evaluation instruments (including logic modelling, evaluation strategy and plan, a management information system, fieldwork protocols, surveys and discussion guides)
- training and support to develop evaluation capacity and capability locally.
- support to local coproduction, ensuring every aspect of the evaluation is developed with input from IMPACTAgewell service users and partners.
Employing an asset-based approach and with coproduction at its core, MEAAP’s IMPACTAgewell pilot uses a ‘social prescribing’ approach. There are six local hubs (three urban and three rural) which include and centre on GP practices, community pharmacists and staff from the health and social care sectors, to offer person centred care and community-based support. Over a three-year period, IMPACTAgewell, will be delivered to 1,100 older people aged 70 years and over, via up to 13,200 funded ‘alternative care prescriptions’. The older people will be supported by a key worker who will help provide them with person-centred services and support to improve and meet the health outcomes that matter most to them.
The primary objectives for MEAAP’s evaluation are to examine how and to what extent IMPACTAgewell:
- establishes new and develops further existing partnerships
- is successful in delivering tailored holistic health and social care services to older people
- is successful in adopting and sustaining a coproduced approach to all activity
- demonstrates and increases the sustainability of community based health initiatives
- provides a cost effective contribution to community health and social care services.
The evaluation is based on MEAAP’s and SCIE’s co-production and engagement model of working; hence engagement, involvement, enablement and sustainability are fundamental in this evaluation.
HC-One – evaluation of development of nursing assistant roles (October 2015 – January 2016) Open
Care home provider, HC-One, commissioned SCIE to conduct an independent evaluation of their development programme for their senior carers to become nursing assistants.
As part of a strategic transformation of its services, HC-One developed the new role of nursing assistant in order to retain high-performing staff and improve the quality of care. The nursing assistant is a new intermediate role that sits between the senior carer and the qualified nurse. It provides a new opportunity for senior carers to grow their skills, responsibility and pay, while reducing the pressure on nursing staff.
HC-One provides an assessment and training package to support applicants in making the transition to becoming nursing assistants – called the Care Assistant Development Programme.
The aim of SCIE’s review was to:
- evaluate the short-term impact of the Care Assistant Development Programme (CAPD) on the quality and consistency of care
- inform the development, implementation and ongoing evaluation of the Programme
- validate the programme against national good practice markers for the delivery of safe and good care.
Our methodology included qualitative and quantitative research, action research with selected sites and stakeholders, and reviews of key resources. It involved:
- Scoping interviews with stakeholders
- 60 nursing assistants took part in before and after survey
- 42 homes took part in survey of residents and relatives
- 6 qualitative case studies with homes
- 50 interviews with home managers, deputy managers, nursing assistants, senior carers, carers, residents and relatives
- 2 interviews with HC-One stakeholders
- 3 interviews with external stakeholders
- 16 homes took part in analysis of Key Performance Indicators
- Review of key documents
The findings were launched at a care sector seminar, hosted by SCIE, on 13 June 2016.
SCIE also produced two short films about the CAPD from the perspective of a HC-One nursing assistant and a care home resident.
- Read the report and view videos
- Read SCIE press release
- Blog from HC-One’s Alison Innes-Furguhar
- SCIE’s evaluation service
The Care Assistant Development Programme is an enterprising scheme to release the ambition and increase the skills of kind, caring staff. SCIE grasped the potential of this immediately and developed their evaluation in a dynamic and constructive way. This really is promoting social care excellence and I commend their capacity for active, critical friend evaluationsJohn Ransford, Non-Executive Director at HC-One
Lloyds Bank Foundation Trust and the Big Lottery Fund – evaluation of Homeshare Open
SCIE managed the evaluation of a two-year pilot of the UK’s first national Homeshare programme. The evaluation, managed by SCIE and undertaken by the Traverse, focused on testing and finding out what works to make the Homeshare model of care and accommodation a sustainable and economically viable business. Homesharing is typically where an older person shares a home with a younger person, who in turn needs affordable rent and who can offer support and companionship.
Macmillan Cancer Support - Evaluation of the Local Authority Partnerships Programme Open
Leading national cancer charity - Macmillan Cancer Support - has commissioned SQW and the Social Care Institute for Excellence (SCIE) to evaluate their Local Authority Partnerships (LAP) Programme.
The Programme provides a vision for delivering integrated care and support which builds on the key policies affecting health and social care in the UK. It focuses on developing new models of care and support for people affected by cancer which are personalised, coordinated and promote wellbeing and independence.
SQW and SCIE have been commissioned to undertake a formative evaluation of the LAP Programme, generating context specific insights and learning to inform future roll-out and scalability of the approach. This will involve the design and development of a programme level theory of change model, an evaluation framework, and providing onsite support and engagement to develop and support local monitoring and evaluation processes. The evaluation will run from January 2016 to December 2019.
The work aims to build capability amongst local stakeholders in undertaking evaluation and monitoring activities, whilst sharing emerging findings with national and programme stakeholders. The evaluation will involve a mixed methods approach, including secondary desk-based analysis as well as primary data collection (including interviews, focus groups and observations). The work will generate both formative learning and a robust baseline for future evaluation.
The plan is to publish two interim reports, a final report and an engaging toolkit, to enable others to learn from the programme and to inform its wider roll-out.Contacts
For more information about the programme, please contact:
Fiona Smith, Social Care Projects Manager and Interim Commissioning Support Programme Lead, Macmillan Cancer Support. Email: firstname.lastname@example.org
For more information about the evaluation, please contact:
Lauren Roberts, Director, SQW. Email: email@example.com
Ewan King, Director of Business Development and Delivery, SCIE. Email: firstname.lastname@example.org
NWG - Child sexual exploitation - evaluation support Open
NWG Network (formerly The National Working Group for Sexually Exploited Children and Young People) - Evaluation Support
The NWG Network has commissioned SCIE to evaluate the network’s performance and impact. This includes benchmarking it against similar activities and programmes. The network supports professionals working on the issue of child sexual exploitation (CSE) and trafficking within the UK. The NWG is a charitable organisation with over 12,000 practitioners who disseminate information down through their services.
The network covers voluntary and statutory services and private companies working in this field. They provide support, advice and raise the profile, provide updates, training, share national developments, influence the development of national and local policy informed by practice.
SCIEs evaluation will help and support the NWG Network to:
- Review performance and outcomes data
- Produce a baseline picture of performance
- Review existing evaluation and monitoring systems, such as the survey tools and feedback forms.
This is a crucial time for the NWG Network as we build our reach and impact. We are delighted to have SCIE helping us analyse our performance and assess our impact in the future.Sheila Taylor MBE, Chief Executive, NWG Network.
The Reading Agency - Evaluation of Reading Friends Open
The Reading Agency has commissioned social enterprise Renaisi and SCIE to work with them on the evaluation of Reading Friends – a new UK-wide befriending project, funded by The Big Lottery, to empower, engage and connect vulnerable and isolated older people, people with dementia and carers through social reading activities.
The evaluation is based on a partnership and learning relationship with the project. Until mid-2018, Reading Friends will be tested in six sites, with a pilot stage following the year after in ten sites, rolling out nationally in 2019.
The aims of the evaluation are to:
- understand the impact of Reading Friends on older people who are taking part in the programme
- understand which approaches work in different settings and with different primary audiences
- make recommendations to improve Reading Friends in the future and help build a sustainable delivery framework .
The focus of the evaluation is on the impact on the vulnerable and isolated older people who are supported by the programme. It will also explore the impact on community volunteers who will be using reading to befriend and connect older people, and the partner organisations and settings where activities are taking place.
SCIE’s role in the project is to:
- help co-produce the evaluation with older people
- ensure methods are appropriate and ethical for working with vulnerable people (including those with dementia) by working alongside Renaisi’s researchers
- provide access to leading national health and social care organisations to ensure that the evaluation is informed by an understanding of what influences health and social care funders and decision makers.