SCIE Report 4: Using systematic reviews to improve social care

By Geraldine Macdonald

Published November 2003

Context

A 'systematic review' is a systematic and rigorous review of all available evidence, designed to eliminate bias and assemble as complete as possible a picture of the knowledge available. SCIE conducts systematic reviews of topics in social care and uses the findings as the basis for its good practice guidance.

Purpose

This report examines the relevance of systematic reviews to SCIE's goal of promoting good practice. It is part of a larger programme of SCIE work looking at using knowledge in social care.

Audience

The report is primarily for the use of SCIE but will also be of interest to systematic reviewers generally.

Messages from the report

Some issues about systematic reviews

• Systematic reviews are not synonymous with meta-analysis.
• Systematic reviews should be international in scope.
• Systematic reviews should not be the sole basis for decision making.
• Systematic reviews need to be conducted by teams with an appropriate skills mix.
• Systematic reviews require adequate resourcing. It is possible to estimate the cost of a systematic review.
• A thorough editorial process greatly enhances the standard of a systematic review and this also requires adequate resourcing.
• Explicitness and transparency are the central features of a systematic review rather than any particular decision.
• A choice needs to be made about whether to undertake a series of related reviews in an area, or to conduct one or two reviews which seek to examine the effects of more than one intervention. The cautious approach would be to conduct a series of reviews.
• Systematic reviews need to be interpreted in the light of other sources of evidence and other factors influencing decision making, such as values, resources and priorities.
• Reviewers in social care face particular challenges in identifying all relevant studies. Information scientists are crucial to this endeavour.
• The preparation of systematic reviews highlights the need for more and better quality studies, that are better reported. This raises questions about the transferability of the findings.

A systematic review methodology suitable for social care

• SCIE's biggest potential methodological contribution lies in establishing sound methods for developing its practice guidelines.
• Systematic reviews should be an essential ingredient in the development of all practice guidelines, but need to be set in the context of other factors and other kinds of evidence.
• Guidelines are the most appropriate place for bringing together different kinds of evidence, and interpreting these in the context of policy priorities, resources and so on.
• There are some criteria which SCIE might consider in developing this area of its work.

Recommendations

• All those who conduct systematic reviews for SCIE should adhere to the underpinning principles of systematic reviews.
• SCIE should establish and maintain collaborative arrangements with other organisations engaged in systematic reviews. It can usefully influence the work of these organisations, particularly in terms of relevance.
• SCIE should be thorough in identifying all systematic reviews relevant to its work programme.
• SCIE should commission systematic reviews where none exist that are relevant or adequate to its work.
• SCIE should ensure that all its commissioned reviews are carried out systematically.
• SCIE should require those preparing a systematic review on its behalf to first publish a protocol. This should then be carefully and independently scrutinised by people with relevant expertise.
• SCIE might consider whether to encourage those from whom it commissions systematic reviews to register their review with either the Campbell or Cochrane collaborations. This would provide technical support to reviewers, and additional procedures for quality control.
• Relevant systematic reviews should be at the heart of practice guidelines, even if they demonstrate that little is known about a particular issue.
• When SCIE commissions broader pieces of work, it should be transparent about how conclusions and recommendations are reached.
• SCIE should ensure that its work programme includes the updating of both commissioned reviews and practice guidelines.

Related links

SCIE's resources on using knowledge in social care