Systematic reviews and Rapid Evidence Assessments (REAs) can help policy-makers draw out data from a number of studies to produce much stronger evidence than by just examining one study. They can also reveal where evidence is not available and primary research is needed.

A systematic review is a valuable tool which has been developed to provide a broad analysis of the research evidence. It is “a review in which bias has been reduced by the systematic identification, appraisal, synthesis, and, if relevant, statistical aggregation of all relevant studies on a specific topic according to a predetermined and explicit method.” (Moher et al Lancet 1999)

A good systematic review:

• Addresses a clearly defined question;
• Uses transparent, reproducible methods to find relevant studies, to determine the risk of bias and to extract information;
• Summarises the totality of evidence available;
• Is updated regularly as new evidence arises.

A REA is a quicker version of a systematic review, with some sacrifice of rigour to be able to get answers more quickly and within typical policy timeframes. Systematic reviews and REAs originated in the medical field led by the Cochrane Collaboration, a response to how much medical practice was happening without evidence that the treatment worked. The Cochrane Collaboration is an international, non-profit organisation that aims to help people make well-informed decisions about healthcare by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of healthcare interventions (published monthly in the Cochrane Library).
It was established in Oxford in 1993 at a meeting attended by 90 people and now has over 22 000 contributors in 91 countries.

The Campbell Collaboration is a similar organisation which coordinates systematic reviews in the social science field. It has undertaken systematic reviews of issues such as the effects of interventions in education, criminal justice and social welfare.

Systematic reviews and REAs both involve reviewing all the evidence on a particular research question (whether electronic, published or grey literature), critically analysing the research for relevance and quality, and then collating the findings from the range of evidence. They seek to take out the bias and ensure that evidence, where available, is as credible as possible. Systematic reviews can take 6-24 months to complete and are the “Rolls Royce” of evidence collation, but with the time constraints facing policy-makers, the shorter REA method can be considered for policy-making.
The UK government referred to a REA as “a tool for getting on top of the available research evidence on a policy issue, as comprehensively as possible, within the constraints of a given timetable”, and used it to look at the effectiveness of issues such as neighbourhood watch, hot spots policing.

REAs typically take 3-6 months and therefore are quicker and require fewer resources in terms of human resources and funding. Due to the rapid nature of this method, the grey literature might not always be included and preference may be given to the more readily available research which has been published and written in English. This means the rigour and comprehensiveness is reduced, and therefore, where possible, a REA should be followed by a full systematic review. Nevertheless, the REA is likely to pick up 80% of the story and enable policy-makers to proceed more quickly with the evidence they have found.

The PSPPD has commissioned an example of a REA on Why crime is so violent in South Africa, which will be available shortly, and funded training of researchers and policy-makers in REAs in November 2010 and March 2011. Additional training will be run in July 2011. The Programme has also collaborated with the University of the Free State and the Campbell Collaboration to run training in systematic reviews in August 2010 and will run further training in August 2011.

Useful links on systematic reviews:

Campbell Collaboration
Centre for Reviews and Dissemination, University of York
Cochrane Collaboration
King’s College Evidence Network
The Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre)
Case study: Rapid Evidence Assessment of the economic and social consequences of worsening housing affordability
Case study: A Rapid Evidence Assessment of the impact of mentoring on re-offending: a summary
Overview: HLWIKI Canada: Rapid Evidence Assessments
Toolkit: Government Social Research (GSR) Unit Rapid Evidence Assessment Toolkit