The use of Cochrane Reviews in NICE clinical guidelines

Dirk Wiersma/Science Photo Library The National Institute for Health and Clinical Excellence (NICE) produces clinical guidelines for the National Health Service (NHS) in England and Wales. These guidelines are developed by groups that combine people with expertise in conducting systematic reviews and health economic analyses, with those with expertise in the clinical area (from healthcare professionals and patients). During guideline development, the group poses about 20 to 25 clinical questions, which are then addressed by systematic reviews. As described in the NICE Guidelines Manual,[1] groups routinely search for existing relevant systematic reviews. The NHS funds NICE and makes substantial contributions to The Cochrane Collaboration, so there is a common interest in making the best use of Cochrane Reviews for informing guidelines. One way of assessing the extent to which systematic reviews are used in guidelines is to look at the number of citations of reviews in each guideline. In a recent piece of work, we aimed to find out the extent of citation of Cochrane Reviews in NICE guidelines. For all NICE guidelines published before the end of February 2011 we used full‐text searching of the full version of the guideline (and appendices) to identify the word ‘Cochrane’. Where the occurrence of ‘Cochrane’ was a citation of a Cochrane Review, we looked for the current version of the Cochrane Review (or the record of a withdrawn Cochrane Review) in the Cochrane Database of Systematic Reviews (CDSR); we then identified and assigned the name of the Cochrane Review Group (CRG) developing the Cochrane Review to each citation. We tried to examine the context of the use of the Cochrane Review, and decide whether it was used as a source of evidence or as a background document (for example, a review describing how an intervention has been used for a completely different condition). By the end of February 2011 NICE had published 116 clinical guidelines. Ten of these had been replaced by a more up‐to‐date version, leaving 106 guidelines in the sample. There were 731 citations of Cochrane Reviews in the 106 guidelines, ranging from no citations to 44 citations, with a mean of 6.90 (standard deviation 9.23). Some Cochrane Reviews were cited more than once in different guidelines; therefore, the figures do not represent the number of Cochrane Reviews cited. There were 23 citations that we judged to be for background use rather than directly used to address guideline questions, but we have included these in the data below, as it was difficult at times to be sure of this coding. Table 1 shows the frequency of citations in guidelines. Open in table viewerTable 1. Frequency distribution of Cochrane Review citations in NICE clinical guidelines Of the 52 currently registered CRGs, 46 had Cochrane Reviews cited. The maximum number of citations for any CRG was 192. Summary frequencies of citations for the CRGs are shown in Table 2. Open in table viewerTable 2. Frequency distribution of Cochrane Review citations by CRG The coding of Cochrane Reviews by CRG used the CRG listed in the latest version of the CDSR; if the Cochrane Review cited in the guideline was not in the latest version, we used the Cochrane Review with the most similar title to the citation. The citations in guidelines may be several years old, authors and titles of Cochrane Reviews change, and some Cochrane Reviews are allocated to new CRGs. Therefore, there may be some errors in these data, but it is likely these are only a small percentage. The data show that some CRGs do not have any Cochrane Reviews cited in NICE guidelines, which may be because they do not map clearly on to the topics covered to date in NICE clinical guidelines. More recently established CRGs would also have had less chance to both develop Cochrane Reviews and have them cited. Even with these limitations, the data show the extensive use of Cochrane Reviews in the development of NICE clinical guidelines. Preliminary work in analysing the types of questions posed in NICE guidelines suggests that around half of the questions concern interventions of the type addressed by most Cochrane Reviews, with a mean of about 14 intervention questions per guideline. Although the data show an impressive level of use of Cochrane Reviews in NICE clinical guidelines, there is no doubt that we could do more to make efficient use of Cochrane Reviews, and work together for mutual benefit in a number of ways. We could make better use of the knowledge contained in CRGs when we draw up the scope for guidelines, and encourage more involvement from Cochrane Review authors on NICE guideline development groups. We could do more to ensure that Cochrane Reviews and guideline questions are better aligned, work harder at sharing knowledge from Cochrane Reviews and guidelines in development, and try to speed up the editorial process of turning relevant Cochrane Protocols into Cochrane Reviews, or updates of Cochrane Reviews, so that they can be considered for a clinical guideline. As guideline developers, we need to ensure that appropriate credit is given when Cochrane Reviews are used, and we could be more active in giving constructive feedback to Cochrane Review authors. It is very difficult to convert non‐Cochrane Reviews into Cochrane Reviews, but perhaps we can work harder to make our different timelines and processes work together to reduce duplication of effort. Updating is a common challenge, and there must be some scope for sharing intelligence. This idea of working together is not new, and there have been a number of initiatives around particular clinical guidelines to make better use of Cochrane Reviews.[2, 3] Within England and Wales, there is an increasing appetite to try to make this relationship work better for mutual benefit. In September 2011, there will be a workshop between the clinical guidelines team at NICE and several people with various roles in The Cochrane Collaboration. Let's make this the start of a constructive dialogue to make sure all our efforts better inform clinical practice.