Rahul P. Bhamkar.
Department of Pediatrics, MGM Medical College, Navi Mumbai.
ADDRESS FOR CORRESPONDENCE
Dr. Rahul P. Bhamkar, Flat No 9, Plot No 19, Nest Building,
Sector 1, Kalamboli, Navi Mumbai. Maharashtra. India. | Introduction
Cochrane collaboration is an international non-profit organization established in 1993 with an aim to put clinical research into practice and revolutionize a new era of evidence-based medicine. Its objective is to help people make well-informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions. The collaboration is named after the epidemiologist Archie Cochrane (1909-1988), a British medical researcher who contributed greatly to the development of epidemiology as a science.
Since its establishment Cochrane collaboration has rapidly evolved as an international organization. At present, it consists of fifty-one Cochrane Review Group (CRG), which are responsible for preparing and maintaining Cochrane reviews. A CRG is a group of researchers, health care professionals, and consumers who share a common interest in generating reliable up-to-date evidence relevant to prevention, treatment and rehabilitation of particular health care problems. Each CRG work on separate field of medical science and maintains reviews relevant to that field. Cochrane work is carried out by 14 centres, which are spread all over the world.
What are systematic reviews?
Cochrane reviews follow a significantly different methodology from the reviews those are published in print journals. Preparation of Cochrane reviews follows a systematic methods and standard protocol to efficiently integrate valid information to limit bias, reduce chance effects, and provide more reliable results upon which to draw conclusions and make effective decisions. It also tells about the fields for which further research will be required to answer the concerned question. Unlike reviews in print journals, which reviewers submit to the journals in the end, Cochrane reviewers initially register a title and a protocol with a CRG. The CRG will then supervise and guide the preparation of the review from the beginning itself to increase its authenticity and reliability.
What is a meta-analysis?
The primary studies included in the Cochrane reviews select and collect data from individual patients and analyze the results while Cochrane reviews analyze the primary studies. Many of the Cochrane reviews in the end perform a meta-analysis i.e. statistical combination of the results from two or more separate studies to improve the precision of estimate of the final outcome. The results of the meta-analysis are then plotted in a forest plot. However, not all systematic reviews can perform a meta-analysis. There are times when it is inappropriate or impossible to perform a meta-analysis. A meta-analysis simply does not add together results of different studies, but it calculates a weighted average after looking at results in each study.
Getting involved
The task of preparation of systematic review is cumbersome and requires commitment on the part of reviewers. All the CRGs are overburdened with work and are always in need of manpower. A person can devote to Cochrane collaboration by working as an advisor, a hand searcher, a reviewer or a fund provider. Review preparation is difficult to be performed by a single person so it is suggested that two or more person should come together to share the work and to reduce the bias by one reviewer.
Title registration
Preparation of Cochrane review starts with a title registration. A reviewer has to look for his topic of interest, contact the relevant CRG and register a title with them. The list of various CRGs is available on Cochrane website. (www.cochrane.org) Cochrane review titles have fixed structure - Intervention for a condition. It should make clear the interventions to be reviewed and the problem at which intervention is directed e.g. "Antenatal steroids for respiratory distress syndrome."
Registering a protocol
After a title has been accepted by a CRG, the review authors submit a protocol for consideration at the editorial base. Once the editors have finalized a protocol, it is included in CRG's module for publication and dissemination in Cochrane Database of Systematic Review (CDSR). The CRG gives a time limit to complete the review. Commitment is required on the part of reviewers to follow the deadline. A CRG provides every possible help and guidance to reviewers at different stages of preparation of review.
Cochrane Library
Cochrane library is an output way of Cochrane collaboration work. It consist of:
- Cochrane Database of Systematic Review: (CDSR):
It consists of list of all the reviews prepared, published, maintained and updated by Cochrane collaboration. The reviewers update a review at least once in a two-year to add up the latest information. Currently around 300 reviews are added in CDSR every year.
- Cochrane Central Register of Controlled Trials:
The idea behind this register is that it should be central place to put all reports of controlled trials identified as a result of searching MEDLINE, EMBASE, other databases, various journals, books and conference proceeding.
- Cochrane Collaborative Review Group Specialized Register:
Each CRG is building up a register of studies relevant to its scope so that a reviewer instead of searching lots of databases and journals for clinical trials will search these sources as their first place. Completeness of these registers varies with each group.
Study Search
Systematic reviews generally focus on randomized controlled trial so that results will be more reliable. Search for such trials from electronic databases is easiest and least time consuming. Though CENTRAL and CRG specialized registers are the most comprehensive sources, reviewers should make thorough search of MEDLINE, EMBASE and other databases so that no study that is important for the review is left behind. Hand searching is a manual page-to-page examination of the entire content of journal issues to identify eligible trials. Some studies are never published. Such unpublished studies should also be traced from the authors of published studies. Details of the search strategy have to be documented in the review.
Selection of Study
After searching the databases, reviewers decide which studies they want to include and which to exclude from their review. Most people use some eligibility forms so that the judgment goes in the right way. While deciding the inclusion criterion reviewers should concentrate on the design of the study and not on the results. Selection of the study is a multistage process. Inclusion and exclusion criterion and list of included and excluded studies have to be clearly documented in the review.
Getting into RevMan
Review manager software (RevMan) is the Cochrane Collaboration's programme for preparing and maintaining Cochrane Reviews. The software is updated from time to time by the collaboration. It allows the user to enter the protocol as well as a full review.
The main window of the RevMan is the tree view. The tree view is the structured representation of a review. It can be used to see the entire review or access a particular portion of a review for editing and viewing. It also performs a meta-analysis of the data entered and represents the results graphically. A reviewer who is going to prepare Cochrane review should learn how to use the RevMan effectively, the details of which are available in RevMan User Guide.
Assessing quality of studies
The section Methodological quality of included studies summarizes the quality of included studies and any flaws in them. Review result follows the principal of "'Garbage in, garbage out"'. Inclusion of poor quality studies in the review results in unreliable and biased results. So the included studies should be of high quality. The quality or validity of the study means how well the reviewer thinks that the study has measured "'the true effect"'. But since no one knows what the true effect is, there is an amount of guesswork and judgment involved in deciding which factors are most likely to affect the results of the study. Therefore, this is a hotly debated topic in Cochrane Collaboration. Some people use quality assessment scales to get overall quality scores. But many scales contain items, which cannot be applied to a study and has nothing to do with the validity. So in all there is no "'Gold Standard"' for quality assessment. A CRG may have a policy in quality assessment of the studies, so it is worth to check with them.
Planning the analysis
A reviewer has to tell readers, in the protocol itself, what he is planning to do after collecting all the study results e.g. whether he is going to compare the intervention with control or whether the intervention offers benefit over some other intervention considering the outcome. Accordingly, all possible comparisons have to be entered in the RevMan.
Analyzing the data
For analyzing the data we need to have some knowledge of the basic statistics. There are several different types of data we come across in the included studies. Dichotomous data are the data from outcomes that can be divided into two categories e.g. no of dead or alive. On the other hand continuous data are outcomes measured on a continuous scale e.g. blood pressure etc. For ease of analysis we often have to convert continuous data into dichotomous data e.g. converting blood pressure into "'high"' and "'not high"' blood pressure. The data can then be used to compare two groups in the form of following effect measures:
- Risk Ratio: Risk Ratio is simply the risk of event in one group compared to other group.
RR = no with event in treatment group/ no with event in control group
-------------------------------------- -------------------------------
No in treatment group no in control group
- Odds Ratio: Odds ratio compare how likely the events are between two groups.
OR = no with event in treatment group / No with event in control group
----------------------------------------- ----------------------------
No without event in treatment group No without event in control group
- Risk Difference: Risk in the experimental group--risk in the control group.
- Number needed to treat (NNT): Number of patients needed to treat to prevent a single event. NNT=1 / risk difference.
A reviewer has to decide which effect measure to use to express the results of the study in his review. The RevMan will then perform the meta-analysis using the measure he used. NNT is useful for presenting the results and not for analysis while the rest can be used for either.
Combining the Studies
As mentioned earlier a meta-analysis is performed by not just adding up the study but it is the weighted average of all studies in which some of the studies make a greater contribution to the total than others. Sample size and event rate are main factors determining weight attributed to a study. Variance of the study takes into account both of these factors. Variance is the standard error squared while standard error is the extent to which an estimate might be wrong due to random error. So, variance is inversely proportional to weight of the study i.e. less the variance, more weight is attributed to the study. RevMan 4.2 calculates weight of the study directly based on this assumption by inverse variance method. Other methods used for meta-analysis are Mantel-Haenszel method, the Peto method and DerSimonian Laird method. The first two are used when data are sparse and when it is assumed that every study is evaluating a common treatment i.e. fixed effect while the later is used when no fixed effect exits.
Diversity and Heterogeneity
Why all of the systematic reviews cannot perform meta-analysis in the end; the answer will be explained here. Any kind of variability in the studies is term as heterogeneity. Variability in participants, interventions and outcomes is described as clinical heterogeneity; variability in trial design and quality as methodological heterogeneity and variability in treatment effects evaluated is termed as statistical heterogeneity. If we suspect that an important heterogeneity is present in the review, a meta-analysis should not be performed because an unwise meta-analysis leads to highly misleading conclusions.
Presenting and applying results
The forest plots and summary statistics generated by RevMan are the results of the review which are discussed in the terms of how effective an intervention may be and how the results of the review may be applied to individual health care situations. Any limitations of the review and the assumptions made, the strength of the evidence i.e. how strong the overall case for use or cessation of use of the intervention and the relevance or the applicability of the evidence to the different settings are also discussed. Its better to present the results in a way that will allow user to make a balanced objective "'trade-off"' given their personal circumstances.
Conclusions
It is the main part of the review in which users are actually interested in!
Here, reviewers put their own conclusion about whether they think the intervention actually works or not and whether it can be introduced in the clinical practice. Reviewers also have to comment in the end what are the implications for further research and in which direction it should be carried out.
Maintaining the review
The main aim of Cochrane reviews is to give best available and up-to-date information to the users to bring improvement in health care. Evidence on a subject is a dynamic thing and systematic reviews run the risk of becoming out dated and even misleading. Therefore, a review has to be updated at least every two year or a commentary to be added to explain why update has not been done. New trials identified should be assessed for inclusion in the review in the same way as original review.
Day by day, the importance of Cochrane Reviews is increasing. Clinical practice guidelines are often based on Cochrane reviews and funding agencies internationally are requiring them to justify need of new clinical trials. So the numbers of reviews have rapidly increased over the last few years.
But finally, no matter how well the review has been done, it can never make a health care decision; it can only assist in the process by providing the best available evidence for the treatment choices. There can never be certainty how individuals and group of individuals will respond to the intervention.
Conflict of Interest: None
Source of Funding: None | | | | Compliance with Ethical Standards | | Funding None | | | | Conflict of Interest None | | |
- Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.0 [updated February 2008]. The Cochrane Collaboration, 2008. Available from www.cochrane-handbook.org.
- Phil Alderson Sally Green (editors) Cochrane Collaboration Open learning material for reviewer Version 1.1( updated November 2002) The Cochrane Collaboration, 2008 Available from www.cochrane.org.
- RevMan 5 User Guide. Available from www.cochrane.org.
- RevMan 5 Tutorials. Available from www.cochrane.org.
|
| Cite this article as: | | Bhamkar R P. Cochrane Review Methodology. Pediatr Oncall J. 2009;6: 34. |
|