Covid 19 Evidence Based Project Worksheet

DOI: 10.1111/j.1471-1842.2009.00848.x Review Article Blackwell Publishing Ltd A typology of reviews: an analysis of 14 review types and associated methodologies Maria J. Grant* & Andrew Booth†, *Salford Centre for Nursing, Midwifery and Collaborative Research (SCNMCR), University of Salford, Salford, UK, †School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK Abstract Background and objectives: The expansion of evidence-based practice across sectors has lead to an increasing variety of review types. However, the diversity of terminology used means that the full potential of these review types may be lost amongst a confusion of indistinct and misapplied terms. The objective of this study is to provide descriptive insight into the most common types of reviews, with illustrative examples from health and health information domains. Methods: Following scoping searches, an examination was made of the vocabulary associated with the literature of review and synthesis (literary warrant). A simple analytical framework—Search, AppraisaL, Synthesis and Analysis (SALSA)—was used to examine the main review types. Results: Fourteen review types and associated methodologies were analysed against the SALSA framework, illustrating the inputs and processes of each review type. A description of the key characteristics is given, together with perceived strengths and weaknesses. A limited number of review types are currently utilized within the health information domain. Covid 19 Evidence Based Project Worksheet

Conclusions: Few review types possess prescribed and explicit methodologies and many fall short of being mutually exclusive. Notwithstanding such limitations, this typology provides a valuable reference point for those commissioning, conducting, supporting or interpreting reviews, both within health information and the wider health care domain. Background The advent of evidence-based practice (EBP) in the early 1990s has seen the role of the health library and information worker in the ascendancy, with clinicians increasingly relying on health care literature in their decision making. With their knowledge of information sources and their skills to retrieve information to inform health care decisions, library and information sector workers Correspondence: Maria J. Grant, Salford Centre for Nursing, Midwifery and Collaborative Research, School of Nursing, University of Salford, Salford M6 6PU. E-mail: m.j.grant@salford.ac.uk have played,1 and indeed continue to play, an important role in assisting in the uptake of EBP principles and practice. It quickly became apparent that synthesized summaries of ‘all’ evidence within a particular domain would be required, in addition to the evidence from primary studies, if clinicians were to make truly informed decisions within a typical consultation. However, the review article of the time seemed ill-equipped to meet such a challenge. Medical review articles of the pre-EBP era were generally unsystematic and lacked formal statistical methods to derive best estimates of treatment effects from the available © 2009 The authors Journal compilation © 2009 Health Libraries Group. Health Information and Libraries Journal, 26, pp.91–108 91 92 A typology of reviews, Maria J. Grant & Andrew Booth information. Consequently, they tended to reach conclusions that were biased and wrong.2 Archie Cochrane, a famous British epidemiologist, noted: ‘It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials.’3 In answer to this challenge, the worldwide Cochrane Collaboration was formed in 1992 to provide an expanding resource of updateable systematic reviews of randomized controlled trials (RCTs) relating to health care. Thus began the modern incarnation of the review article, a tool that had for many centuries been the mainstay for updating scientific knowledge. Rise of the review While it is well established that, in the 18th century, James Lind was the instigator of the first reported RCT, a lesser-known fact records that he was probably the first to describe the systematic review method. Six years after his first RCT was published,4 Lind wrote: ‘As it is no easy matter to root out prejudices … it became requisite to exhibit a full and impartial view of what had hitherto been published on the scurvy … by which the sources of these mistakes may be detected. Indeed, before the subject could be set in a clear and proper light, it was necessary to remove a great deal of rubbish.’5 Gathering research, getting rid of rubbish and summarizing the best of what remains captures the essence of the science of systematic review. Covid 19 Evidence Based Project Worksheet

Nevertheless, although the need to synthesize research evidence has been recognized for well over two centuries, it was not until the 20th century that researchers began to develop explicit methods for this form of research. A fuller account of the rise of the discipline of research synthesis has been published by Iain Chalmers and other distinguished proponents.6 Recent years have seen recognition that the typical timescale commanded by the rigour of the systematic review process may be unsuited to the decision-making windows available to most policymakers. The need to trade off rigour and relevance has become a central theme to recent methodological developments and has led to a bewildering plethora of review designs developed to meet a variety of demands from the domains of research and policy. An early example of a review was published in the non-health library and information sector,7 whilst the early 1980s saw the first of many manuscripts seeking to answer the eternal question of how best to undertake bibliographic instruction in the health sciences.8 Coinciding with the first Evidence-Based Library and Information Practice (EBLIP) conference in 2001, Booth proposed that the library and information science (LIS) sector could follow the trend within medical EBP by graduating to the development of more systematic reviews once a critical mass of rigorous studies has been attained.9 A recent evaluation of the evidence base has continued to sound such a call for establishing a solid evidence base within the LIS sector.10 To ‘review’ has been defined as: ‘To view, inspect, or examine a second time or again’.11 This definition broadly characterizes all review types now in existence. What remain largely unacknowledged are the subtle variations in the degree of process and rigour within the multifarious review types. Such variations are most clearly evidenced in the structures and methodologies that distinguish one review type from another. As more professions have adopted and adapted the systematic review method, the LIS sector, as elsewhere, has been afforded access to an everincreasing variety of methods and techniques for summarizing the evidence base. The model of the systematic review of randomized controlled trials has limited potential within the LIS literature, given that no more than a score of such studies had been identified. This has necessitated the identification of a greater range of review types, opening up the prospect of summarizing case studies, qualitative research and even theoretical and conceptual published and unpublished outputs. Different types of reviews Analysis of download statistics from the Health Information and Libraries Journal electronic archive © 2009 The authors Journal compilation © 2009 Health Libraries Group. Health Information and Libraries Journal, 26, pp.91–108 A typology of reviews, Maria J. Grant & Andrew Booth since 2006 indicates that reviews figure prominently amongst the most highly sought articles. This suggests that, as with medicine before it, the LIS sector values the opportunity to access already synthesized evidence in informing its practice. Indeed, in 2008, Ankem noted that there was evidence of systematic reviews making a ‘substantial contribution to medical library and information literature’.12 Ankem, in an evaluation of methods in systematic reviews and meta-analyses published between 1996 and 2006 in the library and information science sector, identified a total of eight manuscripts. These manuscripts, combined with reviews published in Health Information and Libraries Journal following an editorial commitment in 2007 to seek to publish a review in each future issue,13 provide an illuminating insight into the flourishing terminology synonymous with this genre. Covid 19 Evidence Based Project Worksheet

This terminology includes such terms or phrases as review of the evidence,14,15 comprehensive review,16 literature review,17 overview18 and systematic review.19–30 Given the importance evidence-based practice places upon the retrieval of appropriate information, such diverse terminology could, if unchecked, perpetuate a confusion of indistinct and misapplied terms. Objective The objective of this study is to provide a descriptive insight into the most common examples of review, illustrated by examples from health and health information domains. Method After initial scoping searches of the literature, the authors drew on their combined experience of 26 years of having worked with both the theory and practice of reviews in multifarious guises to examine the vocabulary used in the published literature, unpublished documents and other source material. The purpose was to determine the prevalent terminology; a process known as literary warrant.31 From this, common review types and their associated key attributes were identified and mapped against a Search, Appraisal, Synthesis and Analysis (SALSA) framework. Each review type was analysed, its characteristics were described and its perceived strengths and weaknesses were outlined. An example of each type of review was identified and selected, primarily for its usefulness in illustrating review characteristics. No judgement of quality is implied by each selection. Results: characterizing types of review Fourteen review types and associated methodologies were analysed against the SALSA framework (see Table 1). To inform the subsequent analysis, the same framework had been previously applied to 17 review manuscripts identified from combining those figuring in the Health Information and Libraries Journal review feature between 2007 and 2009 with papers cited in Ankem’s Review of Reviews (see Table 2). A descriptive summary appears below. Critical review Description. A critical review aims to demonstrate that the writer has extensively researched the literature and critically evaluated its quality. It goes beyond mere description of identified articles and includes a degree of analysis and conceptual innovation. An effective critical review presents, analyses and synthesizes material from diverse sources. Its product perhaps most easily identifies it—typically manifest in a hypothesis or a model, not an answer. The resultant model may constitute a synthesis of existing models or schools of thought or it may be a completely new interpretation of the existing data. Perceived strengths. The ‘critical’ component of this type of review is key to its value. Under normal circumstances, conceptual innovation develops through a process of evolution or accretion, with each successive version adding to its predecessors. A critical review provides an opportunity to ‘take stock’ and evaluate what is of value from the previous body of work. It may also attempt to resolve competing schools of thought. As such, it may provide a ‘launch pad’ for a new phase of conceptual development and subsequent ‘testing’. Covid 19 Evidence Based Project Worksheet

Perceived weaknesses. Critical reviews do not typically demonstrate the systematicity of other © 2009 The authors Journal compilation © 2009 Health Libraries Group. Health Information and Libraries Journal, 26, pp.91–108 93 94 Methods used (SALSA) © 2009 The authors Journal compilation © 2009 Health Libraries Group. Health Information and Libraries Journal, 26, pp.91–108 Label Description Search Appraisal Synthesis Analysis Critical review Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or model Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness. May include research findings Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature Seeks to identify most significant items in the field No formal quality assessment. Attempts to evaluate according to contribution Typically narrative, perhaps conceptual or chronological Significant component: seeks to identify conceptual contribution to embody existing or derive new theory May or may not include comprehensive searching May or may not include quality assessment Typically narrative Analysis may be chronological, conceptual, thematic, etc. Completeness of searching determined by time/scope constraints No formal quality assessment May be graphical and tabular Meta-analysis Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics Quality assessment may determine inclusion/ exclusion and/or sensitivity analyses Requires either a generic appraisal instrument or separate appraisal processes with corresponding checklists Graphical and tabular with narrative commentary Mixed studies review/mixed methods review Aims for exhaustive, comprehensive searching. May use funnel plot to assess completeness Requires either very sensitive search to retrieve all studies or separately conceived quantitative and qualitative strategies Characterizes quantity and quality of literature, perhaps by study design and other key features. Covid 19 Evidence Based Project Worksheet

May identify need for primary or secondary research Numerical analysis of measures of effect assuming absence of heterogeneity Literature review Mapping review/ systematic map Overview Qualitative systematic review/qualitative evidence synthesis Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies Typically both components will be presented as narrative and in tables. May also employ graphical means of integrating quantitative and qualitative studies May or may not include May or may not include Synthesis depends on comprehensive searching quality assessment (depends whether systematic or not. (depends whether whether systematic Typically narrative but may systematic overview or not) overview or not) include tabular features May employ selective Quality assessment Qualitative, or purposive sampling typically used to narrative synthesis mediate messages not for inclusion/exclusion Analysis may characterise both literatures and look for correlations between characteristics or use gap analysis to identify aspects absent in one literature but missing in the other Analysis may be chronological, conceptual, thematic, etc. Thematic analysis, may include conceptual models A typology of reviews, Maria J. Grant & Andrew Booth Table 1 Main review types characterized by methods used Methods used (SALSA) Label Description Search Appraisal Synthesis Analysis Rapid review Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research) Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives on issue or point out area for further research Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review Completeness of searching determined by time constraints Time-limited formal quality assessment Typically narrative and tabular Quantities of literature and overall quality/direction of effect of literature Completeness of searching determined by time/scope constraints. May include research in progress Aims for comprehensive searching of current literature No formal quality assessment Typically tabular with some narrative commentary No formal quality assessment Typically narrative, may have tabular accompaniment Characterizes quantity and quality of literature, perhaps by study design and other key features. Attempts to specify a viable review Current state of knowledge and priorities for future investigation and research Aims for exhaustive, comprehensive searching Quality assessment may determine inclusion/exclusion Typically narrative with tabular accompaniment Combines strengths of critical review with a comprehensive search process. Covid 19 Evidence Based Project Worksheet

Typically addresses broad questions to produce ‘best evidence synthesis’ Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results Aims for exhaustive, comprehensive searching May or may not include quality assessment Minimal narrative, tabular summary of studies What is known; recommendations for practice. What remains unknown; uncertainty around findings, recommendations for future research What is known; recommendations for practice. Limitations May or may not include comprehensive searching May or may not include quality assessment Typically narrative with tabular accompaniment What is known; uncertainty around findings; limitations of methodology Identification of component reviews, but no search for primary studies Quality assessment of studies within component reviews and/or of reviews themselves Graphical and tabular with narrative commentary What is known; recommendations for practice. What remains unknown; recommendations for future research Scoping review State-of-the-art review Systematic review Systematic search and review Systematized review Umbrella review A typology of reviews, Maria J. Grant & Andrew Booth © 2009 The authors Journal compilation © 2009 Health Libraries Group. Health Information and Libraries Journal, 26, pp.91–108 Table 1 Continued 95 96 Methods described (SALSA) © 2009 The authors Journal compilation © 2009 Health Libraries Group. Health Information and Libraries Journal, 26, pp.91–108 Authors (year) Description No. of included studies Search Appraisal Synthesis Analysis Ankem (2006)19 Systematic review of the research literature Systematic review 110 studies 3 databases None 29 14 databases Meta-analysis and descriptive statistics Thematic using32 Not specified 24 Not specified 3 databases 54 7 databases Brown (2008)24 Systematic review 23 databases Articles from popular press, magazine and newspaper articles reviewed for types of information published Childs et al. (2005)25 Systematic re … Covid 19 Evidence Based Project Worksheet