BEST PRACTICES: HOW TO REVIEW THE LITERATURE

Transcription

BEST PRACTICES: HOW TO
REVIEW THE LITERATURE
Karen E
E. Luh
Luh, PhD
PhD, CQE
Corporate Director, Clinical Information Analysis
p Health System
y
Saint Joseph
Objectives
Understand definition and importance of EBPs
Know how to identify two good sources for EBP
information
Explain how quality and strength of evidence are
determined
Know how to apply EBP methods to IP issues
Understand how to evaluate studies
EVIDENCE-BASED
PRACTICE
Evidence-Based Practice
What is it?
What is driving EBP?
What are the barriers?
What are the steps?
Evidence-Based Practice - Definition
Treatment decisions based on
Best available, current evidence
Clinical expertise
Patient values (Source: IOM, 2001)
In the absence of EBP, physicians and nurses rely on:
What they learned in school
Experience (what we’ve always done)
Advice from co-workers
EBP helps us avoid this:
Evidence-Based Practice - Drivers
IOM report Crossing the Quality Chasm (2001).
Ten rules to cross the chasm from the care patients
receive to the care patients should receive
receive, including:
Evidence-based decision making
Demand
e a d for
o greater
g eate accou
accountability
tab ty
Evidence that clinicians often not aware
of most current information
Large variations in the care delivered
IOM report: “illogical”
EBP – Barriers/ Strategies
ENORMOUS increase in available evidence
Strategies: Synopses/ Reviews
Complexity of evaluating quality of
evidence
Strategies: Synopses/ Reviews
Knowledge gaps – no methods to
identify and link clinical research
priorities to research being
p
g done
Strategies: AHRQ supports research into quality of care,
guidelines, EBPs
Strategies:
g
“Smart” EMRs p
push info ((but must be kept
p up-to-date)
p
)
EBP – Barriers/ Strategies
Peer group - Practice patterns based on local standards
Strategies: P4P, public reporting
Knowledge/ skills
Strategies: Continuing education
Strategies: “Expert” caregiver leaders
Attitudes
Attit d - Resistance
R i t
tto “cookbook”
“
kb k” medicine
di i
Strategies: EBP balances evidence with expertise
Strategies: Clinician expertise is key to applying evidence to individual
Patients’ preferences
Strategies: EBP explicitly includes patient in decision-making
Evidence-Based Practice – Steps
Formulate the question - PICO
What is the Patient group or Problem?
What is the Intervention?
What is the Comparison?
What are the Outcomes?
Formulate the question - P
Patient group or Problem
How would I describe a group of patients similar to
mine?
To identify inpatients colonized with MRSA . . .
Formulate the question - I
Intervention
What is the intervention I am considering?
is it cost-effective to do 100% screening at
admission . . .
Formulate the question - C
Comparison
What alternative might I consider?
as compared to using risk factors to screen
selected patients . . .
Formulate the question - O
Outcome
What am I hoping to learn/ accomplish?
to identify need for extra precautions/ isolation?
PICO
P – To identify patients colonized with MRSA
I - is
i it cost-effective
t ff ti to
t do
d 100% screening
i att
admission
C - as compared to using risk factors to screen
selected patients
y need for extra p
precautions/ isolation?
O - to identify
Formulate the question – HH Compliance
Problem –Among healthcare workers in acute care
faclities
Intervention – what is best strategy
Comparison – as compared to current actions
Outcomes – to improve
p
hand hygiene
yg
compliance?
p
SOURCES OF EVIDENCE
Hierarchy of Evidence
EBM P
Pyramid
id and
d EBM P
Page G
Generator.
t
(c) Copyright 2006-2011. Trustees of
Dartmouth College and Yale University.
TOP OF EVIDENCE
PYRAMID (FILTERED)
Systematic Reviews
Best source – pre-appraised/ prefiltered
Performed by subject matter experts
Overview
O
i
off existing
i ti lit
literature
t
Unbiased search of literature
Explicit rules for selecting/ evaluating studies
Evidence is synthesized– may have meta-analysis
Conclusion/ recommended action
Not narrative reviews
Rigor of research method applied to review
Systematic Reviews - examples
AHRQ National Guideline Clearinghouse
http://www.guideline.gov/about/
Cochrane Database of Systematic Reviews
Cochrane.org
Abstracts free – your library may have subscription
Clinical Evidence
ClinicalEvidence.bmj.com
FIRSTConsult
Database with summaries and reviews, including systematic reviews
Centre
C t ffor E
Evidence-Based
id
B
d Medicine
M di i
Bandolier
PubMed: Find Systematic
y
Reviews
Appraise Systematic Reviews
Well-defined
W ll d fi d ttopic;
i h
homogenous sett off studies?
t di ?
Thorough search strategy?
Inclusion/ exclusion criteria clear/ unbiased?
Reviewers blind to sources/ independent?
Review included follow-up
follow up with study authors to fill in
missing information?
Robustness of findings evaluated; possible publication
bi considered?
bias
id d?
Conclusions firmly linked to evidence?
Systematic Review - HH Compliance
Interventions to improve
p
hand hygiene
yg
compliance
p
in
patient care (Review) (2011). Gould, D.J., Moralejo D.,
Drey, N., & Chudleigh, J.H.
Key conclusions:
1 Quality of intervention studies disappointing;
1.
Only four studies met criteria
2. Alcohol-based hand rubs plus education insufficient
3. Multiple strategies, involvement of staff in planning
campaigns may be helpful
4 More
4.
M
research
h needed
d d
Other Syntheses
Databases
TRIP Database (Turning Research into Practice)
UpToDate
DynaMed
Journals
Evidence-Based Nursing
International Journal of Evidence-Based Healthcare
Bandolier
These are just a sampling
pp y
Same evaluation rules apply
BOTTOM OF EVIDENCE
PYRAMID (UNFILTERED)
RCTs
Gold standard
Random
R d
assignment
i
t removes bi
bias b
by randomly
d l
distributing known and unknown variables
Control group means we can be sure that outcome is
related to intervention
Possible to make causal inferences
Hand Hygiene Compliance
Huang (2002) from Cochrane review
100 nurses randomly
d l assigned
i
d to
t educational
d
ti
l vs control
t l
groups
Direct observation of hand hygiene both before and after
intervention
Findings – significant increase in compliance post
education
d
ti compared
d tto b
baseline
li and
d control
t l group
Cohort Studies
Observational longitudinal (over time)
Compare
C
ttwo groups (cohorts)
( h t ) with
ith diff
differing
i condition/
diti /
treatment/ exposure
Retrospective or prospective
No random assignment – other variables related to the
group could have an effect on the outcome
Not possible to make causal inferences
Case – Control Studies
Compare two groups one with vs without outcome of
interest
Retrospective – often rely on medical records, patient
recall
No random assignment – other variables
ariables related to the
Not p
possible to make causal inferences
Descriptive, Qualitative Studies or Expert
Descriptive
Opinion
Case reports or series
Based on interviews, observations, focus groups
No comparison groups, no hypothesis testing
No
N generalizability
li bilit
Good for generating hypotheses (ideas)
Worst case scenario
Worst-case
FINDING THE EVIDENCE
Finding the evidence - Medline
Start with y
your reference librarian!
Start with your reference librarian!!
Database searches
Boolean logic - and, or, not
Truncation – use word stems (control$ vs controlled)
MeSH (Medical Subject Headings) – key index words
Standard filters – built into search engines
Hand Hygiene Compliance Search
From Cochrane review:
EVALUATING THE
EVIDENCE
Evaluating the evidence
Source – differentiate scientific evidence,, opinion,
p
, and
marketing
Strength – hierarchy of evidence pyramid
Quality – Is the study valid? Well done?
Even in peer-reviewed journal quality not guaranteed
Applicability to your population
Strength
Hierarchy
y
All things being equal, consider evidence hierarchy
Design
g AND execution
Well-done observational study trumps poorly executed RCT
Replication or consistency
Has this been found more than once?
Quality
Randomization – best if offsite
Attrition – are patients lost to follow-up? This could
undermine randomization
Double blinding – group assignment unknown to both
patient and individual(s) assessing outcome
Internal validity – can we be sure that intervention under
study is causing outcome and not some other variable?
Were groups treated identically EXCEPT for intervention?
External validity – can these results be generalized
beyond
y
studyy sample?
p
Integrating evidence with clinical
expertise, patient preference
Efficacy versus effectiveness
Experts working in controlled environment vs “real-world” application
Do these results apply to my patient?
Generalizability plus patient choice
Does this intervention benefit this patient?
Clinical expertise needed
Is this intervention in keeping with this patient’s values
and preferences?
Patient-centered care mandates centrality of patient choice
Summary
EBP is shift:
From reliance on experts
To integration of evidence WITH clinical expertise WITH patient’s
circumstances and values
EBP Resources
Systematic reviews
EBP journals
R f
Reference
librarians
lib i
NOT CONVINCED WE
NEED EBP?
Semelweis (1847)
Post-childbirth infection and handwashing
Current Practice in Infection Prevention –
What does the literature tell us?
Surveillance and public reporting of central line-
associated bloodstream infections
Surveillance vs administrative coding data
Nursing and physician attire as possible source of
h
hospital
it l – acquired
i d iinfections
f ti
Other examples
Quality of Surveillance for CLA-BSI
Quality of traditional surveillance for public reporting of
nosocomial bloodstream infection rates. Lin MY, Hota B, Khan
YM, Woeltje KF, Borlawsky TB et al. (2010). JAMA, 304(18): 2035-2041.
Introduction
I t d ti
lays out rationale
closes with clear statement of research questions:
measure relationship between traditional surveillance and computer
algorithm for identifying CLA-BSI
determine if there is heterogeneity in this relationship among 4 institutions
studied
Research
R
h questions
ti
and
d rationale
ti
l ffor th
those questions
ti
very
clear.
Methods Sample clearly described – 20 ICUs in 4 academic medical centers
Time frame, methods for obtaining sample described
IPs
IP blinded
bli d d to participation
i i i
Computer algorithm described; reference to publication
Discloses power analysis and statistical tools and methods
Good clear description, it would be possible to replicate
this studyy based on what we’ve learned in the methods
section
WHAT KIND OF STUDY IS
THIS?
Cohort Studies
Observational longitudinal (over time)
Compare
C
ttwo groups (cohorts)
( h t ) with
ith diff
differing
i condition/
diti /
treatment/ exposure
Retrospective or prospective
No random assignment – other variables related to the
Not possible to make causal inferences
Results
statistics with measures of spread
analyses clearly relate to research questions:
measure relationship
l ti
hi b
between
t
ttraditional
diti
l surveillance
ill
and
d
computer algorithm for identifying CLA-BSI: weak relationship
Spearman ρ (rho) = 0.34
determine if there is heterogeneity in this relationship among
4 institutions studied: ρ ranged from 0.83 to nonsignificant
0.10.
Results sufficiently detailed to allow us to determine
whether we agree with the analyses and the findings;
analyses address research questions.
Discussion or Comment
Relates results to original research questions
Relates findings to other research
Discusses
Di
possible
ibl b
bases ffor fifindings
di
(Remember they cannot make causal statements as this isn’t RCT)
Discusses limitations and ideas for future research
Authors avoid over-generalizing their results – stay within bounds of
their methods.
Administrative data compared to NHSN
criteria for identifying HAIs
Administrative coding data, compared with CDC/NHSN criteria,
are poor indicators of health care-associated infections. Stevenson
KB, Khan YM, Dickman J, Gillenwater T, Kulich P, et al. (2008). Amer J Infect
Control, 36: 155-164.
Introduction
Lays out rationale – embeds issue in current P4P/ public reporting
requirements
Closes with clear statement of research question:
“compare directly the accuracy of HAIs identified by ICD-9-CM . . . codes to those
identified by traditional epidemiologic methods . . . .
Clearly lays out concern that coding data are overly – sensitive; does NOT
address issue of whether NHSN criteria are under-sensitive
Doesn’tt adequately address arguments for use of coding data
Doesn
No star
Methods

Sample clearly described – 1 academic complex
Time frame, methods for obtaining sample described
IPs blinded for initial surveillance since study done retrospectively
Experience/ reliability of surveillance personnel not addressed
Used codes identified by independent sources – avoids investigator
bias in code selection
Methods for review of discordant cases unclear – IPs were NOT
blinded during this review – possible source of bias
Might be possible to replicate this study based on what we’ve
we ve
learned in the methods section with some reservations
No star
Results
Statistics with measures of spread
Analyses relate to research question:
Discordant cases with coding-identified
coding identified HAIs reviewed by IPs to determine if
infections also matched NHSN definitions
Subsample of discordant CLA-BSIs reviewed (50/ 485) but results
extrapolated to total sample
Accuracy computed AFTER adjustment
Results poorly described – difficult to determine actual findings and
therefore difficult to evaluate
Results somewhat hard to follow – actual numbers of cases buried in
text – tables with counts laid out as described in methods would
have been helpful.
No star
Discussion or Comment
Relates results to original research questions
Questionable evaluation of finding that ICD-9 codes flagged 59 HAIs
missed by IPs
“only a small number of these discordant cases (59/879, 7%) were subsequently
found to meet CDC/NHSN criteria, suggesting that the case-finding methods
outlined by the CDC/NHSN are reasonably robust.” (p. 161)
BUT initial surveillance identified 318 HAIs and 59/ (318 + 59) = 15.7%
Relates findings to other research
Discusses possible bases for findings
(Remember
(R
b they
th cannott make
k causall statements
t t
t as thi
this iisn’t
’t RCT)
Discusses limitations and ideas for future research
Some appearance of bias in interpretation of results
No star
QUESTIONS?
Contact information:
Karen E. Luh
@ j
g
luhke@sjhlex.org