Autoverification in Clinical Chemistry

12/13/2011
2
Autoverification in
Clinical Chemistry:
From Theory to Practice
Disclosure
I have NO financial interests in
any of the commercial products
discussed in this presentation.
By:
Raffick Bowen, MLT (CSMLS), PhD, DClChem, FCACB, DABCC, MHA (c)
Assistant Professor of Pathology
Associate Director of Chemistry and Immunology Laboratory
Stanford University
December 13, 2011
Webinar- Lab Administration
3
SUMC Chemistry Lab Information
Stanford University Hospital (SHC)
~ 456 active beds
Lucile Packard Children’s Hospital (LPCH)
~ 218 beds
4
3
SHC Chemistry Volumes
• Tests per year at SHC Chemistry: 5 -6 million
 30% LPCH specimens
 70% SHC specimens
 1500 – 2500 tubes per day
 Total of 5-6 million chemistry tests per year
Clinical Chemistry Section:
• Tubes: 1500 – 2500 tubes per day
AM Shift: 7 CLSs and 4 LTs
PM Shift: 4 CLSs and 2 LTs
MN Shift: 3-4 CLSs and 2-3 LTs
• Urines: 40-50 per day
• Misc fluids (e.g. CSF, pleural, peritoneal) 10-20/day
Major instruments:
4 Siemens RxLs
3 ABL Blood Gas analyzers
3 Osmometers
1 Immulite 1000
2IMx
2 TDx/FLx
Hillview Special Chemistry lab ~ 5 miles away from Stanford Hospital
• 40 – 60% STAT specimens –
At peak times (5am – 9am and 3pm to 5pm) it is much higher
• 65 – 80% AV
• Over 180 tests at SHC chemistry
1
12/13/2011
5
SHC Main Chemistry Panels
Panel
~ 15,000 per month
Met C
~ 15,000 per month
Chem 23
~ 1000 – 1500 per month
Blood gases & co-oximetry
Other Notable SHC Chemistry Tests
Panel
# Per Month
Met B
6
~ 4000 ABGs per month
~ 1700 VBGs per month
~ 200 – 300 cord BGs per month
# Per Month
Sweat chloride
35 – 40 per month
Osmolality
440 per month
Cortisol
40 per month
IOPTH
2 – 5 patients per month
iCa
~ 4000 per month
TnI
~ 2300 per month
Lactates
~ 1600 per month
7
Patient Safety
From the Institute of Medicine (IOM) :
“To Err is Human” and “Crossing the Quality Chasm” report in 1999,
it estimated that of 33.6 million admission to US hospitals in 1997
~44,000 to 98,000 admission die as result of medical errors
Cost: 37.6 to 50 billion for adverse events and 17 to 29 billion for preventable adverse events
A 2005 JAMA article suggests that 5 years after the IOM report there have NOT
been substantial improvements in patient safety.
The IOM report has no explicit information on lab errors
~60-80% of clinical decision is based on laboratory data
Bonini et al (1992):
- most lab errors occur in pre-analytical phase (32-75%) - CPOE
- analytical phase (13-32%)- better instruments
8
What is Autoverification?
College of American Pathologists (CAP):
“Autoverification is the process by which patient results are
generated from interfaced instruments and sent to the LIS, where
they are compared against laboratory-defined acceptance
parameters.
If the results fall within these defined parameters, the results are
automatically released to patient reporting formats without any
additional laboratory staff intervention.
Any data that fall outside the defined parameters is reviewed by
laboratory staff prior to reporting.”
- post-analytical phase (9-31%) –interfaced instruments
NPSG 2c-Timely reporting of critical values and critical results
(Kohn et al 1999;Leape and Berwick, 2005)
http://www.cap.org/apps/docs/laboratory_accreditation/checklists/laboratory_general_sep07.pdf
2
12/13/2011
9
10
Why Autoverification?
California State Legislation on
Autoverification
• ↑ Consistency:
▫ Autoverification removes technologist’s “subjectivity” and improves
consistency of reporting (regardless of the number and skill set of the
technologists in the lab)
California Business and Professions Code Section 1209.5
 Governor Schwarzenegger signed AB 2156 (Niello) into law,
effective January 1, 2007:
• ↑ Quality:
Items discussed in law:
▫ Autoverification reduces errors/mistakes, and improves quality
1. Lab Director or Authorized designee to establish, validate, and
document AV criteria
• ↓ TAT:
2. Lab Director or Authorized designee must annually revalidate AV
criteria.
▫ Autoverification reduces amount of labor required for validation of results
▫ About 60-80% of results could be automatically verified, while 20-40%
require further attention.
3. Authorized designee must be licensed to work in clinical laboratory
according to CA state regulations.
Holy Trinity of Lab Testing
1. Increase Patient Safety
2. Decrease TAT
3. Cost-Savings
4. A licensed person must be physically present onsite and held
responsible for accuracy and reliability of results
www.dhs.ca.gov/ps.ls.lfsb
11
How to go about implementing Autoverification?
SUMC Autoverification Team Members:
~ 11-15 people:
•
•
•
•
•
•
•
•
•
LIS personnel
IT personnel
Supervisors
Operation Manager
Reference Technologist
Chemistry Medical Director (Dr. Faix)
Chemistry Associate Director (Dr. Bowen)
CLSs
Hospital Administrators (only for approval of project)
•
Weekly meetings
•
8-9 months from planning to implementation for each phase
•
2 months post autoverification implementation for each phase
•
•
Phase I (routine plasma chemistry tests) - Jan 2008
Phase II (DOAs, TDMs, Urine chemistry tests) - Jan 2009
12
What Guideline is Available for Autoverification?
 CLSI AUTO 10-A; Volume 26 Number 4
Autoverification of Clinical Laboratory Test Results
Guideline
o Provides general framework that will allow labs to
design, implement, validate and customize rules for
autoverification based on the needs of its own patient
population
o Includes supporting sections that deal with different
aspects of regulatory compliance and validation of
algorithms that are essential to establishing and
maintaining a modern autoverification program
3
12/13/2011
13
CAP Checklists Questions
GEN.43850 Autoverification Approval
What to autoverify (SUMC AV Phase I)?
Phase II
There is a policy signed by the laboratory director approving the use of autoverification procedures.
GEN. 43875 Autoverification Validation
14
Analytes autoverified at SUMC clinical chemistry laboratory
Phase II
There is documentation that the autoverification process was validated initially, and is tested at least annually and
whenever there is a change to the system that could affect the autoverification logic.
GEN.43878 Autoverification QC Samples
Phase II
For all test results subject to autoverification, the laboratory ensures that applicable quality control samples have
been run within an appropriate time period, with acceptable results.
GEN.43881 Autoverification Result Comparability
Phase II
Results are compared with an appropriate range of acceptable values prior to autoverification.
GEN.43884 Result Flags
Phase II
Results are checked for flags or warnings prior to autoverification.
GEN.43887 Autoverification Audit Trail
Phase II
The audit trail in the computer system identifies all test results that were autoverified, and the date/time of
autoverification.
GEN.43890 Delta Checks
Phase I
The autoverification process includes all delta checks that the laboratory performs prior to manual release of test r
results.
GEN.43893 Autoverification Suspension
Phase I
The laboratory has a procedure for rapid suspension of autoverification.
15
What NOT to Autoverify?
• Specimens NOT autoverified in Clinical Chemistry
section of the Clinical Laboratory at Stanford
• Beta-hCG (plasma)
• Ethanol (need osmolality result)
• Tricyclic antidepressants (manual tests)
16
Where to implement AV?
•
Technology available to implement AV:
▫ Technology Layers:
• Instrument layer - RxLs, LX-20s, Centaurs, etc
• Middleware layer –Data Innovations, Dawning Technologies
e.g. Easylink™ on Siemens Vista instruments
Centralink™ on Bayer Advia
• LIS layer – Sunquest (Misys), Meditech, Soft Lab, etc
• Body fluidse.g. CSF, transudates, exudates, etc.
• Combination(s) of the above
At SUMC, we built the AV rules in Sunquest™ (Misys)
4
12/13/2011
17
Dynamic
Download
LIS
Interfaced
Orders
Streamlab
RxLs
• EPIC-SHC
• Cerner –LPCH
Interfaced
Results
How to deactivate AV?
•
Host-Query
Download
HIS
18
Procedure for AV deactivation
• Function: AF (AV functionality) in
Sunquest system
• Downtime Protocol – unscheduled
– scheduled
Results
Upload
LIS
Of course, you need AV reactivation protocols
Autoverification in our LIS system (Sunquest™)
19
Before AV Testing:
• Instrument(s) / Method(s) must be thoroughly
validated
• Staff must be trained and competent with analyzer(s)
• Calibrations and Quality Control must be acceptable
• Adequate staff for AV testing
• Awareness of pre-analytical, analytical, and postanalytical issues in your setting.
• LIS, LIS, LIS !!! (knowledge of people and system)
• Always expect the unexpected!!
20
2007 Training
AV Timeline for RxLs at SUMC
 August to December 7 (Friday) –
 Testing the AV criteria in Misys
 December 10 (Monday) –
 Completed final documentation of AV testing in Misys
 December 11 (Tuesday) to December 31 (Monday) –
 Training of staff in AV testing in Misys (instrument flags, calculations, HIL)
 December 17 (Monday) at 11:00am –
 GO LIVE with instrument flags, calculations, HIL in Misys (NO
AUTOFILING)
 January 8 (Tuesday) at 11:00am 2008
 GO LIVE with AV in Misys for phase I
5
12/13/2011
21
22
Quality Control
The control of the testing process to ensure that test results meet
their quality requirements
Control
Data
AV Parameters (RULES)
to be developed and tested
12s
QC – Westgard’s Rules:
No
IN – CONTROL
ACCEPT RUN
Yes
13s
Yes
No
No
22s
Yes
No
R4s
Yes
OUT-OF-CONTROL
No
4is
No
10-
x
Yes
Yes
REJECT RUN
At SUMC, we do NOT have any quality control rule failures built into our
AV parameters (rules)!
QC review is done manually because:
23
1.
To identify any QC specimens that were not performed
against a QC checklist
2.
To monitor trends and shifts on LJ charts
24
Reference Intervals
• Based on patient’s age, gender, etc
Reference interval at SUMC:
e.g. Sodium: 135-145 mmol/L
• What would you do with a sodium of 127 mmol/L ?
At SUMC panic sodium values are < 125 and > 160 mmol/L
6
12/13/2011
25
26
Most panic results are called by
our Customer Service personnel
27
Delta Check:
The difference between a patient's present laboratory result and
the previous result which exceeds a predefined limit is referred
to as a delta check
•
•
Types of Delta Checks - absolute (e.g. ± 4 mmol/L for sodium)
- % change (e.g. ± 10%)
- rate change (e.g. ± 10% over 7 days)
Delta checks are investigated by the lab internally to rule out:
1) Mislabeling
2) Clerical error
3) Possible analytical error- QNS
▫
Delta Checks: How to determine?
•
•
•
•
•
Technologist Experience
Data Review
Clinician Input
Literature
Reference Change Values
28
Calculation of RCV (Significant Change):
• Calculation of RCV (Significant Change):
1. Analytical Variation (Imprecision)
2. Biologic Variation
 Experimental data
RCV = Z x 21/2 x (CVA2 + CVI2)1/2
• Z –Probability or Degree of Significance
Z= 2.58 at 99% Probability (Highly Significant)
Z = 1.96 at 95% Probability (Significant)
• CVA –Analytical Variation
• CVI –Biological Variation (within-subject)
 Fraser CG, Biologic Variation: From Principles to Practice, AACC Press, 2001
 Ricos C, et al., Current databases on biologic variation: pros, cons, and progress. Scand J Clin Lab Invest 1999;59:491500. www.westgard.com/guest17.htm
7
12/13/2011
29
30
Instrument Flags:
Instrument physical status flags
• Examples of RxL Instrument Flags and Errors Criteria
Flag
Description
Cup or Test
Level Failures
AL
Above linearity
Test Level
BM
Breaker/mixer slipped
Cup Level
DE
Decode
Cup Level
MP
Missing pack
Cup Level
PL
Pump lost steps
Cup Level
TE
Temperature
Cup Level
AB
Absorbance
Test Level
BL
Below linearity
Test Level
31
32
Example of Calculation Flags:
Misys/SQ Code
(RXL Code)
Calculation
Auto Verification
Impact
AGAP (CALC)
Na – (Cl +CO2)
Results <2 or > 30 will
FAIL auto verification
for the CUP
LDL (CALC)
Total Cholesterol – (HDL TRIG/5)
Results for TG >400
mg/dL will FAIL auto
verification for the
CUP
ALB (CALC)
Albumin > Total Protein
FAIL auto verification
for the CUP
CK-MB (CALC)
CK-MB > Total CK
FAIL auto verification
for the CUP
IBIL (CALC)
IBIL, DBIL > TBIL
FAIL auto verification
for the CUP
8
12/13/2011
33
“HIL” Indices Flags:
HIL Comments
Examples of Hemolysis, Icterus, and Lipemia messages
Hemolysis (HIL) Related Comments
Serum Indices
4 (any H value ≥ 4)
(405 nm)
Range of HIL
Test codes with
Results
Appended Comments
HIL value >410
K , MG
4 (any I value ≥ 4)
Range of HIL
Test codes with
Results
Appended Comments
141-146
CHOL, CK, GL, TP
5 (any H value ≥ 5)
HEMIN
HEMOLYSIS
ICTERUS
LIPEMIA
Code to append
ICTDE
HEMDE: Hemolysis
ICTDE: Icteric specimen
TURDE: Turbidity present
-may tend to decrease result
result
-may tend to decrease result
-may tend to decrease
(700 nm)
Lipemia (HIL) Related Comments
Serum Indices
(0-1, Slight; 2-4, Moderate; 5-6, Gross)
Code to append
(452 nm)
Icterus (HIL) Related Comments
Serum Indices
34
Range of HIL
Test codes with
Results
Appended Comments
115-116
GL, PHOS
Code to append
HEMIN: Hemolysis present
ICTIN: Icteric specimen
TURIN: Turbidity present
TURIN
-may tend to increase result
result
-may tend to increase result
-may tend to increase
LIS automatically appends the appropriate comment(s) with results
35
Electronic Simulated Data Validation:
36
Clinical Specimens:
9
12/13/2011
37
38
Failed AV (K < 3.0 mmol/L)
Criteria for Autoverifying Specimen
 Check to see if calibrators / QC has been run and within
acceptable limits
 Check to ensure that there are no instrument errors /
flags
 Results are within instrument / AV range
-the results should be held for manual review when the
results are outside the range
Passed AV
 Not a critical value
-those results that need immediate attention
 Passed delta check criteria
-absolute, percent change, rate change
 No calculation errors (e.g. anion gap)
(We have TEN 3 inch binders FULL of AV validation documentation)
39
40
ED Basic Metabolic Panel (Met B)
Receipt to Verify
Bench to Verify
Receipt to Bench
Target Receipt to Verify
MINUTES, 90th Percentile
60
Streamlab installation project
50
50
45
Target
41 41 41
40
35
34 33 33 33
30
32
30
26
20
10
24
34 34
32 32 32 31
25 25 24
23
26
32
29 29
23 23 24
34
30 29 29
28
25 25 25 24 25
20
29
31
29 28
31 31 32 32
26 26
26 25 27 26
23 22 23 23 23
33 34
35
29
27 28
37 38 36 36
33
26
30 30 31 29 29
24
18
12 12 12
10 10 11
12
10 9
9 10
11
9
7 8 7
7 8
12
8 8 8 8 9
7 6 7 7 7 7
8
10
8
11 10 10
8
0
10
12/13/2011
41
90th percentile ED TAT (CAP Q-Probe Studies)
42
October 2011
• Met B
• Met C
• Chem. 23
surrogates
Total
Specimens
Total
Autofiled
Percent
Autofiled
K
22409
19035
85%
Met B
10190
8711
85%
Met C
9572
8075
84%
TnI
1077
724
67%
Total
Specimens
Total
Autofiled
Percent
Autofiled
K
6374
5371
84%
Met B
1585
1284
81%
Met C
3795
3219
85%
TnI
386
293
76%
Total
Specimens
Total
Autofiled
Percent
Autofiled
K
28783
24406
85%
Met B
11775
9995
85%
Met C
13367
11294
85%
TnI
1463
1017
70%
Routine specimens from SHC and LPCH
STAT specimens from SHC and LPCH
October 2011
• CBC
surrogate
Total Autofiled
(Routine and STAT specimens)
October 2011
43
Improve quality of laboratory results
HIL:
- Comments appear with patient results
Delta checks:
- helps identify mislabeled specimens (e.g. sodium +/- 8 mmol/L)
AV range:
- set a AV range to help identify clotted specimens or QNS
e.g. ALP: 5 - 3500 U/L, if < 5 U/L, check specimen
Added comments to CLS:
“Check Printout” for calculation errors or “Repeat Lytes” for low anion gaps (< 2)
Standardize reporting of results like anion gaps (<2 instead of
actual number like -15)
44
Some Benefits of Implementing AV in the Chemistry Section of
the Clinical Laboratories at Stanford
o Less phone calls (from 25-30 to <5 per shift); less complaints
o Pending log has been reduced from 15-20 pages to 3-6 pages per shift (20-30
specimens per page)
o Increase staff morale and less fatigue and stress; less sick calls
o Decrease TAT of chemistry results; meeting our TAT goals!
o More organized workflow – Standardize SOPs
o More consistent reporting of results-decrease errors and improve quality
o Dynamic screen has less specimens to track
o Less overtime ~ 30 mins/day (7 hours per pay period)
Instrument Flags:
– on screen, no need to look at instrument printout
o Identification lab staff who needed some re-education on SOP, policies, etc
11
12/13/2011
45
$$$
COST SAVINGS
46
Monitoring AV
-Chart review (quarterly):
- results (RxL to LIS to HIS)
- check AV rules
- calculations
- HIL comments
$$$
1 FTE saved :
 2 Chemistry Verifiers Down to 1
RxL
 The 1 FTE moved to Sweat Chloride bench
and cortisol and IO-PTH bench
We now offer sweat testing everyday
- this was NOT a budgeted position.
LIS
HIS
January: CrCl, AGAP, Globulin, IBIL, BUN/CR, LIPID
April:
Urine ratios and excretion
July :
TRFSN, eGFR, Delta checks, HIL
October: CK-MB relative index, Osmolal gap, Instrument errors
47
48
Some References:
1. Duca DJ. Autoverification in a laboratory information system. Lab Medicine 2002;33:21-5.
2. Pearlman ES, Bilello L, Stauffer J, Kamarinos A, Miele R, Wolfert MS. Implications of
autoverification for the clinical laboratory. Clin Leadersh Manag Rev 2002;16:237-239
3. Crolla LJ, Westgard JO. Evaluation of rule-based autoverification protocols. Clin Leadersh
Manag Rev. 2003;17:268-72.
4. Clinical and Laboratory Standards Institute (CLSI). Autoverification of Clinical Laboratory
Test Results, Approved Guidelines (AUTO 10-A) Wayne, PA: Clinical and Laboratory
Standards Institute, 2006; Vol 26, No 32.
5. College of American Pathologists Commission on Laboratory Accreditation, Laboratory
General Checklist, 2009.
6. Torke, N. Process Improvement and Operational Efficiency through Test Result
Autoverification. Clin Chem 2005; 51: 2406-2408.
7. Biological Variation: From Principles to Practice. Callum G. Fraser. Washington, DC:
AACC Press, 2001.
Any Questions?
Email:
rbowen@stanfordmed.org
8. Fraser CG, Stevenson HP, Kennedy IMG. Biological variation data are necessary
prerequisites for objective autoverification of clinical laboratory data. Accred Qual Assur
2002;7: 455-460
12