Chapter 4 Additional Quality Control Statistics - YouTube

Welcome to Chapter 4: Additional Quality Control Statistics.

This module will expand your knowledge of basic statistics used in the Laboratory.

Let’s begin with Coefficient of Variation [CV].

CV is the ratio of the standard deviation to the mean and is expressed as a percentage.

The CV allows the technologist to make easier comparisons of the overall precision.

Since standard deviation typically increases as the concentration of the analyte increases,

the CV can be regarded as a statistical equalizer.

If the technologist/technician is comparing precision for two different methods and uses

only standard deviation, he or she can be easily misled.

For example, a comparison between hexokinase and glucose oxidase (two methods for assaying

glucose) is required.

The standard deviation for the hexokinase method is 4.8 and it is 4.0 for glucose oxidase.

If the comparison only uses standard deviation, it can be incorrectly assumed that the glucose

oxidase method is more precise that the hexokinase method.

If, however, a CV is calculated, it might show that both methods are equally precise.

Assume the mean for the hexokinase method is 120 and the glucose oxidase mean is 100.

The CV then, for both methods, is 4%.

They are equally precise.

The Coefficient of Variation can also be used when comparing instrument performance.

In this example, Instrument #1 and Instrument #2 have similar precision for calcium and

glucose, but Instrument #1 demonstrates much better precision than Instrument #2 for phosphorus.

Because the precision was calculated from data for the same lot number and level of

control, the differences in precision are likely due to the instrument or reagent.

In the second table, the difference in performance is probably due to the change from Reagent

#1 to Reagent #2.

However, it could also be due to lack of regular maintenance or some other cause.

The data in this table is for three different kits for testing ß-hCG.

Kits #1, #2 and #3 exhibit similar performance in the normal range (mid-range) and at the

high end of the method curve.

However, Kit #3 has a much higher CV at the low end of the curve.

This lack of precision at the low end of the method curve for ß-hCG provides justification

to use either Kit #1 or #2, rather than Kit #3 for testing.

Imprecision and inaccuracy are most important at the clinical decision levels.

For ß-hCG, the clinical decision levels are at low concentrations (corresponding to the

early pregnant state in the female and early testicular cancer in the male) or at moderate

concentrations (to diagnose the progression of pregnancy).

The previous examples have shown how CV can be used to compare and evaluate instruments

or reagents.

So, what is an acceptable CV?

There are several sources which may be referenced to determine expected levels of precision.

These include: Precision information provided in the product insert or instrument manual.

Interlaboratory comparison programs.

Proficiency surveys.

Evaluations of instruments and methods published in professional journals.

CLIA proficiency limits (US).

There are several sources that provide performance expectations to which the laboratory can compare

its standard deviation.

These include the instrument manual or test method description, proficiency surveys and

interlaboratory QC programs.

Instrument manuals and test method descriptions publish expectations for between-run and withinrun

precision.

These expectations are determined by the manufacturer through repetitive testing and may reflect

ideal conditions.

If the method description defines a between-run precision of 0.1 mmol/L for potassium, then

the laboratory performance in the example meets manufacturer specifications.

If however, the between-run specification is 0.05 mmol/L, then the standard deviation

calculated for the example indicates that the laboratory is less precise than the manufacturer’s

expectation.

This may indicate a possible problem exists.

However, before any final assessment is made, the laboratory should compare its results

to proficiency and/ or interlaboratory QC reports which are more indicative of “real

world” experience.

Laboratories participating in a proficiency testing program receive a set of “unknown”

liquid or lyophilized samples.

The samples are assayed by the laboratory for each test performed.

Results are obtained and reported to the proficiency agency.

The agency collects the data and, using various statistical models, determines what the consensus

value of the unknown sample should be for each test.

Then, the test result reported by each laboratory is compared to this consensus value and the

laboratory is graded for accuracy.

In an interlaboratory comparison program, laboratories submit monthly data collected

for each control product tested.

These data are combined with data from other laboratories which use the same instrument.

The benefit of an interlaboratory program over a proficiency program is that the interlaboratory

program provides statistics collected from repeated daily testing whereas the proficiency

program provides statistics collected from single events that occur only 3 times a year

in the United States and somewhat more frequently in other countries.

There are a number of published performance limits for commonly tested analytes in the

United States CLIA regulation.

These limits can be accessed on the internet at the following web address.

Although accuracy of test results is paramount in the clinical laboratory, precision is just

as important.

One way a laboratory can determine whether the precision of a specific test is acceptable

is to compare its precision to that of another laboratory performing the same test on the

same instrument using the same reagents (laboratory peer group).

An easy way to make this comparison is to divide the laboratory CV by the laboratory

peer group CV obtained from an interlaboratory comparison report.

For example, if the CV for potassium on a particular instrument is 4% and the potassium

for all other laboratories using the same instrument is 4.2%, then the coefficient of

variation ratio [CVR] is 4/4.2 or 0.95.

Any ratio less than 1.0 indicates that precision is better than the peer group.

Any score greater than 1.0 indicates that imprecision is larger.

Ratios greater than 1.5 indicate a need to investigate the cause of imprecision and any

ratio of 2.0 or greater usually indicates need for troubleshooting and corrective action.

Something in the test system is causing the increased imprecision and patient test results

may not be entirely reliable.

Certainly, repeated tests such as glucose for diabetic patients or prothrombin times

for patients taking coumadin will not be reliable when the imprecision is high.

Now let’s move on to the standard deviation index [SDI].

SDI is a peer-based estimate of reliability.

If the peer group mean is defined as meanGroup, the standard deviation is defined as sGroup

and the laboratory’s mean is defined as meanLab.

The target SDI is 0.0 which indicates a perfect comparison with the peer group.

The following guidelines may be used with SDI.

A value of: 1.25 or less is considered acceptable.

1.25 – 1.49 is considered acceptable to marginal performance.

Some investigation of the test system may be required.

1.5 – 1.99 is considered marginal performance and investigation of the test system is recommended.

2.0 or greater is generally considered to be unacceptable performance and remedial action

is usually required.

We have reached the end of this module.

Let’s review some basic points.

The Coefficient of Variation [CV] is the ratio of the standard deviation to the mean and

is expressed as a percentage.

The CV allows the technologist to make easier comparisons of the overall precision.

This is the formula for CV.

The Coefficient of Variation can also be used when comparing instrument performance.

There are several sources which may be referenced to determine expected levels of precision.

Precision information provided in the product insert or instrument manual.

Interlaboratory comparison programs.

Proficiency surveys.

Evaluations of instruments and methods published in professional journals.

CLIA proficiency limits (US).

There are several sources that provide performance expectations to which the laboratory can compare

its standard deviation.

These include the instrument manual or test method description, proficiency surveys and

interlaboratory QC programs.

Instrument manuals and test method descriptions publish expectations for between-run and within-run

precision.

These expectations are determined by the manufacturer through repetitive testing and may reflect

ideal conditions.

Laboratories participating in a proficiency testing program receive a set of “unknown”

liquid or lyophilized samples, the samples are assayed by the laboratory for each test,

results are reported to the proficiency agency, using various statistical models the agency

determines what the consensus value of the unknown sample should be for each test, and

the laboratory is graded for accuracy.In an interlaboratory comparison program, laboratories

submit monthly data collected for each control product tested.

These data are combined with data from other laboratories which use the same instrument.

There are a number of published performance limits for commonly tested analytes in the

United States CLIA regulation.

Coefficient of Variation Ratio allows a laboratory to compare its precision to that of another

laboratory performing the same test on the same instrument using the same reagents.

This is the formula for CVR.

The standard deviation index [SDI] is a peer-based estimate of reliability.

This is the formula for SDI.

The target SDI is 0.0 which indicates a perfect comparison with the peer group.

The following guidelines may be used with SDI.

A value of: 1.25 or less is considered acceptable.

1.25 – 1.49 is considered acceptable to marginal performance.

Some investigation of the test system may be required.

1.5 – 1.99 is considered marginal performance and investigation of the test system is recommended.

2.0 or greater is generally considered to be unacceptable performance and remedial action

is usually required.

For all your laboratory QC needs go to www.qcnet.com.