|
Systematic
analysis is a way to evaluate the design, analysis, and
conclusions of several studies on the same research topic.
Such analysis integrates and weighs the studies according
to several factors:
-
The degree to which the aims of the study match the aim
of the systemic analysis;
-
Similarity of measures;
-
Quality of study design as measured by its place in a
hierarchy of designs such as that from the Preventive
Services Task Force (Lawrence
and Mickalide, 1987);
-
The variation associated with the estimates;
-
The p values associated with the tests of hypotheses;
and
-
The size of the study.
Quality
improvement projects that involve several sites working
on the same topic lend themselves to systematic analysis.
The aim of systematic analysis for quality improvement is
to determine the extent to which the redesigned system produces
similar results over a variety of sites, working groups,
or organizations. A quality improvement approach to systematic
analysis integrates and weighs the projects according to
the following factors:
-
The degree to which the aims of the projects address similar
improvement goals;
-
Similarity of measures;
-
The clarity of the description of the redesigned system;
-
The description of the learning that took place during
the evolution to the new system;
-
The extent to which the organizations can produce a time
series of the key, measures annotated with the sequence
of the changes, and other significant events;
-
The length of the time series after the changes have been
made; and
-
The stability of the time series before and after the
changes have been made.
The
systematic analysis will be concerned with absolute performance
and with patterns of performance over time including: the
initial performance, the rate of transition to the improved
performance, and the level of stabilized performance. Consider
the work on pain management at the Tucson VA Hospital. The
figure below (discussed previously) shows results from two
inpatient units.
Figure
6.1 Percent of In-Patients with Severe Pain
(Veterans Administration Hospital - Tucson, AZ)
|
|---|
 |
|
Source:
Southern Arizon VA Health Care System, Tucson, AZ. Reprinted
with permission from Anne Gooden, RN.
The
Tucson VA Hospital then implemented the same changes in
an outpatient oncology clinic and produced the following
results:
Figure
6.2 Percent of Oncology Patients with Severe Pain
(Veterans Administration Hospital - Tucson, AZ)
|
|---|
 |
|
Source:
Southern Arizon VA Health Care System, Tucson, AZ. Reprinted
with permission from Anne Gooden, RN.
A
systematic analysis might begin by comparing the results
in the two inpatient units and the oncology clinic. The
inpatient units had a much greater percentage of patients
in severe pain at the start compared to the oncology unit.
However, after an initial quick drop in the percentage,
the pattern of improvement and stabilization between the
two units is similar. One should not make too much of patterns
from only two sites or the stability of the system after
four months. However, an initial hypothesis is that the
new pain management system will reduce the incidence of
severe pain among patients to between five and eight percent.
Reducing the incidence further will require additional system
changes.
As
more sites are added the generalizability of the pain management
system and its performance can be further analyzed. Such
an analysis may raise other questions such as: "Does performance
vary systematically between locations or by the source of
the pain?" Continuing the time series will also help answer
whether the improved performance can be sustained. To the
degree that they are generalizable, such results will help
build a willingness for change in others within the social
system (Rogers, 1995). Change
can be encouraged in many ways including peer pressure,
published findings, leadership initiative, or regulations.
The
factors suggested above for a quality improvement approach
to systematic analysis can be used to evaluate the rigor
of any quality improvement project, not just those that
will be included in a systematic analysis. These factors
are consistent with the elements essential to reliable learning
through quality improvement described in this paper. With
a rewording of the first two factors to focus on an individual
improvement project, the criteria for a rigorous quality
improvement project become:
-
The aim of the project is focused on performance improvement;
-
A balanced set of performance measures is included that
is sufficiently related to the aim;
-
The description of the redesigned system is clear;
-
The learning that took place during the evolution to the
new system is documented;
-
Time series of the key measures annotated with the sequence
of the changes and other significant events are provided;
and
- The
time series continue well after the changes have been
made to ensure sustainability.
Researchers
submitting proposals for projects associated with quality
improvement initiatives can use this list as an outline
for the methods section of their proposals.
|