| |
Although there is
ample research to show that randomized control trials are the
strongest type of research design to measure the efficacy of a
treatment modality, what about the quality of the randomized control
trial itself? Do higher quality studies yield different results
than lower quality studies? What study features seem to be most
important?
Multiple research studies
have recently begun to address these pertinent questions, and,
although the answers are far from certain, specific features of
RCTs that investigate treatments in humans have been shown to
be especially valuable.
Common rating schemas
have been established by both the Cochrane Collaboration www.cochrane.org
and by CONSORT (consolidated standards
of reporting trials).
Altman and others (Altman
et al., 2001), for example, developed a checklist of 22 items
to include in published reports of RCTs, and this is listed in
Table
6.1. The National Library of Medicine, the Journal of the
American Medical Association, the International Committee
of Medical Journal Editors, and over 20 other biomedical journals
have all endorsed the CONSORT criteria.
Out of these 22 items,
not surprisingly, larger trials (group sizes greater than
1000) have been shown to yield results that are less variable,
more conservative, and closer to the overall mean effect of all
trials averaged together than smaller trials (Kjaergard
et al., 2001a; Juni
et al., 2001c). Very small trials (less than ~30 subjects
per group) have been shown to do the opposite; these trials inflate
estimates of treatment effectiveness, yield widely varying results,
and are generally farther from the average of all treatment studies
(Kjaergard
et al., 2001b).
As a general rule,
RCTs that adhere to strict criteria are more likely to yield studies
that are closer to the average of all trials conducted, whether
they are large or small in size (Juni
et al., 2001a), although a recent paper was unable to replicate
this finding (Balk
et al., 2002).
An excellent series
of articles by Turk and colleagues discuss many usually neglected
factors that influence outcomes in chronic pain treatment studies
(Turk
and Rudy, 1990; Turk
and Rudy, 1991; Turk
et al., 1993). The issues raised include non-compliance with
study tasks and lack of agreement on outcome evaluation criteria,
which are especially critical when attempting RCT's involving
clinical conditions defined more by subjective symptom states
than by objective pathophysiologic markers.
|