CARDIOVASCULAR JOURNAL OF AFRICA • Vol 23, No 10, November 2012
AFRICA
547
outliers by chance alone.
12
Estimated performances are stabilised
and shrunk towards the population average; the degree of
shrinkage being larger for small hospitals than for large hospitals.
Bayesian methods provide complete probabilistic information
in determining the probability that a hospital-specific risk-
adjusted rate exceeds a specified threshold.
11
Furthermore, a
researcher is able to place credible intervals on the derived ranks
to quantify the uncertainty associated with institutional ranking
before relative performance can be assessed.
11,18
In the current study, rather than calibrate the methods, we
concentrated on comparing the performance of four methods
and assessing how well they agreed with one another, using
Marshall and Spiegelhalter,
11
and Austin’s approaches.
12
The
methods were applied to data on short-term mortality in acute
coronary syndrome (ACS) patients. The data are part of the
Myocardial Infarction National Audit Project (MINAP), which
currently reports percentage attainment of standards on five
clinical process variables, namely door-to-needle and call-to-
needle thrombolysis times, and the use of aspirin, beta-blockers
and statins post-acute myocardial infarction (AMI).
19,20
A use of
the MINAP data for hospital comparison was presented in Gale
et al
.,
5
using funnel plots on the same five process variables.
To the best of our knowledge, the present study is the first
to use an outcome measure and to control for any variation,
specifically for case mix, with contemporary data on ACS. We
did not duplicate MINAP tabulations or the Gale
et al
.
5
funnel
plot methodology. Instead, we determined (a) whether or not
a hospital’s risk-adjusted mortality rate exceeded a specified
threshold, and (b) the hospital’s rank, based on its risk-adjusted
mortality rate using two statistical models: fixed and hierarchical
models, on the number of deaths among patients admitted by
the hospital. While this article does not add sufficient new
methodological questions on profiling methods, the topic of
healthcare performance is timely, important and interesting
within the medical and health services domain.
Methods
MINAP was established in 1998. It is reported to be the largest and
most comprehensive clinical database of ACS care in the world
and is a valuable resource for monitoring coronary heart disease
audit standards for patients presenting with AMI in England and
Wales.
20
All hospitals in England and Wales that treat patients
with acute AMI submit data to MINAP. The project collects
information on the quality of care and outcome of patients. Each
patient entry offers details of the patient’s journey, including the
method and timing of admission, in-patient investigations, results
and treatment, and, if applicable, dates of death from linkage to
the Office of National Statistics, United Kingdom.
Prospective data are collected locally, electronically encrypted
and transferred to a central database. The database may be used for
identifying performance indicators to identify examples of good
practice.With such data, it is feasible to evaluate contemporary care
practices consistent with national guidelines for the management
of ACS, investigate whether hospital performance varies between
hospitals, identify hospital characteristics predictive of adherence
to guidelines, and assess whether adherence to guidelines is
associated with mortality rates.
7
We examined all 187 069ACS events entered into the MINAP
database from 1 January 2003 to 31 March 2005. We selected
first (index) admissions reported to MINAP and therefore
excluded re-admissions. We then analysed all patients who were
aged between 18 and 100 years, who had an admission systolic
blood pressure between 49 and 250 mmHg, and an admission
heart rate between 20 and 200 beats/min.
In total there were 134 hospitals, six of which were discarded
from the analyses because of sparse data, i.e. not sufficiently
varied (two with one admission, three with fewer than five
deaths, one with excessive missing codes on death status). For
the remaining 100 686 patients, the overall in-hospital mortality
rate was 8.1%, the total mortality rate was 17.8%, and the 30-day
mortality rate was 10.2%. Hospital-specific 30-day mortality
rates ranged from 5 to 21%, with a median rate of 8.3%.
Statistical models
We assumed that
O
i
is the observed number of 30-day deaths
in patients admitted to hospital
i
(
i
=
1, …, 128)
and
E
i
is the
expected number of deaths, given the case mix of its patients. The
number of deaths in the period 1 January 2003 to 31 March 2005
can be assumed to follow a Poisson distribution with unknown
mean
λ
i
.
Therefore
O
i
~
Poisson (
λ
i
)
taking log
λ
i
=
log
E
i
+
θ
i
where log
E
i
is an
offset
that
adjusts for the patient effects and
θ
i
is a residual representing hospital-specific effect of interest.
The expected number
E
i
is obtained from a logistic regression
on the pooled data, adjusting for relevant risk factors, to determine
each patient’s predicted probability of 30-day mortality. These
probabilities are then summed within a hospital to give the
expected number of deaths at that hospital, given its case mix.
The hospital-specific effect
θ
i
is the log-relative risk or logarithm
of the hospital’s standardised mortality ratio (log SMR).
Other than to compare hospitals using their SMRs, we used the
hospital risk-adjusted 30-day mortality rate (RAMR),
7
defined as
RAMR
=
µ
30
exp (
θ
i
),
where
µ
30
(
=
10.2%)
is the overall 30-day
mortality rate. The RAMR can be thought of as the estimated
30-
day mortality rate for a hospital admitting a population of
patients identical to the overall case mix.
11
We adopted Bayesian
methods in estimating a hospital-specific random effect
θ
i
to
obtain its specific risk-adjusted mortality rate using 10.2
×
exp
(
θ
i
),
which we used in this study for institutional profiling.
In order to estimate the hospital-specific effect, we firstly
assumed that it has a prior normal distribution with mean 0
and variance 1 000. This is the fixed-effects model, and the
prior distribution implies that the hospital-specific standardised
mortality rate has a prior mean of 1.
Secondly, as an alternative, we considered a Bayesian random-
effects model, which, by using hierarchical modelling, pools data
across hospitals. This approach produces more reliable estimates
of hospital performance, in that genuinely low or high hospital
outliers are identified. It reduces the chance of a small hospital
being classified as an outlier by chance alone.
11
Under the latter modelling approach, it was assumed that
the hospital-specific random effects
θ
i
were drawn from a
normal distribution with an unknown mean
µ
0
and variance
σ
0
2
.
Therefore,
θ
i
~
Normal (
µ
0
,
σ
0
2
),
where the hyper-parameters
µ
0
and
σ
0
2
were the underlying overall log-standardised mortality
ratio and between-hospital variance, respectively.
In order to complete the Bayesian implementation of the
model, we also needed to specify prior probability distributions
for the hyper-parameters
µ
0
and
σ
0
2
for the hospital-specific
random effects,
θ
i
distribution. The hyper-mean,
µ
0
was assigned
