CARDIOVASCULAR JOURNAL OF AFRICA • Vol 23, No 10, November 2012
AFRICA
551
Marshall and Spiegelhalter
11
and Greenwood
18
).
The main interest of this work was not to find the best model
for hospital profiling, but to investigate whether or not the
methods agree. In order to inform which method gives a better
fit would require other model-checks statistics, such as posterior
predictive checks.
Conclusion
The main overall finding from our example is that the choice
of ways to classify a hospital is less critical than the statistical
method used. We suggest profiling hospitals using a hierarchical
model and RAMR with an appropriate threshold, which seems
to offer more reliable results. However, these methods warrant
further investigation, possibly of simulated data sets in which the
impact of underlying assumptions (and derivation thereof) may
be evaluated. There is a need for robust systems of ‘regulation’
or ‘performance monitoring’, which, with more rigorous work,
we hope to achieve in the future.
We thank Dr JS Birkhead, clinical director of MINAP, the National Audit of
Myocardial Infarction, National Institute for Clinical Outcomes Research,
and the Heart Hospital, London, for providing the extract from MINAP. We
acknowledge all hospitals in England and Wales for their contribution of data
to MINAP. We also thank Darren Greenwood for helpful comments.
References
1.
Department of Health. The new NHS: modern, dependable. The
Stationary Office: London, 1997 (Cm 3807).
2.
Baker R. Harold Shipman’s clinical practice, 1974–1998. The Stationary
Office: London, 2001.
3.
Kennedy Inquiry. Report of the public inquiry into children’s heart
surgery at the Bristol Royal Infirmary. The Stationary Office: London,
2001.
4.
Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and
misuse of process and outcome data in managing performance of
acute medical care: avoiding institutional stigma.
Lancet
2004;
363
:
1147–1154.
5.
Gale CP, Roberts AP, Batin PD, Hall AS. Funnel plots, performance
variation and the Myocardial Infarction National Audit Project 2003–
2004.
BMC Cardiovasc Disord
2006;
6
: 34.
6.
Birkhead JS, Walker L, Pearson M, Weston C, Cunningham AD,
Rickards AF. Improving care for patients with acute coronary
syndromes: initial results from the National Audit of Myocardial
Infarction Project (MINAP).
Heart
2004;
90
: 1004–1009.
7.
Bradley EH, Herrin J, Elbel B, McNamara BL, Magid DJ, Nallamothu
BK,
et al
.
Hospital quality for acute myocardial infarction: correlation
among process measures and relationship with short-term mortality.
J
Am Med Ass
2006;
296
: 72–78.
8.
Stukenborg GJ, Wagner DP, Harrell FE, Oliver MN, Heim SW, PriceAL,
et al
.
Which hospitals have significantly better or worse than expected
mortality rates for acute myocardial infarction patients? Improved risk
adjustment with present-at-admission diagnoses.
Circulation
2007;
116
; 2960–2968.
9.
Grant SW, Grayson AD, Jackson M, Au J, Fabri BM, Grotte G,
et al
.
Does the choice of risk adjustment model influence the outcome of
surgeon specific mortality analysis: a retrospective analysis of 14,637
patients under 31 surgeons?
Heart
2007:
doi:10.1136/hrt.2006.110478.
10.
Normand SL, Glickman ME, Gatsonis CA. Statistical methods for
profiling providers of medical care: issues and applications.
J Am Statist
Ass
1997;
92
: 803–814.
11.
Marshall EC, Spiegelhalter DJ. Institutional performance. In: Leyland
AH, Goldstein H (eds).
Multilevel Modelling of Health Statistics
.
Chichester: Wiley 2001: 127–142.
12.
Austin C. A comparison of Bayesian methods for profiling hospital
performance.
Medical Decision Making
2002: 163–172.
13.
Thomas N, Longford N, Rolph J. Empirical bayes methods for estimat-
ing hospital-specific mortality rates.
Statist Med
1994;
13
: 889–903.
14.
Marshall C, Best N, Bottle A, Aylin P. Statistical issues in the prospec-
tive monitoring of health outcomes across multiple units.
J R Statist Soc
Am
2004;
167
: 541–559.
15.
Ohlssen DI, Sharples LD, Spiegelhalter DJ. A hierarchical modelling
framework for identifying unusual performance in health care provid-
ers.
J R Statist Soc Am
2007;
170
: 865–890.
16.
DeLong ER, Peterson ED, DeLong DM, Muhlbaier LH, Hackett S,
Mark DB. Comparing risk-adjustment methods for provider profiling.
Statist Med
1997;
16
: 2645–2664.
17.
Leyland AH, Boddy FA. League tables and acute myocardial infarca-
tion.
Lancet
1998;
351
: 555–558.
18.
Greenwood D. Reliability of journal impact factor rankings.
BMC Med
Res Methodol
2007;
7
: 48.
19.
Birkhead JS, Walker L. National audit of myocardial infarction
(
MINAP): a project in evolution.
Hosp Med
2004;
65
: 452–453.
20.
Walker L, Birkhead J, Weston C, Pearson J, Quinn T. Myocardial
Infarction National Audit Project (MINAP): How the NHS manages
heart attacks. Royal College of Physicians. London, 2007.
21.
Bland JM, Altman DG. Statistical methods for assessing agreement
between two methods of clinical measurement.
Lancet
1986;
327
:
307–310.
22.
Fleiss J.
Statistical Methods for Rates and Proportions.
New York:
Wiley, 1981.
23.
Spiegelhalter DJ, Thomas A, Best NG, Lunn D.
BUGS: Bayesian
Inference Using Gibbs Sampling
,
Version 1.4.1, 2004. Medical
Research Council Biostatistics Unit, Cambridge University.
24.
Gelman A, Rubin DB. Inference from iterative simulations using multi-
ple sequences.
Stat Sci
1992;
7
: 457–472.
25.
Gale CP, Manda SOM, Batin PD, Weston CF, Birkhead JS, Hall
AS. Predictors of in-hospital mortality for patients admitted with
ST-elevation myocardial infarction: a real-world study using the
Myocardial Infarction National Audit Project (MINAP) database.
Heart
2007;
doi:10.1136/hrt.2007.127068.
26.
Gale CP, Manda SOM, Weston CF, Birkhead JS, Batin PD, Hall
AS. Evaluation of risk scores for risk stratification of acute coro-
nary syndromes in the Myocardial Infarction National Audit Project
(
MINAP) Database.
Heart
2008: 8.
27.
Yan AT, Yan RT, Tan M, Casanova A, Labinaz M, Sridhar K,
et al
.
Risk
scores for risk stratification in acute coronary syndromes: useful but
simpler is not necessarily better.
Eur Heart J
2007;
28
: 1072–1078.
28.
Resengren A, Wallentine L, Simoons M, Gitt AK, Behar S, Battler A,
Hasdai D. Age, clinical presentation, and outcome of acute coronary
syndromes in Euroheart acute coronary syndrome survey.
Eur Heart J
2006;
27
: 789–795.
29.
Normand S-LT, Shahian DM. Statistical and clinical aspects of hospital
outcomes profiling.
Stat Sci
2007;
22
: 206–226.
30.
Gelman A. Prior distributions for variance parameters in hierarchical
models.
Bayesian Anal
2006;
1
: 515–533.
31.
Mohammed MA, Stevens A. The value of administrative databases.
Br
Med J
2007;
334
: 1014–1015.
32.
Granger CB, Goldberg RJ, Dabbous O, Pieper KS, Eagle KA, Cannon
CP,
et al.
Predictors of hospital mortality in the global registry of acute
coronary events.
Arch Intern Med
2003;
163
: 2345–2353.
33.
Gale CP, Manda SOM, Greenwood, West RM, Hall AS. Characterising
hospital performance for acute coronary syndromes using the
Myocardial Infarction National Audit Project (MINAP) database: Data
incompleteness, multiple imputation and development of performance
indicators. Research Funded by the British Heart Foundation.
34.
Manda SOM. A Bayesian ordinal model for heterogeneity in a multi-
centre myocardial infarction clinical trial.
Statist Med
2002;
21
:
2011–3122.
35.
Ohlssen D. Bayesian methods for institutional comparisons and perfor-
mance monitoring. Available at <
/ festi-
val/programme/bay/ohlssen.pdf >, Accessed July 30, 2007.
36.
Luft HS, Brown BW. Calculating the probability of rare events: why
settle for an approximation?
Hlth Services Res
1993;
28
: 419–439.
