About the French hospitals rankings

Transcription

About the French hospitals rankings: a MCDA point of
view
Brice Mayag
University Paris Dauphine
LAMSADE
FRANCE
DA2PL 2014
Brice Mayag (LAMSADE)
French Hospital rankings
DA2PL 2014
1 / 24
About French Hospital rankings
Motivations & Aim
1
J.-C. Billaut, D. Bouyssou, and P. Vincke. Should you believe in the shanghai
ranking? - an mcdm view [2007].
2
In France, the subventions given by the government to hospitals is based on
their activity (Tarification à l’activité: T2A)
3
Several government agencies try to evaluate the “performance” of French
hospitals (HAS; ANAP; IGAS, . . . ).
4
In this case: No automatic decision, but they have preferences during the
evaluation process
5
Aim: To convince the MCDA and Preference Learning communities to
investigate and to contribute together in the development of methodologies
dedicated to hospital ranking.
DA2PL 2014
2 / 24
Hospital rankings are published every year
DA2PL 2014
3 / 24
Hospital rankings are published every year
Published last week.
DA2PL 2014
4 / 24
About French Hospital rankings: the context
They manipulate data coming from some official databases like PMSI
The databases contain around eighty indicators which are likely to be filled
each year by all the hospitals.
There are approximately 1600 hospitals classified as public, nonprofit private
and commercial private.
All the newspapers propose a ranking per surgery specialty, for instance a
ranking of weight loss surgery.
DA2PL 2014
5 / 24
About French Hospital rankings: weight loss surgery
Weight loss surgery
Bariatric surgery (weight loss surgery) includes a variety of procedures
performed on people who are obese.
Weight loss is achieved by reducing the size of the stomach with a gastric
band or through removal of a portion of the stomach
or by resecting and re-routing the small intestines to a small stomach pouch
(gastric bypass surgery)
DA2PL 2014
6 / 24
About French Hospital rankings: weight loss surgery
Some indicators used
(CR2 ) Activity : number of procedures performed during one year (to be
maximized).
(CR3 ) Average Length Of Stay (ALOS): a mean calculated by dividing the
sum of inpatient days by the number of patients admissions with the same
diagnosis-related group classification (to be minimized).
(CR4 ) Notoriety : Its corresponds to the reputation and attractiveness of the
hospital (to be maximized).
(CR8 ) Technicality : measures the ratio of procedures performed with an
efficient technology compared to the same procedures performed with
obsolete technology (to be maximized).
DA2PL 2014
7 / 24
Ranking given in 2013 by “Le Point”
20 best hospitals in weight loss surgery evaluated on Activity (CR2 ), (Notoriety)
(CR4 ) ; ALOS (CR3 ) and Technicality: (CR8 ).
Hospitals
CR2
CR4
CR3
CR8
FP
Bichat
Nice
Nancy
Ambroise-Paré
Colmar
Caen
Strasbourg
Georges-Pompidou
Lille
Antibes
Orleans
Rouen
Jean-Verdier
Conception
Louis-Mourier
Poissy/St Germain
Montpellier
Toulouse
Amiens
Laon
372
253
208
140
165
167
289
394
247
156
167
237
174
332
166
192
297
181
170
242
80
19
60
85
14
47
25
80
18
13
35
29
40
19
51
34
25
73
28
23
7.8
8.2
8
6.5
3.8
6.7
6.3
5.5
4.8
5.5
6.7
5.1
9.7
3.8
5.3
4.1
5.6
4.6
3.8
1.4
94
95
90
96
99
96
82
56
63
96
86
48
82
24
86
60
33
50
10
0
17.84
17.59
17.37
17.23
17.20
17.14
17.13
17.06
17.02
16.75
16.66
16.55
16.45
16.44
16.36
16.30
16.24
15.94
15.63
15.54
DA2PL 2014
8 / 24
Main criticisms
No methodology is given:
How indicators are normalized?
Which aggregation function is used?
The chosen indicators are relevant?
How weights are given?
No sensitivity analysis
How to take uncertainty?
The process is not transparent
...
=⇒ Unreliable information
The rankings are made by newspapers =⇒ Are they the decision makers?
They are called “puff piece”: a newspaper article using exaggerated praise
that often ignores opposing viewpoints or evidence to the contrary.
Everybody agrees on the fact that they are not made seriously
DA2PL 2014
9 / 24
Some insufficiencies mentionned by “Le Nouvel Observateur” in their website:
=⇒ In their ranking: Eleventh position ∼ Thirtieth position; Tenth position ∼
Fiftieth position,. . . ;
DA2PL 2014
10 / 24
But some hospitals promote these rankings
DA2PL 2014
11 / 24
Some remarks
More generally, hospital ranking problem is a problem of performance/quality
evaluation (global hospital or per specialty)
=⇒ A complex problem
French government is interested to solve it (ANAP, Scope santé website . . . )
In USA, the newspaper USNEWS establishes the rankings in collaboration
with the Research Triangle Institute, a scientific organism. Their
methodology is free available (129 pages).
DA2PL 2014
12 / 24
A MCDA point of view: Good practices
Objective: We have to Start with a a number of crucial questions when trying to
build an evaluation (ranking) model in MCDA:
1
What is the definition of objects to be evaluated?
2
What is the purpose of the model? Who will use it?
3
How to structure objectives?
4
How to achieve a “consistent family of criteria”?
5
How to take uncertainty, imprecision, and inaccurate definition into account?
All the French hospital ranking fail this last point.
DA2PL 2014
13 / 24
What is the suitable MCDA method?
The considered indicators are independent? (The linear models)
Does it exist any interaction between indicators? (The non linear models)
Hospital
Hospital
Hospital
Hospital
1
2
3
4
Notoriety
35
37
35
37
ALOS
80
80
40
40
Technicality
90
89
90
89
Hospital 1 is strictly preferred to the Hospital 2
⇒ if ALOS is “weak”, it is preferable to have an hospital with good evaluation
in Technicality
Hospital 4 is strictly preferred to Hospital 3
⇒ If ALOS is “good”, an hospital with good evaluation in Notoriety is
preferred.
DA2PL 2014
14 / 24
A question . . .
Does it exist a MCDA method which matches with the ranking given by “Le
Point”?
DA2PL 2014
15 / 24
An outranking method: ELECTRE TRI
A = {a1 ; a2 ; . . . ; am } a set of m alternatives;
N = {1; 2; . . . ; n} a set of n criteria;
C = {C1 (the worst); C2 ; . . . ; Ct (the best)} a set of ordered categories.
B = {b1 ; . . . ; bt−1 } a set of profiles that separate consecutive categories
Ci is limited by bi (the upper limit) and bi−1 (lower limit), i 6∈ {1, t}.
ELECTRE TRI assigns alternatives to categories by using the concept of
outranking relation S on A × B.
DA2PL 2014
16 / 24
ai outranks bh (ai S bh ) ⇔
n
X
kj cj (ai , bh ) ≥ λ.
1
Where
cj (ai , bh ) =
1 if ai %j bh
.
0 otherwise
The relation ai %j bh means that the value of ai on the criterion j is at least
as good as the value of bh on the same criterion j.
n
X
kj is the importance (weight) of criterion j such that
kj = 1.
1
λ is the cutting level i.e. a threshold that indicates whether the credibility is
significant or not. This parameter is often taken between 0.5 and 1.
DA2PL 2014
17 / 24
ELECTRE TRI assigns the alternative ai to the highest category Ch such that ai
outranks bh−1 i.e.
For h = 2, . . . , t − 1,

ai belongs to C1 ⇔ not(ai S b1 )





ai belongs to Ch ⇔ ai S bh−1 and not(ai S bh ),





ai belongs to Ct ⇔ ai S bt−1
DA2PL 2014
18 / 24
ELECTRE TRI applied to “Le Point”’s data
DA2PL 2014
19 / 24
ELECTRE TRI applied to “Le Point”’s data
5 hospitals incorrectly classified
Activity (CR2 ), (Notoriety) (CR4 ) ; ALOS (CR3 )
and Technicality: (CR8 )
[16.5; 18]: The class VeryGood
[15; 16.49]: The class Good
Hospitals
CR2
CR4
CR3
CR8
Bichat
Nice
Nancy
Ambroise-Paré
Colmar
Caen
Strasbourg
Georges-Pompidou
Lille
Antibes
Orleans
Rouen
Jean-Verdier
Conception
Louis-Mourier
Poissy/St Germain
Montpellier
Toulouse
Amiens
Laon
372
253
208
140
165
167
289
394
247
156
167
237
174
332
166
192
297
181
170
242
80
19
60
85
14
47
25
80
18
13
35
29
40
19
51
34
25
73
28
23
7.8
8.2
8
6.5
3.8
6.7
6.3
5.5
4.8
5.5
6.7
5.1
9.7
3.8
5.3
4.1
5.6
4.6
3.8
1.4
94
95
90
96
99
96
82
56
63
96
86
48
82
24
86
60
33
50
10
0
FP
17.84
17.59
17.37
17.23
17.20
17.14
17.13
17.06
17.02
16.75
16.66
16.55
16.45
16.44
16.36
16.30
16.24
15.94
15.63
15.54
DA2PL 2014
20 / 24
Another classification approach: Decision tree
Figure: A decision tree algorithm applied to “Le Point” data
DA2PL 2014
21 / 24
Another classification approach: Decision tree
Hospitals
CR2
CR4
CR3
CR8
Bichat
Nice
Nancy
Ambroise-Paré
Colmar
Caen
Strasbourg
Georges-Pompidou
Lille
Antibes
Orleans
Rouen
Jean-Verdier
Conception
Louis-Mourier
Poissy/St Germain
Montpellier
Toulouse
Amiens
Laon
372
253
208
140
165
167
289
394
247
156
167
237
174
332
166
192
297
181
170
242
80
19
60
85
14
47
25
80
18
13
35
29
40
19
51
34
25
73
28
23
7.8
8.2
8
6.5
3.8
6.7
6.3
5.5
4.8
5.5
6.7
5.1
9.7
3.8
5.3
4.1
5.6
4.6
3.8
1.4
94
95
90
96
99
96
82
56
63
96
86
48
82
24
86
60
33
50
10
0
FP
17.84
17.59
17.37
17.23
17.20
17.14
17.13
17.06
17.02
16.75
16.66
16.55
16.45
16.44
16.36
16.30
16.24
15.94
15.63
15.54
[16.5; 18]: The class VeryGood
[15; 16.49]: The class Good
8 hospitals incorrectly classified
DA2PL 2014
22 / 24
Hospital rankings: a new challenge to use simultaneaously
MCDA and preference learning?
How to help people to construct their rankings properly?
How to elaborate the MCDA decision model?
By applying good practices
There is a Decision Maker (ANAP, HAS, . . . )
Preferences exists
How to threat missing data?
How to choose relevant criteria (indicators),?
E.g. By using machine learning (datamining) algorithms like feature selection
algorithms.
Elaborate training examples (benchmark) from the existing free available
databases (PMSI, Hospidiag,. . . ) in order to
Define a predictive model?
Validate the approach?
DA2PL 2014
23 / 24
Hospital rankings: a new challenge to use simultaneaously
MCDA and preference learning?
If so then we will have:
From Multiple Criteria Decision Aid to Preference Learning
And
From Preference Learning to Multiple Criteria Decision Aid
DA2PL 2014
24 / 24

About the French hospitals rankings

Transcription

Similar documents

Personality Info and Rankings

case study shleppers

the presentation here. - Eat Smart, Move More South

Georgetown Prep Boys Lead Season

FAST-TRAK™ - Stock-Trak

Short - William R. Brice, Ph.D

SUWS #EUCup - Waterman League

Syed Ferhat Anwar, PhD Professor Professor Institute of Business

1 Guidelines for Stakeholders of Academic Rankings

Personality Info and Rankings

Table Layout Guide - Main Events Party Rental

Anchoring effects in world university rankings: exploring biases in