[-0.2cm]Visualisation of big time series data

Transcription

Visualisation of
big time series
data
Rob J Hyndman
Visualisation of big time series data
1
Visualisation of
big time series
data
Rob J Hyndman
with Earo Wang, Nikolay Laptev
Yanfei Kang, Kate Smith-Miles
1
Visualisation of
big time series
data
Rob J Hyndman
1
Visualisation of
big time series
data
Rob J Hyndman
1
Visualisation of
big time series
data
Rob J Hyndman
1
Visualisation of
big time series
data
Rob J Hyndman
1
Visualisation of
big time series
data
Rob J Hyndman
1
Outline
1 The problem
2 Australian tourism demand
3 M3 competition data
4 Yahoo web traffic
5 What next?
The problem
2
0.0
0.2
0.4
0.6
0.8
1.0
How to plot lots of time series?
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
The problem
3
The problem
4
The problem
4
The problem
4
The problem
4
The problem
4
Key idea
Cognostics
Computer-produced diagnostics
(Tukey and Tukey, 1985).
Examples for time series
lag correlation
size and direction of trend
strength of seasonality
timing of peak seasonality
spectral entropy
John W Tukey
The problem
5
Key idea
Cognostics
lag correlation
spectral entropy
John W Tukey
The problem
5
Key idea
Cognostics
lag correlation
spectral entropy
John W Tukey
The problem
5
Key idea
Cognostics
lag correlation
spectral entropy
John W Tukey
The problem
5
Key idea
Cognostics
lag correlation
spectral entropy
John W Tukey
The problem
5
Key idea
Cognostics
lag correlation
spectral entropy
John W Tukey
Called “features” or “characteristics” in the
machine learning literature.
The problem
5
Outline
1 The problem
4 Yahoo web traffic
5 What next?
Australian tourism demand
6
7
Quarterly data on visitor night from
1998:Q1 – 2013:Q4
From: National Visitor Survey, based on
annual interviews of 120,000 Australians
aged 15+, collected by Tourism Research
Australia.
Split by 7 states, 27 zones and 76 regions
(a geographical hierarchy)
Also split by purpose of travel
Holiday
Visiting friends and relatives (VFR)
Business
Other
304 disaggregated series
7
BEEOth BEDOth
BEEBus BEDBus
BEGBus BEFBus
BEGOth BEFOth
BEEVis BEDVis
BEEHol BEDHol
BEGVis BEFVis
BEGHol BEFHol
BECOth BEBOth
BECBus BEBBus
BECVis BEBVis
BECHol BEBHol
BDFVis
BEAOth BDFOth
BEABus BDFBus
BEAVis
BEAHol BDFHol
BDEOth BDDOth
BDEBus BDDBus
BDEVis BDDVis
BDEHol BDDHol
BDCOth BDBOth
BDCBus BDBBus
BDCVis BDBVis
BDCHol BDBHol
BDAOth BCCOth
BDABus BCCBus
BDAVis BCCVis
BDAHol BCCHol
BCBOth BCAOth
BCBBus BCABus
BCBVis BCAVis
BCBHol BCAHol
BBAOth BACOth
BBABus BACBus
BBAVis BACVis
BBAHol BACHol
BAAVis
BABOth BAAOth
BABBus BAABus
BABVis
BABHol BAAHol
Domestic tourism demand: Victoria
8
An STL decomposition
7.0
6.0
6.1
−0.4
0.0
remainder
5.8
trend
6.4
−0.5
0.5
seasonal
5.0
data
Tourism demand for holidays in Peninsula
Yt = S t + Tt + R t
St is periodic with mean 0
2000
2005
2010
time
9
Seasonal stacked bar chart
Place positive values above the origin while
negative values below the origin
Map the bar length to the magnitude
Encode quarters by colours
Qtr
0.5
Q1
Holiday
Seasonal Component
1.0
0.0
Q2
Q3
Q4
−0.5
−1.0
BAA BAB BAC BBA BCA BCBBCCBDA BDBBDCBDDBDE BDF BEA BEB BEC BED BEE BEF BEG
Regions
10
Seasonal stacked bar chart: VIC
11
11
1.0
0.5
0.0
−0.5
−1.0
1.0
0.5
0.0
−0.5
−1.0
1.0
0.5
0.0
−0.5
−1.0
1.0
0.5
0.0
−0.5
−1.0
Holiday
VFR
Qtr
Q1
Q2
Q3
Q4
Business
Seasonal Component
Other
BAABABBACBBABCABCBBCCBDABDBBDCBDDBDEBDFBEABEBBECBEDBEEBEFBEG
Regions
11
Trend analysis
Linearity: the long-term direction and
strength of trend.
Curvature: the “changing direction” of trend.
Estimate by regression:
Tt = β̂0 + β̂1 φ1 (t ) + βˆ2 φ2 (t ) + et
where φk (t ) is a kth-degree orthogonal
polynomial in time t.
To separate the linearity (β̂1 ) and curvature
(β̂2 ).
12
4
3
2
1
0
Holiday
4
3
2
1
0
VFR
4
3
2
1
0
Business
Trend Linearity
Trend analysis
Other
4
3
2
1
0
Direction
−
+
BAA BAB BAC BBA BCA BCBBCC BDA BDBBDCBDDBDE BDF BEA BEB BEC BED BEE BEF BEG
Regions
13
Trend analysis
13
BEEHol
BEFOth
BEEOth
BDEOth
BEBOth
BEABus
BEFBus
BDCOth
BACHol
BEBBus
BEAVis
BBAHol
BDEHol
BABOth
BAAVis
BAAHol
BDCHol
BBABus
BCBHol
BEGBus
BDDVis
BABVis
BDAVis
BEAOth
BDFHol
BEEBus
BAAOth
BACOth
BDAOth
BDEBus
BCBOth
BACBus
BEBVis
BACVis
BCAOth
BEFVis
BCBVis
BEDHol
BEGOth
BDBHol
BABBus
BEBHol
BDFBus
BECHol
BCAHol
BDBOth
BEAHol
BDCBus
BECVis
BDBVis
BCCHol
BBAVis
BABHol
BBAOth
BCCOth
BCBBus
BCCVis
BEGVis
BDDHol
BECOth
BDCVis
BAABus
BCCBus
BECBus
BCAVis
BDFVis
BEGHol
BDDOth
BEDOth
BEDVis
BDDBus
BDEVis
BEFHol
BEEVis
BDBBus
BDABus
BDAHol
BCABus
BDFOth
BEDBus
Corrgram of remainder
BEEHol
BEFOth
BEEOth
BDEOth
BEBOth
BEABus
BEFBus
BDCOth
BACHol
BEBBus
BEAVis
BBAHol
BDEHol
BABOth
BAAVis
BAAHol
BDCHol
BBABus
BCBHol
BEGBus
BDDVis
BABVis
BDAVis
BEAOth
BDFHol
BEEBus
BAAOth
BACOth
BDAOth
BDEBus
BCBOth
BACBus
BEBVis
BACVis
BCAOth
BEFVis
BCBVis
BEDHol
BEGOth
BDBHol
BABBus
BEBHol
BDFBus
BECHol
BCAHol
BDBOth
BEAHol
BDCBus
BECVis
BDBVis
BCCHol
BBAVis
BABHol
BBAOth
BCCOth
BCBBus
BCCVis
BEGVis
BDDHol
BECOth
BDCVis
BAABus
BCCBus
BECBus
BCAVis
BDFVis
BEGHol
BDDOth
BEDOth
BEDVis
BDDBus
BDEVis
BEFHol
BEEVis
BDBBus
BDABus
BDAHol
BCABus
BDFOth
BEDBus
1
0.8
0.6
0.4
0.2
0
−0.2
−0.4
−0.6
−0.8
−1
14
BEEHol
BEFOth
BEEOth
BDEOth
BEBOth
BEABus
BEFBus
BDCOth
BACHol
BEBBus
BEAVis
BBAHol
BDEHol
BABOth
BAAVis
BAAHol
BDCHol
BBABus
BCBHol
BEGBus
BDDVis
BABVis
BDAVis
BEAOth
BDFHol
BEEBus
BAAOth
BACOth
BDAOth
BDEBus
BCBOth
BACBus
BEBVis
BACVis
BCAOth
BEFVis
BCBVis
BEDHol
BEGOth
BDBHol
BABBus
BEBHol
BDFBus
BECHol
BCAHol
BDBOth
BEAHol
BDCBus
BECVis
BDBVis
BCCHol
BBAVis
BABHol
BBAOth
BCCOth
BCBBus
BCCVis
BEGVis
BDDHol
BECOth
BDCVis
BAABus
BCCBus
BECBus
BCAVis
BDFVis
BEGHol
BDDOth
BEDOth
BEDVis
BDDBus
BDEVis
BEFHol
BEEVis
BDBBus
BDABus
BDAHol
BCABus
BDFOth
BEDBus
BEEHol
BEFOth
BEEOth
BDEOth
BEBOth
BEABus
BEFBus
BDCOth
BACHol
BEBBus
BEAVis
BBAHol
BDEHol
BABOth
BAAVis
BAAHol
BDCHol
BBABus
BCBHol
BEGBus
BDDVis
BABVis
BDAVis
BEAOth
BDFHol
BEEBus
BAAOth
BACOth
BDAOth
BDEBus
BCBOth
BACBus
BEBVis
BACVis
BCAOth
BEFVis
BCBVis
BEDHol
BEGOth
BDBHol
BABBus
BEBHol
BDFBus
BECHol
BCAHol
BDBOth
BEAHol
BDCBus
BECVis
BDBVis
BCCHol
BBAVis
BABHol
BBAOth
BCCOth
BCBBus
BCCVis
BEGVis
BDDHol
BECOth
BDCVis
BAABus
BCCBus
BECBus
BCAVis
BDFVis
BEGHol
BDDOth
BEDOth
BEDVis
BDDBus
BDEVis
BEFHol
BEEVis
BDBBus
BDABus
BDAHol
BCABus
BDFOth
BEDBus
Compute the correlations among
the remainder components
1
0.8
0.6
Render both the sign and
magnitude using a colour mapping0.4
0.2
of two hues
Order variables according to the
first principal component of the
correlations.
0
−0.2
−0.4
−0.6
−0.8
−1
14
BEEHol
BCBHol
BACHol
BDEHol
BDBHol
BABHol
BAAHol
BBAHol
BEGHol
BEAHol
BCAHol
BDCHol
BCCHol
BDFHol
BEDHol
BECHol
BEFHol
BEBHol
BDDHol
BDAHol
BDAHol
1
BDDHol
0.8
BEBHol
BEFHol
0.6
BECHol
BEDHol
0.4
BDFHol
BCCHol
0.2
BDCHol
BCAHol
0
BEAHol
BEGHol
−0.2
BBAHol
BAAHol
−0.4
BABHol
BDBHol
−0.6
BDEHol
BACHol
−0.8
BCBHol
BEEHol
−1
14
FAABus
FBBBus
FCBOth
FBAVis
FCABus
FCBVis
FBAOth
FBABus
FBBOth
FCAOth
FAAOth
FCBBus
FAAVis
FBBVis
FCAVis
FCBHol
FAAHol
FBAHol
FBBHol
FCAHol
Corrgram of remainder: TAS
FCAHol
1
FBBHol
0.8
FBAHol
FAAHol
0.6
FCBHol
FCAVis
0.4
FBBVis
FAAVis
0.2
FCBBus
FAAOth
0
FCAOth
FBBOth
−0.2
FBABus
FBAOth
−0.4
FCBVis
FCABus
−0.6
FBAVis
FCBOth
−0.8
FBBBus
FAABus
−1
15
Outline
1 The problem
4 Yahoo web traffic
5 What next?
M3 competition data
16
M3 forecasting competition
3003 series
All data from business, demography, finance and
economics.
Series length between 14 and 126.
Either non-seasonal, monthly or quarterly.
All time series positive.
M3 competition data
17
M3 forecasting competition
●
M3 competition data
18
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Strength of seasonality: 1 −
Strength of trend: 1 −
Var(Rt )
Var(Yt −Tt )
Var(Rt )
Var(Yt −St )
Rπ
Spectral entropy: H = − −π fy (λ) log fy (λ)dλ,
where fy (λ) is spectral density of Yt .
Low values of H suggest a time series that is
easier to forecast (more signal).
Autocorrelations: r1 , r2 , r3 , . . .
Optimal Box-Cox transformation parameter λ
M3 competition data
19
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Var(Rt )
Var(Yt −Tt )
Var(Rt )
Var(Yt −St )
Rπ
M3 competition data
19
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Var(Rt )
Var(Yt −Tt )
Var(Rt )
Var(Yt −St )
Rπ
M3 competition data
19
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Var(Rt )
Var(Yt −Tt )
Var(Rt )
Var(Yt −St )
Rπ
M3 competition data
19
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Var(Rt )
Var(Yt −Tt )
Var(Rt )
Var(Yt −St )
Rπ
M3 competition data
19
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Var(Rt )
Var(Yt −Tt )
Var(Rt )
Var(Yt −St )
Rπ
M3 competition data
19
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Var(Rt )
Var(Yt −Tt )
Var(Rt )
Var(Yt −St )
Rπ
M3 competition data
19
Candidate features
3000
1000
N0001
5000
Seasonality
1978
1980
1982
1984
1986
1988
N2602
0
10000
20000
1976
1982
1984
1986
6000 10000
1980
2000
N1906
1978
1984
1986
1988
1990
M3 competition data
1992
20
Candidate features
4000
2000
N0125
6000
Trend
1980
1982
1984
1986
1988
5000
1978
3000
N1978
1976
1984
1986
1988
1990
1992
1000
N0546
4000
7000
1982
1975
1980
1985
M3 competition data
20
Candidate features
N0001
5800
6000
6200
ACF1
1988
1989
1990
N2658
3000
5000
7000
1987
1988
1989
1990
1991
8000
7000
N2409
9000
1987
1984
1986
1988
1990
M3 competition data
1992
20
Candidate features
4000
2500
N2487
5500
Spectral entropy
1968
1970
1972
1974
4500
1966
3000
N0794
1964
1988
1990
1992
2000
N0121
2400
2800
1986
1976
1978
1980
1982
1984
M3 competition data
1986
1988
20
Candidate features
4000
2000
N0002
6000
Box Cox
1978
1980
1982
1984
1986
1988
7000
5000
N0468
9000
1976
1970
1975
1980
1985
6000
1965
2000
N0354
1960
1960
1965
1970
1975
1980
M3 competition data
1985
20
Candidate features
0.4
0.8
2
6
10
0.0
0.4
0.8
0.9
0.0
0.0 0.6
0.5
SpecEntr
0.0 0.6
Trend
8
Season
0.6
2
Freq
0.0 0.6
−0.4
ACF
Lambda
0.5
0.7
0.9
0.0
0.4
0.8
−0.4
0.2
0.8
M3 competition data
21
Dimension reduction for time series
●
M3 competition data
22
0.4
0.8
2
6
10
0.0
0.4
0.8
0.9
0.0
0.0 0.6
0.5
SpecEntr
0.0 0.6
Trend
Season
8
Feature
calculation
0.6
2
Freq
0.0 0.6
−0.4
ACF
Lambda
0.5
0.7
0.9
0.0
0.4
0.8
−0.4
0.2
0.8
●
M3 competition data
22
0.4
0.8
2
6
10
0.0
0.4
0.8
0.9
0.0
0.0 0.6
0.5
SpecEntr
0.0 0.6
Trend
Season
8
Feature
calculation
0.6
2
Freq
0.0 0.6
−0.4
ACF
0.5
3
2
PC2
1
0
−1
−2
−3
0.7
0.9
0.0
0.4
Principal Lambda
component
decomposition
0.8
−0.4
0.2
0.8
●
● ●
●
●
●
●
●●
●●
●●
●● ●
●● ●
●● ●● ●● ●●
●● ●
●● ●
●
●● ●●●
● ●●
● ●●
●●
●
● ●
●
●
●●
●●
●
●●
● ●●
●●●●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
● ●●
●● ●
●
●
● ●
●●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●● ●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●●●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●●
● ●● ●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
● ●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●● ● ●
●
●
●
●
●
●● ●●
●
●●● ●
●
●●
● ●●
●
●
●●
●
●
●●●
●
● ●●
●● ●●
●●●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
● ●
●
●
●
●
●
● ●● ●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●● ●
●
●●
●
●
●
●
●●● ●●●●●
●●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●● ●
●
● ● ●●●●
●
● ● ● ●●●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●● ●●●●
●●
●
●
●
●
●
●●
●●●●
●● ●●
●
●●
●
●
●
●
●
●
●● ● ●
●
●●●●● ● ●
●
●
●
●
●●●●
●●●
●
●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●●●
●
●
●
●
●● ● ●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●●
●●
●
●● ●
●
●
●
●
●●
●
●
●
● ●● ● ● ●●
●●●
●
●
●●
●●
●● ●
●●●●●
●
●
●
●● ●●
●
●
●
●
●● ●●
●
●● ●
●
●●●●
●
●
●
●
●●
●
●●
●●●
●●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
● ●●
● ●
●
● ●
●
●●●
●
●
●
●●
●●
●
●
●●●
●●●
●●
●
●
●●●
●●
●●●●●
●●
●●
● ●● ● ● ●●● ● ● ● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
● ●
●● ● ● ●
● ●
●●
●●
●
●● ●
●
● ●● ●
● ●●
●
●
●
●
●
●
●● ●
●●
●● ●●
●
●
● ●
●
●●
●●
●
●●
●
●●
●
●●
●
●
● ●
● ●●●
●
●
●
●●●
●● ●
●
●●●●
●● ● ●●●●● ● ●
●●● ●●
● ●
● ●●
●● ●
●●
●●
●●
●
● ● ●●●
● ●●
●
●●
●
●●
●
●● ●
●●
●●
●● ●●
●● ●●
● ●●
●●●
●● ●
●●
● ●●
●
● ●●
●
●●●
●
●●
●●
●
● ●●
●
●●
●
●
●
●
●●
●
●
●
●
●
● ●● ●
●●
●
●●
●●
●● ●●
●●●
●
●
●
●●
●●●●
● ● ● ●●
●●●
●●
●
●●●
●
●
●
●●
● ●● ●
●
●●
●
●
●●
●●
●
●
●
●●
●
● ●●
●●
●●
●
●●
●
●
●●
●
●
●●●●
●
●●● ●
●
●
●●●
●
●
●●●
●●●
●
●
●
●●●●
●●●●●● ●
●
●●●●
● ●●
●
●
●
●
●●
●●●
●
●
●●
●●
●
●
●●
●
● ● ●● ●
●●●
●●
●
●
●
●
●
●
●● ●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●●
●
●
●●● ●
●●●
●
● ●●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●
●
●●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●●
● ●●●●● ●●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●● ●●
●●●
●
●
●
●
●
●
● ●●
●
●
●●●
●
●
●● ● ●
●
●
●
●●●●
●●●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
● ●
●●●
●
●
●
●●
●●
●
●
●
●
●● ● ●
●
●●
●
●●
●
●
●
●
● ●●
●
●
●
●●●
●
●●
●
●
●
●
●●●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●●
●●●●
● ●● ●●
●●●
●●●
●
●●●
● ●● ●
● ●● ● ●
●●●
●●
● ●●● ● ● ●
●
●●
●
●
● ●● ● ● ●●●
●
●
● ●●
●
●●●
●
●
●
●●●●●
●● ●
● ●●●●
●●
●
●
●
●
●●●
●●
●
● ●●
●
●●
●
●● ●
●●
●●
● ●● ●
●●
●
● ●●
● ●
● ●
●●
●● ●●
●●●● ●
●● ●●●●
●● ●
●
●
●
●
●
●
●
● ●● ● ●
●●
●
●● ●●●
●
● ● ●● ●●●●●●
●
●● ●
● ●● ●● ●
●
●● ●
●●●●
● ● ●
●●
●
●●
●●
●●●
●● ●●●●● ●
●
● ●
●●●●● ●
●●
●
●
●●
● ●●
●●
●
●
●
●●●●●●●
●
●●● ●●
●●
●
●● ●
●● ●●●
●
●
●
● ● ●
● ●
●
●●
●●
●●
●● ●
●
●
●
●
●●
●
●● ●●●●
●● ● ●●
●
●
●
●
●
● ●●● ●
●●
●● ● ●
●
●
●
●
●
●
●
−2
0
2
●
4
PC1
●
M3 competition data
22
Feature space of M3 data
First two PCs explain 68% of variation.
3
2
PC2
1
0
−1
−2
−3
●
● ●
●
●●
● ●●●
●● ●●●
●
●
●●
●● ●● ●● ●●
●● ●
●● ●
●
●● ●●●
● ●●
● ●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●● ●
●●●●● ●
●
●
●●●●
●
●
●
●
●●
●●
●
●
●
●●●●●
●
●●
●●
●●
●
●
●
●
●
●● ●●
●
●
●●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●●●
●
●
●●
●
●●
●●● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●● ●
●●
●
●●
●●
●●●●●● ●● ●●●
●
●●●●●
● ●● ●
●
●
●
●
●
●
●
●●
●
●
●●●● ●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●●
● ●● ●●
●
● ●●
●
●
● ●●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●● ● ●● ●
●●
●
● ●●
●
● ●
●
● ●
●● ●
●
●
●
●
● ●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●●●●
●
●●
●●
●
●
●
●
●
●
●●
●● ● ●
●
●●●
●
●
●
●●●●
●●●
●
●●●
●●
●
●●●●●●● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
● ●●
●●
●
● ●●● ●
●
●● ●
●
●●
●●
●● ●● ● ●●●●
●●●
●●
●
● ●●●
● ●●●● ● ● ● ●●
●●●●●●
●
●
●●
●●●
●
●
●
●
●
●
●●
●●
●
●●●●●
●
●●
●● ●●● ●●
●
●●●●●●●●●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●●
●●
●
●
●
●
●●
●
●● ● ● ●
●●●●
●●●
●●
●
●●
●
●● ●●
●
●
●
●●● ●●
●
●● ●
●
● ● ● ●●
●●●●
●
●
●
●●
●●
●
●●●●●●● ● ●●●
●●
●
●
●
●●●
●●
● ●●
● ●●●
●
●●●
●●●●
●●●●
●●
●
●●●●
●
●●●
●●
● ●● ● ● ●●● ● ● ● ●
●
●
●●
●●● ●●
●
●● ●● ●
●●●●●● ●●●
●
●
●●●● ●
●
● ●●●
● ●●●
●
●●
●●●
●
●●●
● ●
●
●● ●● ●●●●●
●●●●
●●
●
●●
●●
●
●
●
●
●● ●
●
●
● ● ● ●● ●
●
●
●● ●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●● ●
●●
●●●●●●
●
● ●
● ●●
● ●●●●● ● ●●●
●● ●●
●
●● ●
●
●
●
●
●
●
●● ●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●●
● ●● ● ●●● ●●
● ●●●
● ●●●
● ●●
● ● ●● ● ●●
●●●●●● ●●
●●●●
●
● ●●●●●
●
●●●
●●●●●●
●● ●●●●
● ● ●●
●
●●
●●●
●●
●● ●●●
●
● ●●●●●
●● ●●●●
●●
●●●●●
● ●● ●●●●●●
●●● ● ●
●●
●●●●
●
●●
● ●●
●●●● ●●●●●●●●●●
●
●●
●
●
●●
●●●●●
●
●
●●●
●●
●●
●●
● ●● ●● ● ● ● ●● ●
●●●
●
●●●●
●
●
●
●
●●
●●●
●
●●●● ●●●●●
●●
●
●●
●
●●● ●
●●●●
●
●
●
●
●●●●●● ●●●●
●
●
●
●●
●● ●
●
●
●●●●●●●●
●
●●
●
●
●
●●
●
●
●● ●
●
●
●●
●
●●●
●●
●●
●
●
● ● ●● ●
●
● ●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●●●●●●●●●
●
●
●●●
●●
●
●
●●●
●
●
●●●● ●●
●
●●
● ●
●
●●●●●●
●
●
●
● ● ●●● ●●●
●●
●
●
●
●●
●●
●
●●●●●●
●
●
●●
●●
●
●
●
●●
●
●● ●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●●●
●●●●
●●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●
●
●
●
●● ●●
●
●
●
●●
● ●●●
●
●●
●
●
●
●●● ●●●
●
●
●
●
●
●●●●
●
●
●
●●●
●
●
●●●
●● ●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●
●
●●●
●● ●
●
●●
●●
●
●
●●
●●●
●●●
●
●●
● ●●
●●●●
●
●
●
●●
●● ●●●
●● ●● ● ● ●
● ●● ●●●
●
●●
●
●●●
●●
●
●●●
●
●
●
●
●
●
●
●●
●●●
●●
●●
●●●
●
●●●●
●●
●
● ●● ●
●●●●●
●●●●●●●●
●
●
● ●●● ● ●● ●● ● ●●●
●
●
●
●
●●●
●●● ●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
● ●●●●● ●●● ●
●●
●
● ●●
●●●●●●● ●● ●
●
●
●● ●● ●
● ●
● ●
●
● ●●● ● ●●● ●
●●
●● ●●●●● ●●●●●● ●● ●
●
●
● ●●● ●
● ●● ● ●
●● ●
●
●● ●●●
● ● ●● ●●●●●●
● ●● ●
●
● ●●
● ●●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●●
●●
●●●●
●●●
●● ●●● ●●●●
●● ● ● ●● ● ●●●●●
●●
●●
●
●●● ●●
●
● ●
●
● ●●●● ●●
●
●
● ●
●
●●
●● ●
●● ●
●
●●
●
●●●●●
●● ● ●●●●●●
●● ● ●●
●
●
●
●
● ●●● ●
●●●● ● ●
●
●●
●●
●
−2
0
2
4
PC1
M3 competition data
23
Freq
3
2
PC2
1
0
−1
−2
−3
●
● ●
●
●●
●●
● ●
●
●● ●
●
●
●
●●
●● ●● ●●●
●● ●
●●●●
●
●● ●●●
● ●●
● ●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●●●
●
●
●
●●
●●
● ●● ●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
● ●●●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●●●
●
● ●●●
●
● ●●
●
●
●
●
●
●
●
●
●●●●
●● ●●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
● ●
● ●
●
●
●
●●
●
● ● ●●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ● ●
●●
●
●
●
●
●●●●
●
●
●●
●●
●●●
●
●●●●
● ●●●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●●●●
●
● ●●●
●
●
●
●
●●●●
● ●●●
●●●●●
●● ● ● ● ●●
●
● ● ●●
●
●●
●
●
●●
●● ● ●●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●●●●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●●
●
●
●
●
●● ●●
●
●
●
●
●● ●●
●
●
●● ●
●
●● ●● ●●
●●●●
●
●
●●●● ●●
●
●
●
●●
● ●● ●● ●
●
●
●●
●
●
●●
●
●
●
●●●
●
● ●●
● ●
●●●
●●●
●
●
●●
●
●
●●●
●
●
●
●●
●● ●●
●●
●●● ● ● ●●●●● ● ●
●
●●
●
●●
● ●● ●●
●●
●● ●●
●●
●●●
●
●
●
●
●
● ●●
●
●
●●●●
●
● ●
●●●
●
●
●● ● ●●
●
●
●
●● ●
● ●
●
●●
●●●
●●
●
●●
●●
●
●● ●
●
● ●● ●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●●●
● ●●●● ● ●
●●●●
●
●● ●●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●●
● ●●●
●●●●
● ●●
●● ●
●●
●●●
● ●●●
●
● ●●
●
●
● ●●
●●
●
●●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●●● ●
●●
●
●
●● ●●
●●
●
●
●
●
●
●●
● ● ● ●●
●●
●●●
●
●
●●
●●
● ●● ●
●
●●
●
●
●●●●
●●
●●
●
●
●
●●
●
●
●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●●
●
●●●
●
●●
●
●●●
●●●●●● ●
●
●
●●
● ●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●●
● ● ●● ●
●●●●
●
●●
●
●
●
●
●
●
●●●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
● ●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●●
●●
●● ●
●
●
●●●●
●
●
●
●●
● ●●●●● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●●
●
●● ●
●
●
●●●●●●
●
●●● ● ●
●
●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●●
● ●●
●
●●●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●● ●
●
●
●●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●●
●●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●
●●
●●●●
●●
●●
● ●● ●● ●
●●
●● ●
●
●
●
● ●●● ●●●
●
●● ●
●
●●
●
●●●
●
●
●
●●
●
●
●●
●●●●●●
●●●
●
●
●●
● ●
●●
● ● ●●
●
●
●● ●
●●
●
● ●● ● ●
●●
●
●●●●
● ●
●●●
●●
●● ●●
●●●●●
●● ●●●●
●● ●
●
●
●
● ●
●
● ●
● ●● ● ●
●
●
●● ●●●
●
●
● ●●● ●●●●●●
●
●
●
●
●●●●
●● ●●●
● ● ●●
●●●●
●
●●●
●
●●
●
●●
●●● ● ●
●
●
●
●●
●● ● ●
●●●
●
●
●
●
● ●●
●●●●●●
●●●
●
●
●
●●● ●●
●●
●
● ●
●● ●●●
●
●
● ●●
● ●
●
●●●
●●
●● ●
●● ●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●● ●
●
●
● ●●● ● ●
●●
●● ●● ●
●
●
●
●
●
●
−2
0
2
value
12
9
6
3
4
PC1
M3 competition data
23
Season
3
2
PC2
1
0
−1
−2
−3
●
● ●
●
●
●
●●
●●
● ●
●
●● ●
●
●●
●● ●● ●●●
●● ●
●●●●
●
●●●●●
● ●
● ●●
●●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
● ●●● ● ●● ●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●●●
●
● ●●●
●
● ●●
●
●
●
●
●
●
●
●
●●●●
●
●● ●●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●●
●
●
●
●●
●
● ●●
● ●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●● ● ●
●●
●
●
●
●
●●●●
●
●
●●
●●
●●●
●
●
●●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●●●
●
●
●●● ● ●●●● ● ● ● ●●
● ●●●
●
●
●
●
●
●●●●
● ●●●
●●●●●
●
● ● ●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●● ●
●
●● ●● ●●
●●●●
●
●
●●●● ●●
●
●
●
●●
● ●● ●● ●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
● ●●
● ●
●●●
●●
●
●
●●
●
●
●●●
●
●
●
●●
●● ●●
●●
●●● ● ● ●●●●● ● ●
●
●●
●
●●
● ●● ●●
●●
●● ●●
●●
●●●
●
●
●
●
●
●
● ●●
●
●
●●●●
●
● ●
●●●
●
●
●● ● ●●
●
●
●
●● ●
● ●
●
●●
●●●
●
●
●●
●●
●
●● ●
●
● ●● ●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
● ●●●● ● ●
●●●●
●
●● ●●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●●
● ●●●
●●●●
●●
●● ●
●●
●●●
● ●●●●
●
● ●●
●
●
●●●
●●
●●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●● ●
●●
●
●
●●
●●●●●
●
●
●●
●
●
●
●
●
●●
●● ● ●
●●
●
●
●●
●●
● ●● ●
●
●●
●
●
●●●
●●
●
●
●
●
●●
●●●
●
●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●●
●
●●●
●
●●
●●●
●●●
●
●
●
● ●●
●
●
●
●
●
●●
●●● ●
●●●
●●
●●
●
●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
● ●● ● ● ●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
● ●●
●
●
●●●
●
●
●
●
●
●●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●● ●
●
●
●●●●
●
●
●
●●
● ●●●●● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●●
●
●● ●
●
●
●●●●●●
●
●●● ● ●
●
●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
● ●●
●
●●●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●● ●
●
●
●●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●●
●
●●
●
●
●●
●●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●●●
●●
●●
●●
●
●●
● ●● ●● ●
●●
●● ●
●
●
● ●●● ●●●
●
●● ●
●
●
●●●
●
●●
●
●
●●
●
●
●●
●
●●●●●
●
●●●
●
●●
● ●
●●
● ● ●●
●
●● ● ●●
●●
●
● ●● ● ●
●●
●
●●●●
● ●
●●●
● ●● ●
●● ●●
●●●●●
●
●● ●
●
●
●●● ●
● ●
●
● ●
● ●● ● ●
●
●
●● ●●●
●
●
● ●●● ●●●●●●
●
●
●
●
●●●●
●● ●●●
●● ●
●●●●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●● ●●
●●●
●
●
●
●
● ●●●
● ●●
●●●●●●
●●●
●
●
●●
●●
●
● ●
●● ●●●
●
●
●●● ●
● ●
●
●●●
●●
●● ●
●● ●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●● ●
●
●
● ●●● ●●
●●
●● ●● ●
●
●
●
●
●
●
−2
0
2
value
0.75
0.50
0.25
0.00
4
PC1
M3 competition data
23
Trend
3
2
PC2
1
0
−1
−2
−3
●
● ●
●
●
●
●●
●●
● ●
●
●● ●
●
●●
●● ●● ●●●
●● ●
●●●●
●
●●●●●
● ●
● ●●
●●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
● ●●● ● ●● ●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●●●
●
● ●●●
●
● ●●
●
●
●
●
●
●
●
●
●●●●
●
●● ●●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●●
●
●
●
●●
●
● ●●
● ●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●● ● ●
●●
●
●
●
●
●●●●
●
●
●●
●●
●●●
●
●
●●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●●●
●
●
●●● ● ●●●● ● ● ● ●●
● ●●●
●
●
●
●
●
●●●●
● ●●●
●●●●●
●
● ● ●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●● ●
●
●● ●● ●●
●●●●
●
●
●●●● ●●
●
●
●
●●
● ●● ●● ●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
● ●●
● ●
●●●
●●
●
●
●●
●
●
●●●
●
●
●
●●
●● ●●
●●
●●● ● ● ●●●●● ● ●
●
●●
●
●●
● ●● ●●
●●
●● ●●
●●
●●●
●
●
●
●
●
●
● ●●
●
●
●●●●
●
● ●
●●●
●
●
●● ● ●●
●
●
●
●● ●
● ●
●
●●
●●●
●
●
●●
●●
●
●● ●
●
● ●● ●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
● ●●●● ● ●
●●●●
●
●● ●●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●●
● ●●●
●●●●
●●
●● ●
●●
●●●
● ●●●●
●
● ●●
●
●
●●●
●●
●●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●● ●
●●
●
●
●●
●●●●●
●
●
●●
●
●
●
●
●
●●
●● ● ●
●●
●
●
●●
●●
● ●● ●
●
●●
●
●
●●●
●●
●
●
●
●
●●
●●●
●
●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●●
●
●●●
●
●●
●●●
●●●
●
●
●
● ●●
●
●
●
●
●
●●
●●● ●
●●●
●●
●●
●
●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
● ●● ● ● ●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
● ●●
●
●
●●●
●
●
●
●
●
●●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●● ●
●
●
●●●●
●
●
●
●●
● ●●●●● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●●
●
●● ●
●
●
●●●●●●
●
●●● ● ●
●
●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
● ●●
●
●●●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●● ●
●
●
●●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●●
●
●●
●
●
●●
●●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●●●
●●
●●
●●
●
●●
● ●● ●● ●
●●
●● ●
●
●
● ●●● ●●●
●
●● ●
●
●
●●●
●
●●
●
●
●●
●
●
●●
●
●●●●●
●
●●●
●
●●
● ●
●●
● ● ●●
●
●● ● ●●
●●
●
● ●● ● ●
●●
●
●●●●
● ●
●●●
● ●● ●
●● ●●
●●●●●
●
●● ●
●
●
●●● ●
● ●
●
● ●
● ●● ● ●
●
●
●● ●●●
●
●
● ●●● ●●●●●●
●
●
●
●
●●●●
●● ●●●
●● ●
●●●●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●● ●●
●●●
●
●
●
●
● ●●●
● ●●
●●●●●●
●●●
●
●
●●
●●
●
● ●
●● ●●●
●
●
●●● ●
● ●
●
●●●
●●
●● ●
●● ●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●● ●
●
●
● ●●● ●●
●●
●● ●● ●
●
●
●
●
●
●
−2
0
2
value
0.75
0.50
0.25
4
PC1
M3 competition data
23
ACF
3
2
PC2
1
0
−1
−2
−3
●
● ●
●
●
●
●●
●●
● ●
●
●● ●
●
●●
●● ●● ●●●
●● ●
●●●●
●
●● ●●●
● ●●
● ●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●●●
●
●
●
●●
●●
● ●● ●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
● ●●●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●●●
●
● ●●●
●
● ●●
●
●
●
●
●
●
●
●
●●●●
●● ●●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
● ●
●●●
●
●
●
●●
●
● ●●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ● ●
●●
●
●
●
●
●●●●
●
●
●●
●●
●●●
●
●
●●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●●●●
●
●●● ● ●●●● ● ● ● ●●
● ●●●
●
●
●
●
●
●●●●
● ●●●
●●●●●
●
● ● ●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●● ●
●
●● ●● ●●
●●●●
●
●
●●●● ●●
●
●
●
●●
● ●● ●● ●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
● ●●
● ●
●●●
●●●
●
●
●●
●
●
●●●
●
●
●
●●
●● ●●
●●
●●● ● ● ●●●●● ● ●
●
●●
●
●●
● ●● ●●
●●
●● ●●
●●
●●●
●
●
●
●
●
● ●●
●
●
●●●●
●
● ●
●●●
●
●
●● ● ●●
●
●
●
●● ●
● ●
●
●●
●●●
●
●
●●
●●
●
●● ●
●
● ●● ●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
● ●●●● ● ●
●●●●
●
●● ●●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●●
● ●●●
●●●●
●●
●● ●
●●
●●●
● ●●●●
●
● ●●
●
●
● ●●
●●
●●
●
●●
●
●●
●
●
●
●●
●
●●●
●
●
●
●●● ●
●●
●
●
●●
●●●●●
●
●●
●
●
●
●
●
●●
●● ● ●
●●
●
●
●●
●●
● ●● ●
●
●●
●
●
●●●●
●●
●
●
●
●
●●
●●●
●
●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●●
●
●●●
●
●●
●●●
●●●
●
●
●
● ●●
●
●
●
●
●
●●
●●● ●
●●●
●●
●●
●
●
●
●
●●
● ● ●● ●
●●●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
● ●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●●
●●
●● ●
●
●
●●●●
●
●
●
●●
● ●●●●● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●●
●
●● ●
●
●
●●●●●●
●
●●● ● ●
●
●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●●
● ●●
●
●●●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●● ●
●
●
●●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●●
●●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●
●●
●●
●●
●●
●●
● ●● ●● ●
●●
●● ●
●
●
●
● ●●● ●●●
●
●● ●
●
●●
●
●●●
●
●
●
●●
●
●
●●
●●●●●●
●
●●●
●
●●
● ●
●●
● ● ●●
●
●● ●
●●
●
● ●● ● ●
●●
●
●●●●
● ●
●●●
● ●● ●
●●
●● ●●
●●●●●
●
●● ●
●
●
●●● ●
● ●
●
● ●
● ●● ● ●
●
●
●● ●●●
●
●
● ●●● ●●●●●●
●
●
●
●
●●●●
●● ●●●
●● ●
●●●●
●
●●●
●
●●
●
●●
●●● ● ●
●
●
●
●
●●
●● ● ●
●●●
●
●
●
●
● ●●
●●●●●●
●●●
●
●
●
●●● ●●
●●
●
● ●
●● ●●●
●
●
● ●●
● ●
●
●●●
●●
●● ●
●● ●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●● ●
●
●
● ●●● ● ●
●●
●● ●● ●
●
●
●
●
●
●
−2
0
2
value
0.5
0.0
4
PC1
M3 competition data
23
SpecEntr
3
2
PC2
1
0
−1
−2
−3
●
● ●
●
●
●
●●
●●
● ●
●
●● ●
●
●●
●● ●● ●●●
●● ●
●●●●
●
●● ●●●
● ●●
● ●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
● ●●● ● ●● ●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●●●
●
● ●●●
●
● ●●
●
●
●
●
●
●
●
●
●●●●
●● ●●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
● ●
● ●
●
●
●
●●
●
● ● ●●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ● ●
●●
●
●
●
●
●●●●
●
●
●●
●●
●●●
●
●
●●●●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●●●●
●
● ●●●
●
●
●
●
●
●●●●
● ●●●
●●●●●
●● ● ● ● ●●
●
● ● ●●
●
●●
●
●
●●
●● ● ●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●●●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●● ●
●
●● ●● ●●
●●●●
●
●
●●●● ●●
●
●
●
●●
● ●● ●● ●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
● ●●
● ●
●●●
●●●
●
●
●●
●
●
●●●
●
●
●
●●
●● ●●
●●
●●● ● ● ●●●●● ● ●
●
●●
●
●●
● ●● ●●
●●
●● ●●
●●
●●●
●
●
●
●
●
● ●●
●
●
●●●●
●
● ●
●●●
●
●
●● ● ●●
●
●
●
●● ●
● ●
●
●●
●●●
●
●
●●
●●
●
●● ●
●
● ●● ●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
● ●●●● ● ●
●●●●
●
●● ●●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●●
● ●●●
●●●●
●●
●● ●
●●
●●●
● ●●●●
●
● ●●
●
●
● ●●
●●
●●
●
●●
●
●●
●
●
●
●●
●
●●●
●
●
●
●●● ●
●●
●
●
●●
●●●●●
●
●●
●
●
●
●
●
●●
●● ● ●
●●
●
●
●●
●●
● ●● ●
●
●●
●
●
●●●●
●●
●
●
●
●
●●
●●●
●
●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●●
●
●●●
●
●●
●●●
●●●
●
●
●
● ●●
●
●
●
●
●
●●
●●● ●
●●●
●●
●●
●
●
●
●
●●
● ● ●● ●
●●●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
● ●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●●
●●
●● ●
●
●
●●●●
●
●
●
●●
● ●●●●● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●●
●
●● ●
●
●
●●●●●●
●
●●● ● ●
●
●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●●
● ●●
●
●●●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●● ●
●
●
●●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●●
●●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●
●●
●●
●●
●●
●●
● ●● ●● ●
●●
●● ●
●
●
●
● ●●● ●●●
●
●● ●
●
●●
●
●●●
●
●
●
●●
●
●
●●
●●●●●●
●●●
●
●●
● ●
●●
● ● ●●
●
●● ●
●●
●
● ●● ● ●
●●
●
●●●●
● ●
●●●
● ●● ●
●●
●● ●●
●●●●●
●
●● ●
●
●
●●● ●
● ●
●
● ●
● ●● ● ●
●
●
●● ●●●
●
●
● ●●● ●●●●●●
●
●
●
●
●●●●
●● ●●●
●● ●
●●●●
●
●●●
●
●●
●
●●
●●● ● ●
●
●
●
●
●●
●● ● ●
●●●
●
●
●
●
● ●●
●●●●●●
●●●
●
●
●
●●● ●●
●●
●
● ●
●● ●●●
●
●
● ●●
● ●
●
●●●
●●
●● ●
●● ●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●● ●
●
●
● ●●● ● ●
●●
●● ●● ●
●
●
●
●
●
●
−2
0
2
value
0.9
0.8
0.7
0.6
0.5
4
PC1
M3 competition data
23
Lambda
3
2
PC2
1
0
−1
−2
−3
●
● ●
●
●
●
●●
●●
● ●
●
●● ●
●
●●
●● ●● ●●●
●● ●
●●●●
●
●●●●●
● ●
● ●●
●●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
● ●●● ● ●● ●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●●●
●
● ●●●
●
● ●●
●
●
●
●
●
●
●
●
●●●●
●
●● ●●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●●
●
●
●
●●
●
● ●●
● ●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●● ● ●
●●
●
●
●
●
●●●●
●
●
●●
●●
●●●
●
●
●●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●●●
●
●
●●● ● ●●●● ● ● ● ●●
● ●●●
●
●
●
●
●
●●●●
● ●●●
●●●●●
●
● ● ●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●● ●
●
●● ●● ●●
●●●●
●
●
●●●● ●●
●
●
●
●●
● ●● ●● ●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
● ●●
● ●
●●●
●●
●
●
●●
●
●
●●●
●
●
●
●●
●● ●●
●●
●●● ● ● ●●●●● ● ●
●
●●
●
●●
● ●● ●●
●●
●● ●●
●●
●●●
●
●
●
●
●
●
● ●●
●
●
●●●●
●
● ●
●●●
●
●
●● ● ●●
●
●
●
●● ●
● ●
●
●●
●●●
●
●
●●
●●
●
●● ●
●
● ●● ●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
● ●●●● ● ●
●●●●
●
●● ●●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●●
● ●●●
●●●●
●●
●● ●
●●
●●●
● ●●●●
●
● ●●
●
●
●●●
●●
●●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●● ●
●●
●
●
●●
●●●●●
●
●
●●
●
●
●
●
●
●●
●● ● ●
●●
●
●
●●
●●
● ●● ●
●
●●
●
●
●●●
●●
●
●
●
●
●●
●●●
●
●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●●
●
●●●
●
●●
●●●
●●●
●
●
●
● ●●
●
●
●
●
●
●●
●●● ●
●●●
●●
●●
●
●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
● ●● ● ● ●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
● ●●
●
●
●●●
●
●
●
●
●
●●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●● ●
●
●
●●●●
●
●
●
●●
● ●●●●● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●●
●
●● ●
●
●
●●●●●●
●
●●● ● ●
●
●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
● ●●
●
●●●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●● ●
●
●
●●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●●
●
●●
●
●
●●
●●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●●●
●●
●●
●●
●
●●
● ●● ●● ●
●●
●● ●
●
●
● ●●● ●●●
●
●● ●
●
●
●●●
●
●●
●
●
●●
●
●
●●
●
●●●●●
●
●●●
●
●●
● ●
●●
● ● ●●
●
●● ● ●●
●●
●
● ●● ● ●
●●
●
●●●●
● ●
●●●
● ●● ●
●● ●●
●●●●●
●
●● ●
●
●
●●● ●
● ●
●
● ●
● ●● ● ●
●
●
●● ●●●
●
●
● ●●● ●●●●●●
●
●
●
●
●●●●
●● ●●●
●● ●
●●●●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●● ●●
●●●
●
●
●
●
● ●●●
● ●●
●●●●●●
●●●
●
●
●●
●●
●
● ●
●● ●●●
●
●
●●● ●
● ●
●
●●●
●●
●● ●
●● ●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●● ●
●
●
● ●●● ●●
●●
●● ●● ●
●
●
●
●
●
●
−2
0
2
value
1.00
0.75
0.50
0.25
0.00
4
PC1
M3 competition data
23
Predictability
Three general forecasting methods:
Theta method Best overall in 2000 M3
competition
ETS
Exponential smoothing state
space models
STL-AR
AR model applied to seasonally
adjusted series from STL, and
seasonal component forecast
using the seasonal naive method.
Compute minimum MASE from all three methods
M3 competition data
24
Predictability
Three general forecasting methods:
Theta method Best overall in 2000 M3
competition
ETS
Exponential smoothing state
space models
STL-AR
AR model applied to seasonally
adjusted series from STL, and
seasonal component forecast
using the seasonal naive method.
Compute minimum MASE from all three methods
M3 competition data
24
Predictability
2000
4000
6000
8000
10000
Theta
1975
1980
1985
1990
M3 competition data
25
Predictability
2000
4000
6000
8000
10000
ETS
1975
1980
1985
1990
M3 competition data
25
Predictability
2000
4000
6000
8000
10000
AR
1975
1980
1985
1990
M3 competition data
25
Predictability
3000
4000
5000
6000
Theta
1980
1982
1984
1986
1988
1990
M3 competition data
1992
26
Predictability
3000
4000
5000
6000
ETS
1980
1982
1984
1986
1988
1990
M3 competition data
1992
26
Predictability
3000
4000
5000
6000
STL−AR
1980
1982
1984
1986
1988
1990
M3 competition data
1992
26
Predictability
6000
6500
7000
7500
8000
Theta
1984
1986
1988
1990
1992
M3 competition data
1994
27
Predictability
6000
6500
7000
7500
8000
ETS
1984
1986
1988
1990
1992
M3 competition data
1994
27
Predictability
6000
6500
7000
7500
8000
STL−AR
1984
1986
1988
1990
1992
M3 competition data
1994
27
Predictability
Low
3
PC2
1
0
−1
−2
−3
●
●●●
● ●●
●
●
●
● ●●
●
● ●
●●
●
●●●
●
●
●
●● ●●●
● ●●
●
● ●●
●●
●
● ●
●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
● ●
●●
●
●
●
●
●
●●
●
●● ●
●●
●
●●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●
●●●●●
●
●
●●
●
●
●
● ●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●●
●
●
●
●
●
● ●●
●
●
● ●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ● ●●●
●
●
●
●
●
● ●
●●
●
●
●
●●
●● ●
● ● ●●●
●●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●●
●
●
●
●
●
●
●
●●●●
●
●●●●
●
●●
●
●
●
●
●● ●
●
●●
●●●●● ● ●
●
●●●
●
●●
●● ●●
●●●
●●
●●
●●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●●● ●
●●
●
●
●●
●
●●
●●
●
●
● ● ●
●
●
●●●●
●●●●
● ●●
●
●
●
● ●●
●
●
●● ●●●
●
●
●●●●
●
● ●●● ● ●
●
●●
●●●
●
●
●
●
●
●●●●●
●
●
●●
●
●
● ●● ● ●●●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●● ●
●●●
●
●●
●
●
●● ●
●
●● ●● ● ●
●
●
●
●
●●●● ● ●
●●●
●
●
●
●
●
●
●●
●
●● ●
●
●
●●●
● ● ●●
●
●●●● ● ●● ●
●● ●●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
● ●
●
● ●●
●
●
●
●●●
●●● ●●
●
●●
●
●
●
●●
●●●
●
●
●
●●●●● ● ● ●●●● ● ●
●●
●●
●
●●
● ●●●●● ●●
● ●● ●
●●●●
●●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●● ●●●
●
●●
●● ●
● ●
●
● ●● ●
● ●●
● ●
●
●
●
●● ●
●●●●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
● ● ●
●●●● ●● ● ● ●
●●
●
●●
●●●●
●●● ●●
●● ●
● ●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●● ●●●●●●
●●●
● ● ● ●●●●
●●
●
● ●● ●●
● ●●
●●
● ●● ●
●
●
●●
●
●●
●
●
● ●●●●●
●
●
●●
●
●
●
● ●●
●●
●●
●●
●●●
●
● ● ●
●●●
●
●●
●●●
●
●● ●●
●●●
●●
●
●
●●
●● ●
●●
●●
●●●●●●
●
●
●●
● ● ●
●
●
●● ●●
●
●●●
● ●
●●
●●● ●● ●●
●
●
●
●●
●
●
●
● ●●●
●●
●
●
●
●
●
●
●
●
● ●●
●●
●
●●●● ●
●●
●●
● ● ●
●● ●
●●
● ●● ●●●
● ●
●
●
●
●
●●
●●●
●
●
●●
●
●
● ● ●●
●●●●
●● ●●●
●●
●
●
●
●
●●●●●
●
● ●
●
●
● ●●
●
●
●●●●●●●
●●
● ●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●● ●
●●
●
●
●
●
●
●●
●
●●
●
●
●● ●●
●●●
● ● ●
●
● ●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●● ●
●●●
●●●
●
●●
●●●● ●●●
●
●●
●●● ● ●
●●●●●
●●
●
●
●●
●●
●
●
●
●●● ●
●
●
●●
●●
●
●●
●
●
●●
●●●
●
●
●
●
●●●
●
●
●●
● ●●●● ●●●
●
●●
●
●
●●
●
●●
●
●● ●
●
●
●
● ●●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●●●● ●
●
●
●
●
●● ●●
●
●
●
●
● ●
●●
●
●
●●●
●
●
●
●●●
●
●
●
●●●
●
●
● ●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●●●
●
●
●●●
●
●
●
●●●●
●
● ● ●●●
●
●●
●
●
●●●
●
●
●
●●●
●
●●
● ●●
●
●
●
●
● ●
●
●●●
●
●
●
●
●
●
●●●
●●
●
●●
●
●●●
●
●
●
●
●
●●
●●
●●● ●
●
●
●●
●
●
●●●●
●
●
●●
● ● ●
●
●
●
●
●●●
●
●
●●●
●
●●
●●●●
●
● ● ●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●● ●● ●●
●
●● ●● ● ● ●
●
●●●
●● ●●
●●
● ●●●
● ●●
●●●●● ● ●●
●
●
●
● ●●●
●
●
● ●
●
●●●●●
● ● ● ●●
●
● ● ●● ●
●● ●
● ●● ●
● ●
●
● ● ●● ●
●●
●● ●●
●
● ●● ●
●
●● ●
●
●
●●
● ●
●
●
● ●
● ●
●
●
● ● ●
●● ●●●
● ●● ●
●
●
● ● ● ● ●●● ●
●
●
●
●
●
●● ●
●
● ●
●
●●
● ●●
●●
● ● ●●● ●●
●●
●
●
●●●●●● ● ●●
●●●
●
●● ● ●
●●●
●
●
●
●
● ●
●
● ●●
●● ●
●●●
●
●
●● ●●
●● ●
●● ●
●
● ● ●
●●● ●●
● ●
●
●
●● ●
●●●
●
● ● ● ●●●●●
●
●
●
●
● ●
● ●●
●
●●
●● ● ●●
●● ● ●
●
● ●
●● ●
●
●●●
●
●
●
●
●
●
●
−2
0
2
Low
2 values
MASE
1
PC2
●
●●
●
●●
●
2
3
●
●
4
0
−1
−2
−3
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
● ●●●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●●
●
● ●● ● ●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●● ● ●
●
●
●
●
●
●
●
●
●●●
●●●● ● ●● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
● ●●●●● ●●
●●
●
●
●
● ●
●●●
●
●
●
●
●●
● ●
●
●●
● ●●
●●
● ● ●
●●● ●●
●
●● ●
●
●●●
●●●
●●
●
●
●●
●
●●
●
●●●
●
●
●●
●●
●
●
● ● ●
●
●
●
● ●
●●
●●
●
●
●
●
●
●
● ●●
●●
●
●
●●
●● ●
●●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●● ●
●●●
●
●●●●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●● ●●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●●
●
●
●
●
●● ●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●
●
●
●
●
●
●●●●
●●
● ● ●
●
●
●●●
●●
●
●●
● ● ●
●
●●
●
●●
●● ●
●●
●●
●●●
●
●●
● ●
●
●
●
●
●● ●●
●● ●●
●●
●
●
●● ●● ●●●
●
●
● ●●●
●
●
●●
●
●● ●
● ●● ●
●
●
● ● ● ●
●
●
●
●
●
●
● ● ●
●● ●●●
● ● ● ● ●●●
● ●
● ● ●●
●●
●●
● ●●●
●●
● ● ●
●●●
● ●
●
● ● ● ●●●●●
●
● ●
●
●
●
● ●
●
●
●●●
● ●
●●● ●
●
●●
●
●
●
●
●
●
●
−2
M3 competition data
28
Predictability
Middle
3
PC2
1
0
−1
−2
−3
●
●●●
● ●●
●
●
●
● ●●
●
● ●
●●
●
●●●
●
●
●
●● ●●●
● ●●
●
● ●●
●●
●
● ●
●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
● ●
●●
●
●
●
●
●
●●
●
●● ●
●●
●
●●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●
●●●●●
●
●
●●
●
●
●
● ●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●●
●
●
●
●
●
● ●●
●
●
● ●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ● ●●●
●
●
●
●
●
● ●
●●
●
●
●
●●
●● ●
● ● ●●●
●●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●●
●
●
●
●
●
●
●
●●●●
●
●●●●
●
●●
●
●
●
●
●● ●
●
●●
●●●●● ● ●
●
●●●
●
●●
●● ●●
●●●
●●
●●
●●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●●● ●
●●
●
●
●●
●
●●
●●
●
●
● ● ●
●
●
●●●●
●●●●
● ●●
●
●
●
● ●●
●
●
●● ●●●
●
●
●●●●
●
● ●●● ● ●
●
●●
●●●
●
●
●
●
●
●●●●●
●
●
●●
●
●
● ●● ● ●●●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●● ●
●●●
●
●●
●
●
●● ●
●
●● ●● ● ●
●
●
●
●
●●●● ● ●
●●●
●
●
●
●
●
●
●●
●
●● ●
●
●
●●●
● ● ●●
●
●●●● ● ●● ●
●● ●●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
● ●
●
● ●●
●
●
●●
●
●●●
●●● ●●
●
●●
●
●
●
●●
●●●
●
●
●
●●●●● ● ● ●●●● ● ●
●●
●●
●
●●
● ●●●●● ●●
● ●● ●
●●●●
●●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●● ●●●
●
●●
●● ●
● ●
●
● ●● ●
● ●●
● ●
●
●
●
●● ●
●●●●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
● ● ●
●●●● ●● ● ● ●
●●
●
●●
●●●●
●●● ●●
●● ●
● ●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●● ●●●●●●
●●●
● ● ● ●●●●
●●
●
● ●● ●●
● ●●
●●
● ●● ●
●
●
●●
●
●●
●
●
● ●●●●●
●
●
●●
●
●
●
● ●●
●●
●●
●●
●●●
●
● ● ●
●●●
●
●●
●●●
●
●● ●●
●●●
●●
●
●
●●
●● ●
●●
●●
●●●●●●
●
●
●●
● ● ●
●
●
●● ●●
●
●●●
● ●
●●
●●● ●● ●●
●
●
●
●●
●
●
●
● ●●●
●●
●
●
●
●
●
●
●
●
● ●●
●●
●
●●●● ●
●●
●●
● ● ●
●● ●
●●
● ●● ●●●
● ●
●
●
●
●
●●
●●●
●
●
●●
●
●
● ● ●●
●●●●
●● ●●●
●●
●
●
●
●
●●●●●
●
● ●
●
●
● ●●
●
●
●●●●●●●
●●
● ●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●● ●
●●
●
●
●
●
●
●●
●
●●
●
●
●● ●●
●●●
● ● ●
●
● ●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●● ●
●●●
●●●
●
●●
●●●● ●●●
●
●●
●●● ● ●
●●●●●
●●
●
●
●●
●●
●
●
●
●●● ●
●
●
●●
●●
●
●●
●
●
●●
●●●
●
●
●
●
●●●
●
●
●●
● ●●●● ●●●
●
●●
●
●
●●
●
●●
●
●● ●
●
●
●
● ●●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●●●● ●
●
●
●
●
●● ●●
●
●
●
●
● ●
●●
●
●
●●●
●
●
●
●●●
●
●
●
●●●
●
●
● ●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●●●
●
●
●●●
●
●
●
●●●●
●
● ● ●●●
●
●●
●
●
●●●
●
●
●
●●●
●
●●
● ●●
●
●
●
●
● ●
●
●●●
●
●
●
●
●
●
●●●
●●
●
●●
●
●●●
●
●
●
●
●
●●
●●
●●● ●
●
●
●●
●
●
●●●●
●
●
●●
● ● ●
●
●
●
●
●●●
●
●
●●●
●
●●
●●●●
●
● ● ●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●● ●● ●●
●
●● ●● ● ● ●
●
●●●
●● ●●
●●
● ●●●
● ●●
●●●●● ● ●●
●
●
●
● ●●●
●
●
● ●
●
●●●●●
● ● ● ●●
●
● ● ●● ●
●● ●
● ●● ●
● ●
●
● ● ●● ●
●●
●● ●●
●
● ●● ●
●
●● ●
●
●
●●
● ●
●
●
● ●
● ●
●
●
● ● ●
●● ●●●
● ●● ●
●
●
● ● ● ● ●●● ●
●
●
●
●
●
●● ●
●
● ●
●
●●
● ●●
●●
● ● ●●● ●●
●●
●
●
●●●●●● ● ●●
●●●
●
●● ● ●
●●●
●●
●
●
●
●
● ●
●
● ●●
●● ●
●●●
●
●
●● ●●
●● ●
●● ●
●
● ● ●
●●● ●●
● ●
●
●
●● ●
●●●
●
● ● ● ●●●●●
●
●
●
●
● ●
● ●●
●
●●
●● ● ●●
●● ● ●
●
● ●
●● ●
●
●●●
●
●
●
●
●
●
●
−2
0
2
Medium
2 values
MASE
1
PC2
●
●●
●
●●
●
2
3
●
●
4
0
−1
−2
−3
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
● ●●●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●●
●
● ●● ● ●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●● ● ●
●
●
●
●
●
●
●
●
●●●
●●●● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
● ●●●●● ●●
●●
●
●
●
● ●
●●●
●
●
●
●
●●
● ●
●
●●
● ●●
●●
● ● ●
●●● ●●
●
●● ●
●
●●●
●●●
●●
●
●
●●
●
●●
●
●●●
●
●
●●
●●
●
●
● ● ●
●
●
●
● ●
●●
●●
●
●
●
●
●
●
● ●●
●●
●
●
●●
●● ●
●●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●● ●
●●●
●
●●●●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●● ●●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●●
●
●
●
●
●● ●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●
●
●
●
●
●
●●●●
●●
● ● ●
●
●
●●●
●●
●
●●
● ● ●
●
●●
●
●●
●● ●
●●
●●
●●●
●
●●
● ●
●
●
●
●
●● ●●
●● ●●
●●
●
●
●● ●● ●●●
●
●
● ●●●
●
●
●●
●
●● ●
● ●● ●
●
●
● ● ● ●
●
●
●
●
●
●
● ● ●
●● ●●●
● ● ● ● ●●●
● ●
● ● ●●
●●
●●
● ●●●
●●
● ● ●
●●●
● ●
●
● ● ● ●●●●●
●
● ●
●
●
●
● ●
●
●
●●●
● ●
●●● ●
●
●●
●
●
●
●
●
●
●
−2
M3 competition data
28
Predictability
High
3
●
●
●
●●●
● ●●
2
PC2
1
0
−1
−2
−3
●
●
●
●
●●
●
●●
● ●●
●
●
● ●
●●
●
●●●
●
●
●
●● ●●●
● ●●
●
● ●●
●●
●
● ●
●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
● ●●
●
● ●● ●
●
●●●
●
●
●●●
●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●●●●
●●
●●
●
●
●
●
●
●
●●
●
●● ●
●●
●
●●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●
●●●●●
●
●
●●
●
●
●
● ●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●●
●
●
●
●
●
● ●●
●
●
● ●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ● ●●●
●
●
●
●
●
● ●
●●
●
●
●
●●
●● ●
● ● ●●●
●●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●●
●
●
●
●
●
●
●
●●●●
●
●●●●
●
●●
●
●
●
●
●● ●
●
●●
●●●●● ● ●
●
●●●
●
●●
●● ●●
●●●
●●
●●
●●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●●● ●
●●
●
●
●●
●
●●
●●
●
●
● ● ●
●
●
●●●●
●●●●
● ●●
●
●
●
● ●●
●
●
●● ●●●
●
●
●●●●
●
● ●●● ● ●
●
●●
●●●
●
●
●
●
●
●●●●●
●
●
●●
●
●
● ●● ● ●●●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●● ●
●●●
●
●●
●
●
●● ●
●
●● ●● ● ●
●
●
●
●
●●●● ● ●
●●●
●
●
●
●
●
●
●●
●
●● ●
●
●
●●●
● ● ●●
●
●●●● ● ●● ●
●● ●●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
● ●
●
● ●●
●
●
●
●●●
●●● ●●
●
●●
●
●
●
●●
●●●
●
●
●
●●●●● ● ● ●●●● ● ●
●●
●●
●
●●
● ●●●●● ●●
● ●● ●
●●●●
●●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●● ●●●
●
●●
●● ●
● ●
●
● ●● ●
● ●●
● ●
●
●
●
●● ●
●●●●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
● ● ●
●●●● ●● ● ● ●
●●
●
●●
●●●●
●●● ●●
●● ●
● ●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●● ●●●●●●
●●●
● ● ● ●●●●
●●
●
● ●● ●●
● ●●
●●
● ●● ●
●
●
●●
●
●●
●
●
● ●●●●●
●
●
●●
●
●
●
● ●●
●●
●●
●●
●●●
●
● ● ●
●●●
●
●●
●●●
●
●● ●●
●●●
●●
●
●
●●
●● ●
●●
●●
●●●●●●
●
●
●●
● ● ●
●
●
●● ●●
●
●●●
● ●
●●
●●● ●● ●●
●
●
●
●●
●
●
●
● ●●●
●●
●
●
●
●
●
●
●
●
● ●●
●●
●
●●●● ●
●●
●●
● ● ●
●● ●
●●
● ●● ●●●
● ●
●
●
●
●
●●
●●●
●
●
●●
●
●
● ● ●●
●●●●
●● ●●●
●●
●
●
●
●
●●●●●
●
● ●
●
●
● ●●
●
●
●●●●●●●
●●
● ●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●● ●
●●
●
●
●
●
●
●●
●
●●
●
●
●● ●●
●●●
● ● ●
●
● ●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●● ●
●●●
●●●
●
●●
●●●● ●●●
●
●●
●●● ● ●
●●●●●
●●
●
●
●●
●●
●
●
●
●●● ●
●
●
●●
●●
●
●●
●
●
●●
●●●
●
●
●
●
●●●
●
●
●●
● ●●●● ●●●
●
●●
●
●
●●
●
●●
●
●● ●
●
●
●
● ●●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●●●● ●
●
●
●
●
●● ●●
●
●
●
●
● ●
●●
●
●
●●●
●
●
●
●●●
●
●
●
●●●
●
●
● ●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●●●
●
●
●●●
●
●
●
●●●●
●
● ● ●●●
●
●●
●
●
●●●
●
●
●
●●●
●
●●
● ●●
●
●
●
●
● ●
●
●●●
●
●
●
●
●
●
●●●
●●
●
●●
●
●●●
●
●
●
●
●
●●
●●
●●● ●
●
●
●●
●
●
●●●●
●
●
●●
● ● ●
●
●
●
●
●●●
●
●
●●●
●
●●
●●●●
●
● ● ●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●● ●● ●●
●
●● ●● ● ● ●
●
●●●
●● ●●
●●
● ●●●
● ●●
●●●●● ● ●●
●
●
●
● ●●●
●
●
● ●
●
●●●●●
● ● ● ●●
●
● ● ●● ●
●● ●
● ●● ●
● ●
●
● ● ●● ●
●●
●● ●●
●
● ●● ●
●
●● ●
●
●
●●
● ●
●
●
● ●
● ●
●
●
● ● ●
●● ●●●
● ●● ●
●
●
● ● ● ● ●●● ●
●
●
●
●
●
●● ●
●
● ●
●
●●
● ●●
●●
● ● ●●● ●●
●●
●
●
●●●●●● ● ●●
●●●
●
●● ● ●
●●●
●
●
●
●
● ●
●
● ●●
●● ●
●●●
●
●
●● ●●
●● ●
●● ●
●
● ● ●
●●● ●●
● ●
●
●
●● ●
●●●
●
● ● ● ●●●●●
●
●
●
●
● ●
● ●●
●
●●
●● ● ●●
●● ● ●
●
● ●
●● ●
●
●●●
●
●
●
●
●
●
●
−2
0
2
High
MASE values
4
M3 competition data
28
Predictability
Yearly data
Yearly data
4
4
●
●●●
●
● ●● ●●
●● ●
●●●
●
● ●●
●
● ●●● ● ●
●●
● ●
●●
●
●
●
●
●●
●●
●
●
●●
●●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●● ●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●●
●
●● ●
●
2
Best
Ets
NoDiff
Stlm−ar
Theta
●
0
●
●
PC2
PC2
2
●●●
●
● ●● ●●
●● ●
●●●
●
● ●●
●
● ●●● ● ●
●●
● ●
●●
●
●
●
●
●●
●●
●
●
●●
●●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●● ●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●●
●
●● ●
●
Best
NoDiff
Stlm−ar
0
●
●
●
−2
−2
−4
−4
−2
0
2
PC1
4
6
−2
Quarterly data
2
2
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
● ● ●
●
●●●●●
● ●●
●●
●
● ●●● ●
●
●
●●
●
● ●
●
●
●●
● ●● ● ●● ●
●●●
●
●
●
●
●
●
● ● ●●
●
●
● ●●
●
●●●●
●●
●
● ●
●●
●●
●
●
●●
●
●
● ●
●
●
●
●
● ●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●● ●●●● ●
●
●
●
●
●
●●
●
●
●● ●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●● ● ● ● ●
●
● ●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●●● ●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●● ●●● ● ●
●
●
●
●
●
●
●
●
● ●●
●
● ● ●●
●
●
●●
●●
●
●●●
●● ●●
●
●●●
●
● ●
●
●
● ● ●● ●
●●●
●
●
●
●
●●
● ●
●
●
●
●
●
●
2
PC1
4
6
Quarterly data
Actual
4
2
Best
Ets
●
●
2
4
0
SVM prediction
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
● ● ●
●
●●●●●
● ●●
●●
●
● ●●● ●
●
●
●●
●
● ●
●
●
●●
● ●● ● ●● ●
●●●
●
●
●
●
●
●
● ● ●●
●
●
● ●●
●
●●●●
●●
●
● ●
●●
●●
●
●
●●
●
●
● ●
●
●
●
●
● ●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●● ●●●● ●
●
●
●
●
●
●●
●
●
●● ●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●● ● ● ● ●
●
● ●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●●● ●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●● ●●● ● ●
●
●
●
●
●
●
●
●
● ●●
●
● ● ●●
●
●
●●
●●
●
●●●
●● ●●
●
●●●
●
● ●
●
●
● ● ●● ●
●●●
●
●
●
●
●●
● ●
●
●
●
●
●
●
M3 competition data
Best
29
Ets
●
Predictability
−2
−4
−4
−2
−2
0
2
PC1
4
6
−2
Quarterly data
2
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
● ● ●
●
●●●●●
● ●●
●●
●
● ●●● ●
●
●
●
●
●●
● ●● ●
●
●
●
●
●●
●
●
● ● ●●
●
●
●
●
● ●●●
●
●
●
● ●●
●
●●●●
●●
●
● ●
●●●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●● ●●●
●● ● ● ● ●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ● ●
●
●
●
●●●●●●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
● ● ●●
●
●
●●
●●
●
●●●
●● ●●
●
●●●
●
● ●
●
●
● ● ●● ●
●●●
●
●
●
●
●●
●● ●
●
●
●
●●
●
●● ● ● ●
●●
●
● ●● ●
●
●
●●● ●●
●
●
●●●
●●● ●● ●
●
● ● ● ●● ● ● ● ●
●●●
● ●●●●
●●
●● ●●
● ●●
●●●
●●●
●
● ● ●●
●
●● ●
●● ●
●
● ●
●● ●
● ●●●
● ●● ●● ● ●●●
● ●● ● ● ●
●●
●
●
● ● ●●
●●
● ● ●●
●
● ●●
● ●
●
●●
●
●
●●
● ● ●●●●
●
●
● ● ●●
●
●●
●
●
●
●
●
●
●
● ●●●●
●
● ●● ●
●
●●●
● ● ●●
●●
●
●
●
●
●●●
●●
●
●●●●
●
●●●
●●●
●
●●
●
●
●●
● ●●● ●●
●● ●
●
●
● ●
Best
Ets
NoDiff
Stlm−ar
Theta
●
●
●
0
●
−2
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
● ● ●
●
●●●●●
● ●●
●●
●
● ●●● ●
●
●
●
●
●●
● ●● ●
●
●
●
●
●●
●
●
● ● ●●
●
●
●
●
● ●●●
●
●
●
● ●●
●
●●●●
●●
●
● ●
●●●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●● ●●●
●● ● ● ● ●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ● ●
●
●
●
●●●●●●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
● ● ●●
●
●
●●
●●
●
●●●
●● ●●
●
●●●
●
● ●
●
●
● ● ●● ●
●●●
●
●
●
●
●●
●● ●
●
●
●
●●
●
●● ● ● ●
●●
●
● ●● ●
●
●
●●● ●●
●
●
●●●
●●● ●● ●
●
● ● ● ●● ● ● ● ●
●●●
● ●●●●
●●
●● ●●
● ●●
●●●
●●●
●
● ● ●●
●
●● ●
●● ●
●
● ●
●● ●
● ●●●
● ●● ●● ● ●●●
● ●● ● ● ●
●●
●
●
● ● ●●
●●
● ● ●●
●
● ●●
● ●
●
●●
●
●
●●
● ● ●●●●
●
●
● ● ●●
●
●●
●
●
●
●
●
●
●
● ●●●●
●
● ●● ●
●
●●●
● ● ●●
●●
●
●
●
●
●●●
●●
●
●●●●
●
●●●
●●●
●
●●
●
●
●●
● ●●● ●●
●● ●
●
●
● ●
Best
Ets
NoDiff
Stlm−ar
●
●
●
−4
−2
0
2
PC1
4
6
−2
Monthly data
0
2
PC1
4
6
Monthly data
Actual
4
SVM prediction
2
●
●●
●
●●●
●
●
●●
● ●●
●
●
●●
● ●
●
●
●●
●
●●
●
● ●●
●
●
●
●●
●
●
●● ●
●● ●●
●
●
●●●
● ●
●
●●
●
●
●
● ●●
● ●
●●
●
●
●● ● ●
●
●
●
●●
●● ● ●● ●
●● ●●●
●
●
●●●●
●●
● ●●●
●
●●
● ● ●● ●
●●●
●●
●
●●
●
●
●●
●
● ●●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●●
● ●●
●●●
●●
●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●●●
●●
●●
●
●
●●●●●●
● ●●●
● ●● ●●
●
●●●
●●
●
●
●●
● ●●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●● ●● ●●●● ● ●
●
●●
● ●● ●
●
●●
●●
●
●
● ●●
●●
●●
●●
●●● ● ●
●●
●
● ● ●●
●●● ●●●●
●●
●
Best
Visualisation of big time series data Ets
●
●
2
2
2
6
−2
−4
4
4
4
PC2
PC2
0
2
PC1
Quarterly data
4
2
0
●
●●
●
●●●
●
●
●●
● ●●
●
●
●●
● ●
●
●
●●
●
●●
●
● ●●
●
●
●
●●
●
●
●● ●
●● ●●
●
●
●●●
● ●
●
●●
●
●
●
● ●●
● ●
●●
●
●
●● ● ●
●
●
●
●●
●● ● ●● ●
●● ●●●
●
●
●●●●
●●
● ●●●
●
●●
● ● ●● ●
●●●
●●
●
●●
●
●
●●
●
● ●●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●●
● ●●
●●●
●●
●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●●
●●
●●
●
●
●●●●●●
● ●●●
● ●● ●●
●
●●●
●
●
●●
● ●●
●
● ●●●●
●
●
●
●●
●●
●
●
●●
●
●●
●● ●● ●●●● ● ●
●●
● ●● ●
●
●
●
● ●●
●●
●●●●
●●
●●
●●● ● ●
●●
●
● ● ●●
●●● ●●●●
●●
●
M3 competition data
Best
29
Ets
●
●
●●
●●●●● ●
●●●
●●●
● ●●● ●●
●● ●
●
●
● ●
●
Predictability
−2
−4
−4
−2
−2
0
2
PC1
4
6
−2
Monthly data
4
6
2
●
●●
●
●●●
●
●
●●
● ●●
●
●
●●
● ●
●
●
●●
●
●●
●
● ●●
●
●
●
●●
●
●
●● ●
●● ●●
●
●
●●●
● ●
●
●●
●
●
●
● ●●
● ●
●●
●
●
●● ● ●
●
●
●
●●
●● ● ●● ●
●● ●●●
●
●
●●●●
●●
● ●●●
●
●●
●
● ● ●● ●
●●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●●●●
●●
●
●
●●●●
●
●
●●●●
●●
●
●
●●●●●●
●●
● ●●●
● ●● ●●
●
●
●●●
●●
●
●
●●
● ●●
●
●
●
●
●
●●
●●
●
●
●●
●
●●●
●
●● ●● ●●●● ● ●
●
●
●●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●● ● ● ●
●●
●
●
● ●●●
●
●● ●
●
●●
● ●
●●
● ●
●
●● ●●
●
●●
●●
●●
●
●● ●
●
●● ●
●
●●
● ●
●
●
●
●
●● ●
●
●
●●●
●●
●●
●●
●
●● ●●
●
●●
●
●
●● ● ● ● ●
●●●
●● ●
●●
●● ●
●●●●●●
●
●
●●●
●
●●
●
●
●
●
●
●●
●●●
●
● ● ●●●●
●●
●
●
●●
● ●●
●●
●
●
●●
●● ● ●
●●● ●●
●
●
●
●●
●●●
●●
●
●
●●
●
●●
●●
●
●
● ●●●● ●● ●●●● ●●● ●●● ●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
● ●
●
●
●●
●●●
●
●
● ●●
●
●●●
●●●● ●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●
● ●
● ● ● ●●
●
●
●●
●
●●
● ●●
●●
●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●●
●
● ●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●●●●
●●
●
●
●●●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●
●● ●
●●
●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●
● ●
●●
●
●
●●
●●●
●
●
●
●●
●●●
●
●
● ●●●●
●
●●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●●
● ●●●●
● ● ●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●●
●
●
●●●●
●
●
● ●●
●
●●●●●
● ●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●
●
●
●●
●
●
●●
● ●●
● ●●
●
●
●
● ●
●
●●● ●● ●●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
● ●● ● ●
●●●
●
●●
●●●● ●
●
●
●
●● ● ●
●●●
●
●
●●
●
●●
●●
● ●●●●●
● ●
●
● ●●
●● ●● ●● ●
●● ●
●●●
●●
●
● ●
●
●
●●●● ●●
●
●
● ●
●
●
●
● ● ●● ● ●
●
●●●●
●
●● ● ●●
●●
● ●● ●
●
●
●●
● ●
●●
● ●
● ●●●
● ●
●
● ● ●● ● ● ●
●
●
●
●●● ●
●
● ●● ● ● ●
●●
● ●
●● ●●●
●●
● ●●● ● ●●●
●● ●
●●● ●
●● ● ●●
●● ●●●●
● ●
●●
●
●●
●●
●●
●
● ●●● ● ●
●
●●●
●● ●●● ●
●
●● ●●
●
●
●●●
● ●●
●●●●
●●●
●
●
●● ●●
●●
● ●
●●●●●
●●●●●
●
● ●●
●
●● ●
●●
●
●● ●
●●●●
●
●
●
●
●●● ●●●
●
●
● ●●●
●
●
●
● ●
●●●
●● ●● ●
●●● ●● ●
●
●
●
●
●
Best
Ets
NoDiff
Stlm−ar
Theta
●
●
●
PC2
PC2
2
PC1
4
2
−2
0
Monthly data
4
0
●●
●●●●● ●
●●●
●●●
● ●●● ●●
●● ●
●
●
● ●
0
●
−2
−4
●
●●
●
●●●
●
●
●●
● ●●
●
●
●●
● ●
●
●
●●
●
●●
●
● ●●
●
●
●
●●
●
●
●● ●
●● ●●
●
●
●●●
● ●
●
●●
●
●
●
● ●●
● ●
●●
●
●
●● ● ●
●
●
●
●●
●● ● ●● ●
●● ●●●
●
●
●●●●
●●
● ●●●
●
●●
●
● ● ●● ●
●●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●●●●
●●
●
●
●●●●
●
●
●●●●
●●
●
●
●●●●●●
●●
● ●●●
● ●● ●●
●
●
●●●
●●
●
●
●●
● ●●
●
●
●
●
●
●●
●●
●
●
●●
●
●●●
●
●● ●● ●●●● ● ●
●
●
●●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●● ● ● ●
●●
●
●
● ●●●
●
●● ●
●
●●
● ●
●●
● ●
●
●● ●●
●
●●
●●
●●
●
●● ●
●
●● ●
●
●●
● ●
●
●
●
●
●● ●
●
●
●●●
●●
●●
●●
●
●● ●●
●
●●
●
●
●● ● ● ● ●
●●●
●● ●
●●
●● ●
●●●●●●
●
●
●●●
●
●●
●
●
●
●
●
●●
●●●
●
● ● ●●●●
●●
●
●
●●
● ●●
●●
●
●
●●
●● ● ●
●●● ●●
●
●
●
●●
●●●
●●
●
●
●●
●
●●
●●
●
●
● ●●●● ●● ●●●● ●●● ●●● ●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
● ●
●
●
●●
●●●
●
●
● ●●
●
●●●
●●●● ●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●
● ●
● ● ● ●●
●
●
●●
●
●●
● ●●
●●
●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●●
●
● ●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●●●●
●●
●
●
●●●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●
●● ●
●●
●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●
● ●
●●
●
●
●●
●●●
●
●
●
●●
●●●
●
●
● ●●●●
●
●●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●●
● ●●●●
● ● ●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●●
●
●
●●●●
●
●
● ●●
●
●●●●●
● ●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●
●
●
●●
●
●
●●
● ●●
● ●●
●
●
●
● ●
●
●●● ●● ●●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
● ●● ● ●
●●●
●
●●
●●●● ●
●
●
●
●● ● ●
●●●
●
●
●●
●
●●
●●
● ●●●●●
● ●
●
● ●●
●● ●● ●● ●
●● ●
●●●
●●
●
● ●
●
●
●●●● ●●
●
●
● ●
●
●
●
● ● ●● ● ●
●
●●●●
●
●● ● ●●
●●
● ●● ●
●
●
●●
● ●
●●
● ●
● ●●●
● ●
●
● ● ●● ● ● ●
●
●
●
●●● ●
●
● ●● ● ● ●
●●
● ●
●● ●●●
●●
● ●●● ● ●●●
●● ●
●●● ●
●● ● ●●
●● ●●●●
● ●
●●
●
●●
●●
●●
●
● ●●● ● ●
●
●●●
●● ●●● ●
●
●● ●●
●
●
●●●
● ●●
●●●●
●●●
●
●
●● ●●
●●
● ●
●●●●●
●●●●●
●
● ●●
●
●● ●
●●
●
●● ●
●●●●
●
●
●
●
●●● ●●●
●
●
● ●●●
●
●
●
● ●
●●●
●● ●● ●
●●● ●● ●
●
●
●
●
●
Best
Ets
NoDiff
Stlm−ar
●
●
●
−4
−2
0
2
PC1
4
6
Actual
−2
0
2
PC1
4
6
SVM prediction
M3 competition data
29
Generating new time series
We can use the feature space to:
å Generate new time series with similar features to
existing series
å Generate new time series where there are “holes” in
the feature space.
Let {PC1 , PC2 , . . . , PCn } be a “population” of time
series of specified length and period.
Genetic algorithm uses a process of selection,
crossover and mutation to evolve the population
towards a target point Ti . p
Optimize: Fitness (PCj ) = − (|PCj − Ti |2 ).
Initial population random with some series in
neighbourhood of Ti .
M3 competition data
30
existing series
the feature space.
M3 competition data
30
existing series
the feature space.
M3 competition data
30
existing series
the feature space.
M3 competition data
30
existing series
the feature space.
M3 competition data
30
Evolving new time series
3
●
●
●
●
●
2
1
●●●●
●
●●
●
● ●
● ●
●● ●
●
●
●
●● ●
● ●
●
● ●●
●
●●
●
● ● ● ●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
● ● ● ●
●●
●●
● ● ●●●
● ●●●●
●
●● ●
●
●
●
●
●
●●
●●● ●
● ●● ●● ●
●
●
●
● ●
●
●
●●●
●● ●● ● ●
●
●
●● ●● ●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ●●
●●
●
●
●
●
●
●●
●
●●● ●
●
●
●
●●
●
●
●●
●●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●● ●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ● ● ●● ●●
●
●
●
●●
●
●
●
●
●
●
●●● ●● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●● ●
● ●
●
●
●
●
●
●●
●● ●
●● ●
●● ●
●
●
● ●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
● ●● ● ● ●
●
● ●
●●
●●
●●● ●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●● ● ● ●●
●● ●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●●● ●●●
●
●
●
●
●● ●● ●●
●
●●
●
●
● ●
●
●
●
●●
● ●●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●●
●
●
●
●
●
●●●
●
● ●
●
●●
● ●
●
●
●
●●
●●
●
●
●
●
●● ● ● ●
●
●
●●
●●●●
●
● ● ●
●
●
● ●●
●
●
●● ● ●
●
●●
● ● ●●
●
●
●●●
●●
● ●●●
●
●
● ●●
●
●●
●●
●
●
●● ●
●
● ● ● ●●● ● ● ●
●●●
●●●● ● ● ● ●
●●
●●
●●
● ●●●
●●
●
●●●
●●●●●●
●
●
●
●
●● ●
●
●
●
●
● ●●● ●
●
●
●●
●
●
● ●●
● ●●●
●
●
●
●●●● ●
●
●
●
●
●
●● ●
●
●●● ●●
● ●
●● ●● ●
●
●
●
●●
●●
● ●●
●
●●●
●
● ●● ●
●●
●
●●
●●
●
●●
● ●
●
●
● ● ●● ●
●
●●
●●●
● ● ●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
● ●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
● ●●
●●
●●
●●
●●
●
●
●
●
●
●● ● ●● ● ●● ●●● ●●●
●
●
●
●● ● ●●●●● ● ● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●● ●
●●
●●
●
●
●
●
●●●●● ●●
●●
●
●
●
●
●
●
● ● ●
●●
●
●●
●
●● ●● ● ●●● ●● ●
●● ● ● ●
●●●
● ●●
●
●● ● ● ●
● ●● ●
●●●
●
● ●● ●
●●
● ●●●
●●● ●●●
●
● ● ●●
● ●●● ●● ●
● ● ● ●
●
●●
●●
●
●
●
●●
●● ●●
● ●
●
●● ●
●
●
●● ●
●
● ●● ● ● ●●
●
●
●
●●● ●● ●
● ●●● ●
●
●
●
●
●● ●
●
●
●
● ●●
●
●
● ●● ● ●
●
●
●
● ●
● ●
●
●● ●
●●● ● ●●
●●●
●
●
● ●
●
●●
●
●●
●●
● ●●
● ●
●● ●●
●●●●
●
●
●●
●● ●
●
●
●●
●
●
●● ●
●●
●
● ●
●● ●● ●●
● ●
●
●●
●●
● ●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
● ●
●●●
●
●● ●
●
●
●● ●● ●●
●
●
●●●●● ●●●● ● ●
● ●●
●
●
● ●●
●● ●
●
● ●● ●●●● ●●
●●
● ● ●●
●
●
●● ●
● ● ●●
●
● ● ●
●● ● ●● ●
●●
● ●
●
●● ●
●
●
● ●● ●
●
●
● ●
● ●●
● ●●●
● ●
●● ●●●●●● ●●
●
●● ●
●
●●
●
●
●● ● ●
●●
●
● ●
●
●
●●
●●●● ●●●● ●
●● ●
● ● ●● ● ●● ● ●
● ●
●
●●
● ●●●
●
●● ●
●●
●●●
● ●●
● ●● ● ● ●
●
●
● ●
●●
●
●●● ●●●● ●●●● ●●●●●● ●●
●
● ●● ●
● ● ●
●
●●
●●
●●●
● ●●
● ●●
●●●
●
●
●
●
●●
●
● ● ●● ● ● ●● ●● ●●●
● ●
● ● ●● ●●● ● ●
●●●●
●
●●●●●
●
●
●
●
●
●
●●
●●●
●●
●●●
● ●● ●● ●●
●
●
●●
●
●
● ●
●
●●
●●
●
●
● ●●
● ●
● ●
●
●
●●●
●●●
●●●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ● ● ●● ● ●
●●● ●●●●
●
●
●● ●● ●●
●● ●
●●●●●
●
●
●●●
●
●
●
●●●
●
●
●
● ● ●● ● ●
●●
●●
●
●
●●
●●●
●
●
●●
●
●● ●
●
●
● ● ● ●●
●●●● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●● ●
●● ● ●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●● ● ●
●● ●●●● ●●● ●
●
●●
● ● ● ●
●
●
● ● ●●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ● ●
●
●
●
●
●
●
● ● ●
● ● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●● ● ●●●●●● ●● ●● ●●● ● ● ●
●
●
●
●
●
●
●
●
●
●● ● ●●
● ● ●
●
●
●
●
●●●●●
●
●
● ●● ●
●●
●●●
●●
●●
●
●
●●
●● ●
●●
●
●
●
●
●
●
● ●●
●●
●●
●●
● ●
●
●●
●●●
●
●
●
●
●●
●●
●● ●●
●●
●●
● ●
●
●
●
●
●
● ● ● ● ●
●
● ● ●●
● ●●
●
●●
●
●
●
●
●●● ●
●
● ●●
● ●● ● ● ●
● ●●
●● ● ●●●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
● ●
●
●
● ●
●
●
●● ●
●●●●
●●
●●
● ●●
●
●●
●
●● ● ●
●●●
● ●
● ● ●● ●●
●●
● ●●● ● ●●
● ●●
●
● ●● ● ● ●
●●
●
●
● ●●
●
●
●●
●●●
●
●● ●
● ●
● ●
● ● ●
●●
●● ●
●
●
●
●
● ●
●●
●●
●
●
●●
● ● ●
●●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
● ●
●
●
●●
● ●
●
●
●
●● ● ●
● ●● ●
●
●
●
● ●
● ●● ●
●
●● ●●
●●
●
●
● ●● ● ● ●
●
●
●
●
● ● ●
●
●
●●
● ● ●●
●
●● ●●●●● ●
●
●●
●● ●
●●
● ●
●●
●●
●●●
● ●●●
●●
●
● ●
●●●
●
●●
●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●● ● ● ●
●
●● ●
●
● ● ●
●●●
●
●
●
●● ●● ●●
●
●●
●● ● ● ●
●
●
●
●
●
●
●
●● ●
● ●●
● ●
●● ● ●●
●●
●
●
●●
●
PC2
A
B
0
−1
−2
C
●●
●
−3
−2
0
2
4
PC1
M3 competition data
31
5200
4400
1970
1980
1990
0
Evolved B
5000
10
1985
1990
1995
5
20
25
30
10
3000
5000
Evolved C
4000
2000
15
Time
7000
1980
Target C
15
5000
3000
5
7000
1960
7000
1950
Target B
4800
Evolved A
6000
2000
Target A
1982
1984
1986
1988
1990
1992
1994
0
5
10
15
M3 competition data
20
25
30
31
3
●
●
●
●
●
●
●●●●
●
●●
●
● ●
● ●
●● ●
●
●
●
●● ●
● ●
●
● ●●
●
●●
●
● ● ● ●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
● ● ● ●
●●
●●
● ● ●●●
● ●●●●
●
●● ●
●
●
●
●
●
●●
●●● ●
● ●● ●● ●
●
●
●
● ●
●
●
●●●
●● ●● ● ●
●
●
●● ●● ●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ●●
●●
●
●
●
●
●
●●
●
●●● ●
●
●
●
●●
●
●
●●
●●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●● ●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ● ● ●● ●●
●
●
●
●●
●
●
●
●
●
●
●●● ●● ● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●● ●
●● ●
●● ●
●
●
● ●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
● ●● ● ● ●
●
● ●
●●
●●
●●● ●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●● ● ● ●●
●● ●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●●● ●●●
●
●
●
●
●● ●● ●●
●
●●
●
●
● ●
●
●
●
●●
● ●●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ● ●
● ●●● ●●
●●●
●
●
● ● ●
●
● ●●
●
●● ● ●
●
●●
● ● ●●
●
●●●
●●● ● ●
●●
●
●
●
● ●●
●
●●
●●
●
●
●● ●
●
● ● ● ●●● ● ● ●
●●●
●●●● ● ● ● ●
●●
●●
●●
● ●●●
●●
●
●●●
● ●●●●
●
●
●
●
●● ●
●
●
●
●
● ●●● ●
●
●
●●
●
●
● ●●
● ●●●
●
●
●●●● ●
●
●
●
●
●
●● ●
●
●●● ●●
● ●
●● ●● ●
●
●
●
●●
●●
● ●●
●
●●●
●
● ●● ●
●●
●
●●
●●
●
●●
● ●
●
●
● ● ●● ●
●
●●
●●●
● ● ●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
● ●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
● ●●
●●
●●
●●
●●
●
●
●
●
●
●● ● ●● ● ●● ●●● ●●●
●
●
●●● ● ●●●●● ● ● ● ●●
●
●●
●●
●
●
●
●●
●
●
● ● ●
●
●●
●● ●
●
●●
●
●
●
●
●●●●● ●●
●●
● ●●
●
●
●
●
● ● ●
●●
●
●●
●
●● ●● ● ●●● ●● ●
●● ● ● ●
●●●
● ●●
●
●● ●●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
● ●
● ● ●●
●
● ●● ●●
● ● ● ●
●
●●
●
●
●●
●● ●●
● ●
● ● ●●
●● ●
●
●
●● ●
●● ● ● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
● ●●
●
●
● ●● ● ●
●
●
●
● ●
● ●
●
●● ●
●●● ● ●●
●●●
●
●
● ●
●
●●
●
●●
●●
● ●●
● ●
●● ●●
●●●●
●
●
●●
●● ●
●
●
●●
●
●
●● ●
●●
●
● ●
●● ●● ●●
● ●
●
●●
● ●
●
● ●
●
● ●●●●● ● ●
●●
●
●●
●
●
● ● ●
●●
●●
● ●
● ●
●
●
●
●
●
● ●
●
● ●
●●●
●
●● ●
●
●
●● ●● ●●
●
●
●●●●● ●●●● ● ●
● ●●
●
●
● ●●
●● ●
●
● ●● ●●●● ●●
●●
● ● ●●
●
●● ●
●
●
● ● ●
●● ● ●● ●
●●
● ●
●
●● ●
●●
●
●●
●
● ●● ●
●
● ●
● ●●●
● ●
●● ●●●●●● ●●
●
●● ●
●●
● ●●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●● ● ●●
●● ● ●
●
●
●●● ●● ●
● ●
●●
● ●
● ●●
●●● ● ●●● ●
● ●●●
●
● ●
● ●● ● ● ●
●
●● ●
●
●●●
●●
● ●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●● ● ●●●
●●
●
●●●
● ● ●
● ● ●● ●● ● ●
●
● ●●
●● ●●
● ●●
●
●
●●
●●●
●
● ●●
●●●
● ●● ●● ●●
●●●●●
●●
●
●●
●
●
● ●
●
●
●●
● ●●
● ●
● ●
●
●
●●●
● ●●
●●●●
●●
●●
●
●●●
●
●
●
●
●●
●
●
●●●● ●●
●●●●
●
● ●● ●●
● ●●
●●
●
●●● ●● ●
●
●● ● ●
●●●
● ● ●●
●
●
●
●●
●●
●●
● ●
●
●
●
●
●
●
● ● ●● ● ●
●●
●
●
●●
●●●
●
●●
●
●● ●
●
● ● ● ●●
●
●●
●
●●●● ●●
●
●●
●●●
●
●
●●
●
●●
●
●● ●
●●●
●
●
●●
● ●
●
●
●
●
●
●
●●●
●●
●● ●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●●●
●
●
● ●●
●
● ●●●● ●●● ●
●
●
●●
● ● ● ●
●
●
●●● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ● ●
●
●
●
●
●
●
● ● ●
●●●● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●● ● ●●●●●● ●● ●● ●●● ● ● ●
●
●
●
●
●
●
●
●
●
●● ● ●●
● ● ●
●
●
●
●
●●●●●
●
●
● ●● ●
●●
●●●
●●
●●
●
●
●●
●● ●
●●
●
●
●
●
●
●
● ●●
●●
●●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
● ●●●● ● ● ●
●
●
●
● ● ●●
●
●●
●
●● ● ● ● ● ●●● ●
●
● ●●
● ●●
●● ● ●●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
● ●
●
●
● ●
●
●
●● ●
●●●●
●●
●●
● ●●
●
●●
●
●● ● ●
●●●
● ●
● ● ●● ●●
●●
●
●
●
● ●●● ● ●●
●
●
● ●● ● ●
●●
●
●
● ●●
●
●
●●
●●●
●
●● ●
● ●
● ●
● ● ●
●●
●● ●
●
●
●
●
● ●
●●
●●
●
●
●●
● ● ●
●●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
● ●
●
●
●●
● ●
●
●
●
●● ● ●
● ●● ●
●
● ● ●
● ●
● ●● ●
●● ●●
●●
●
●
● ●● ● ● ●
●
●
●
●
● ● ●
●
●
●●
● ● ●●
●
●● ●●●●● ●
●
●●
●● ●
●●
● ●
●●
●●
●●●
● ●●●
●●
●
● ●
●●●
●
●●
●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●● ● ● ●
●
●● ●
● ● ●
●●●
●
●
●
●● ●● ●●
●
●●
●● ● ● ●
●
●
●
●
●
●
●
●● ●
● ●●
● ●
●● ● ●●
●●
●
●
●●
D
2
E
PC2
1
0
−1
−2
F
●●
●
−3
−2
0
2
4
PC1
M3 competition data
32
7000
5000
3000
Evolved D
10
15
20
25
30
8000
12000
5
4000
Evolved E
0
10
15
20000
0
Evolved F
40000
5
2
4
6
8
10
M3 competition data
32
Targets
4
4
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●
● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ●● ● ● ● ● ● ● ● ●
●●●
●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●●●● ●
●
● ●●● ● ● ● ● ● ● ● ● ●
●
●● ●●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
PC2
● ● ●
● ● ●
0
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
−2
●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
● ●●
●
● ●●
●●
● ●
●
●
● ●
●●
●●
●
● ● ● ● ● ● ● ● ●●●●●●●●
●
●
●● ● ●● ● ● ●● ● ● ● ● ● ● ● ●
●●
● ●●
●
●
●● ●
●
●●
●
●
●●
●
● ●
●
●
●
● ●
●●
● ●
●●
●
●●
●
●
●●●
●
● ● ● ● ● ● ●●
●
●●● ●● ● ● ●●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●●
●
●●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
● ● ● ● ●●●●●●
●
●
●
● ●● ● ●●● ●
● ● ● ● ● ●
●●
●
●
● ●● ● ● ● ● ● ● ● ●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ● ●●
●
●
● ● ● ● ● ● ● ●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
● ● ●
●
●
●
●● ● ● ●●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
● ● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
● ●
●
● ● ● ● ● ● ● ● ● ●● ●
●
●● ●●●●
● ●●●
●
●
●
●●●●●● ●● ● ●
● ● ● ●● ●
●●
●
●
●●
●
●
●
●
● ● ●●
●
● ●
●
●
●
●
●● ● ● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●
●
● ● ●●
●
● ●
●● ●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
● ● ●
●
●
●●
●
●●● ●●●●●
●
●
●
●
●
●
●●●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●● ●
●
●●
●●
●
●
●●
●●● ●
●● ● ●
●
●
●
●
●
●
● ●●●
●
● ●●●
● ●●●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●●●
●
●
●●●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●●●
●
●●●● ●●●●●● ● ●
●●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
● ●●
●●●●
●●● ●
●●
● ●
●●
●
●●
●
●
● ●●● ● ●● ●● ●● ●
● ●●●●●●●●●
●
●
●
●
●
●●● ● ● ●
●
●
●●●●
●●● ●
● ●●
●● ●
●
●
●●
●
●
●●
●
●●●●●
●
●
●●
●
● ●● ●
●●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●●●● ●
●● ●
●
●●
●
●
● ●
●
●● ●● ● ●
●
●
●
●
●●●● ● ●
●
●●●
●
●● ●●
●
●● ●
●
●
●●●
●
●
●● ●●●
● ● ●● ●
●
●
● ●●
●●
●●
●●●●
●
●●
●
●●
●● ● ●● ●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●●●
●
●●●● ●
●
● ●● ● ● ●● ●●●● ●● ●●
●●●●●●●●●●
●
●●●●●●●●●●●●
●
●● ● ●
●●●
●●
●
●
● ●●●●
● ●●
●●
●●
● ●
●
●
● ●
●
● ●
●
●●●●●
● ●
●
● ●●●●● ● ●
●●
●
● ●●
●
●●●
●
●● ●●
●
●
●
●●●
●
● ● ●
●●● ●
● ● ●
●●
●● ●
● ●
●
● ●● ●
● ●●
●
●
●
●● ●
●●●●
●
●●
●●
● ●
●
● ●●
●
●●
● ●
●● ●
●● ● ●
●● ●● ●●
●
●
●
●
● ● ●● ●●
● ●
●●●
● ● ●
●●●● ●
●
●
●
●
●
● ●●●
● ● ● ● ●
●
●●●●● ●●
●
●
●●
●
●● ●
●● ●●
● ●
●● ●
● ●
●● ● ●
●
●●● ● ●●
●
●
●
●
●
● ● ●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●●
● ● ●
●● ●
● ●●●●
●
● ● ● ●●
● ●●
●●●
●●
●●
●
●● ●●
●
●
●
●● ● ● ● ● ● ●
●●
●
●●
●● ●
●● ●●●
● ●
● ●●
●
●
●
●●
● ●●●
●●●●●
● ● ●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●● ●
●● ●
● ●● ●●●
● ●● ●
● ● ●
●
● ● ●●
●
●
●
●● ● ●● ●● ●●●
●●●
● ●
●
●
● ●●
●●
●●●●
●● ● ● ●
● ● ● ●●● ●
●●
● ●
●●
●
●
●●
●●
●
●
● ●
● ● ●
● ●●●
●
●
●
●● ● ● ●
●
●
●●
●
●
●
●
●●●
● ●
●
● ● ●
●
●
●
●
●● ●●●●
●●●●●●●●●●●●●●●●●
●● ●● ●●●●●
●
●●
●
●
●
●
●●
●●●●
●
●
●
●
●●
●
●
●
●
● ● ● ● ● ● ● ● ● ●
●●
●
●
●
●
●●
●
●
●●● ● ●●●●●● ●
●
●
●
●
●
●
●
●●
●
●●
●● ●
●
●●●
●
●●
●●
●
●
●●
●●
●●
●
●
●
● ●● ●●
●●
● ●●
● ●
●
●
●
●●
●
●● ●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●●
●●● ●
●●
●● ●
●
●
●
●●●● ●●
● ●●●
●
●
●
●●
●
●●
●●●●
●
●
●
●●
●
●
●●● ●● ●
●
●
●●●●● ●●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●●●●
●
●●
●
●● ●
●
●
● ●
●
●
● ● ●●●●●●
●
●
●
●
●
●●●●●●●●●●● ●
●
● ●● ●● ● ● ● ● ● ● ●
●
●
●● ●
●
●●
●
●●●●●●●●
●
●
●
●
●●
●
●
●
●●
● ●
●●
●
●
●●
●●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●● ●
●
●
●
●
●● ●●
●
●
●
●
●
● ● ●
●
●●
● ●●
●
●●●
●●
●
●
●
●●●
●
●
●●●
●
●
● ●●
●●
●
●
●●●●
●
●
●
●
●●
●
● ● ●
●
●
●
●
●
●
●
●
●
●●●
● ●●
●
●●●
●
●
●
●●●●
●
●
●
●●
●●
●●
●
●●●
●● ● ●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
● ● ●● ●●●
●
● ●●
●
●●●
● ●●●● ●● ●
●●
●● ●
●
●●● ●
●● ●
●
●●
●
●●
●
●
●●
●●●
● ● ●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●● ●●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ●●
●●
● ●●
●
●●●●
●● ●
●●
● ● ●● ●●●●
●
●
● ●
●
● ●●●●
●●
●
●
●●
●
● ●●
● ●●
● ●●
● ● ●● ●
●●
●● ●●
●
●● ●● ●
● ●
●●
●● ●●
●● ● ● ● ● ● ●
●●● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ●
● ●
● ●●● ● ●● ●● ●
●● ●●
●●
●
●
● ●
●● ●
●
●
●
●
●● ●●●
● ●● ●
●
● ● ● ● ●●● ●
●● ●● ●● ●● ●
● ●● ●
●
●●
● ●●
●●● ●
● ● ● ● ●●
●●●
●
● ●●
●●●●●
●● ● ● ● ● ● ● ● ● ● ●
● ●● ●
●
●● ●●●
●● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
● ●●●●
●
●
●●●
●
●
●●
● ●
●● ●
● ● ● ●
●
● ●● ● ●
● ●● ● ● ● ● ● ● ● ● ● ● ● ●
● ●● ● ●●
●
●
●● ● ● ●
●
●
● ●
●
●●●
●
● ●●●
●
●●
●
●
● ● ●●● ● ●
●
● ●● ●●
●
●
● ● ● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●●
●
●●●
● ●
●● ●
●
● ● ● ● ●●●●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ●
2
● ● ●
● ● ●
● ● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
PC2
2
●
● ●
●● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●
●
● ●●●●
●
● ●●●
0
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
● ● ●
● ● ●
● ● ●
●
● ● ●
● ● ●
● ● ●
●
−2
● ● ●
● ● ●
● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
−4
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
−2
0
2
PC1
4
−4
6
M3 competition data
−2
33
Evolved yearly data
4
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ●● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
2
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ●
● ● ● ● ● ● ● ●
●●
● ● ● ● ● ● ● ●
PC2
●
●● ● ● ●●● ●
●
●
●●
● ● ● ●
●
●●
●
●
●●●●●● ●● ● ●
●●
●
●
●
●
●●
● ● ●●
●
●●
●
● ●
●● ●
●●
●●
● ● ●
●●
●
● ●
●● ●
●
●
●
●●●
●
●
●● ●●●●●● ● ●
●●
●●
●●
●●● ● ● ●
●● ●
●
● ●●●
●
●● ●
●● ● ●
●●
● ●
●
●
0
●
● ● ● ● ● ● ● ●
●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
−2
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
●
●●
●
●
●
●
●
●
●●●●●●
●●
● ●●● ● ●
●●●
●●
●
●●●●●
●
●
●●●
●
●●
●
● ●● ●●●●●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ● ●
●
●
●●
●●
●
●
●
●
● ●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ● ●●
●●
●
● ●
●
●
●
●●● ●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●●●
●●
●●●
●● ●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●● ● ●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●●
●
●●
●
●
●
●
●
●
● ●●● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●●●● ●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
● ●●
●●
●
●
●
●●
●
● ●
●
●
●●
●
●● ●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●●●
●
● ●● ●●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ● ●●
●
●
● ●
●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●● ●●
● ●●
●
●●
●
●
●
● ●
●
●●● ●●●●
●
●
●●
●●
●
●●
●●
●
●●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
● ●●
●
●
●
●
●●
●●
●
●
●
●
● ● ●●●
●
●
●
●
●
●
●
●●●●
●
● ●●●
●
● ●●●
● ●●●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●●●●
●
●
●●
●
●
●
●
●● ●
●
●
●
●● ●●
●●●
●
●●●● ●●●●●● ● ●
●●●
●
●●
●●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●●●●
●
●●
●
●
●
●●●
●●●
●
● ●
●
●
●
●●●●
●●●●
●
● ●●●
●
●
●●
●
●
●●
●
● ●●
●
●
●●●●
●
● ●●● ●
●●●
●
●●
●
●
●
●
●
●●●●●
●
●
●●
●
● ●● ● ●●●●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●● ●
●●●
●
●●
●
●
● ●
●
●● ●● ● ●
●
●
●
●
●●●● ● ●
●●●
●
●
●●●
●
●
●● ●●
●
●● ●
●
●
●●●
●
●
●
● ● ●● ●
●
●
●
●
●●
●● ●●●
●●
● ●●
●
●●●●
●● ● ● ●●●●● ● ●
●
●●
●
●
● ●●
●
● ●●
●
●●
●
●●
●●● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
● ●
●
●
● ●●●● ● ●
●●
●
●●
●
●
●●●
●
●
●
●
●
●●●
●
● ● ●
●●● ●
● ● ●
●●
●● ●
● ●
●
● ●● ●
● ●●
●
●
●
●● ●
●●●●
●
●●
●
●
● ●
●
● ●●
●
●●
● ●
●● ●
●●
●
●
●
●
●
●
●
● ●
●●●
● ●●
●
● ● ●
● ●
●●●●
●
●
●●
●● ●
●● ●●
● ●
●● ●
● ● ●●● ●
●●● ● ●●
●●●●
●● ●
●
●●
●●● ● ● ●
●
●●
●
● ●●●
●● ●●
●
● ●●
●●
●
●● ●
●
●● ●
●●●
● ● ● ●●
● ● ●●
● ●
●●●
●
●
●
●● ●●●●
●
●
●
●●
●
●●
●● ●●●●
● ● ●●
● ●●
●●
●
●
●●
● ● ●
●●●●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●● ●
● ●● ●●●
● ●
● ●● ●
● ● ●● ●
●
● ● ●●
●
●
●
●
●● ●● ●● ●● ●●●
●●●
● ●
●
●
●●●
● ●●
●●
●
●● ● ● ●
● ● ●
●●
●
● ●
●●
●
●
●●
●●
●
●
● ● ●
●●● ●
●●
● ●●●
●
●
●
●
●
●●●●●
●● ●
●
●●
●
●
●
●
●●●
● ●
●● ●
●
●●
●
●●
●●●●
●●
●●●●
●
●
●
●
●●
●
●
●● ●
●
●
●●●
●
●
●●● ●●
●
●●
●
●●● ● ●●●●●● ●
●
●
●
●
●
●
●●
●
●●
●●
●●
●●
●● ●
●
●●●
●
●●
●
●
●●
●●
●
●
●
● ● ●●
●●
● ●●
● ●
● ●
●
●
●
●●
●
●● ●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●●
●●
●
●●
●
●
●
●●●● ●●
●
●●● ●
● ●●●
●
●
● ●●
●
●●
●
●●●
●●●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●●
● ●
●●
●
●
●
●
●●●
●
●● ●
●
●
● ●●● ● ●●●
●
●●
●
●●●
●
●
●●
●
●● ●
●
●
●
● ●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●● ●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●● ●●
●
●
●
●
●
● ●
●
●●
● ●●
●
●●●
●●
●
●
●
●●●
●
●
●●●
●
●
● ●●
●●
●
●
●●●●
●
●
●
●
●●
●
●
● ● ●
●
●
●
●
●
●
●
●
●●●
● ●●
●
●●●
●
●
●
●●●●
●
●
●
●●
●●
●●
●
●●●
●
●●
●
●
●
● ●
●
● ●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
● ● ●● ●●●
●
●
● ●●
●
●●●
● ●●●● ●● ●
●●
●● ●
●
●●● ●
●● ●
●
●●
●
●●
●
●
●●
●●●
● ● ●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●● ●●
●●
● ● ●●
●●
●●
●
● ● ● ●●
● ●●● ● ●●● ● ●
●●
●● ●●●
●
●
●
●
●
● ●●
●
●● ●
● ● ●● ●
●● ●●
●
●● ●● ●
● ●
●●
●● ●●
●● ● ● ● ● ● ●
●● ●
●
● ●●
●
●●
● ●
●
●
●
● ●
●
●
●
●
●
●● ●●●
● ●● ●
●
● ● ● ● ●●● ●
●● ●● ●● ●● ●
● ●
●
●
●●
● ● ●
●●
●
●●
● ● ●● ● ●●
●●
●● ●●●●● ●
●
●
●●●
●●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
● ●●●●
●
●●
●
●●
●●●●
●
●● ●
●
● ●●
●
●● ● ● ●
●
●
● ●
●● ●
●●●
●
●●●●●
●
●●
●
● ●
● ● ●●● ● ●
●
●
● ●
●
●●
●
●●●
● ●
●●● ●
●
●●
●
●
●
●
●
●
●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
4
6
−4
−2
0
2
PC1
4
6
M3 competition data
33
−4
−4
Evolving new time
series
PC1
−2
0
2
4
6
−2
Evolved quarterly data
E
4
4
●
0
−2
2
PC2
PC2
2
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●●●●
●
●●
●●
●
●
●●●
●●
●
●
●
● ●●
●
●
●●●
●
●
●
● ● ● ● ●
● ●
●●
●●
●
●●
●
●● ●
●●
●●
● ●
●●
● ●●
● ●●
●●
●●●● ● ● ● ● ● ● ●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●● ●●● ●
●●
●●
●
●●●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ●
●● ●
●
●●
●
●
● ●●●●
●●●●
●●
●
●● ●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ● ●● ● ●
●
●
●
● ●
●
●
●●
●●●
● ●
●
●●●●
●
●●●
●
●●
●
● ● ●
●
● ●● ● ● ● ● ● ● ● ●
●
●●●
●
●
●
●
●●
●●●
●
●
●
●
●●
●●
●
●
●●
●
●●
● ●
●●
●●
●●
●
●
●
●
●
●●●●
●
●● ●● ●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
● ● ● ●● ● ● ● ● ● ● ● ● ●
●●●●
●
●●●● ● ●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●
●
●
●
● ●
●
●
● ● ●●● ● ●● ●
●●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●● ● ●●●● ● ●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
● ● ● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●● ●●●
●
●
●
●●
●
●
●● ●
●
●● ●●●
●● ● ● ● ●● ● ● ●● ●●●●
●
●
●
● ●●
●
●
●
●
●● ●
●
●● ●●
●
● ●
●
●●●
●
●
●
●●
●●
●
●●
●●
●
●
● ●●
●
● ● ●●●
●
●
●
●
●
●
●●● ●
●
●● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●
●
●
● ●●
● ●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●● ●
●
●
●
●● ●●
●
●
●
●
●
● ●●●
●
●●● ●●●●
● ●● ●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
●●
●●
●
●
●
●
●
●
●● ●●●●● ●●
●
●
●
●
●
● ●●
●
● ●●●
●
●
●
● ● ●●●
●
●
●
●
●●
●
●
●
●
●
●●●● ●● ●● ●●●●●
●●●
●
●
●●●
●
● ●●
●
●
●
●
● ●
●●●
●
● ●●●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●● ●
●
●● ● ● ● ● ●
●
●
●● ●●
●●●
●
●●●● ●●●●●● ● ●
●
●
●●
●●●
●●
●●●
●
●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●●
●
●●●●
●● ●
● ●●●
●
●
● ● ●●●●●●
●
● ●● ●
●●
●
● ●●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●●
●●
●
●
● ● ●●
●●●
●
●
●●●
●
●● ●
●
● ●
●
●
●●
● ● ● ●● ● ●
●●
●
●● ●● ●
●
●
● ● ●●
●
●
●
●● ●●
●●●●●● ●
●
●● ●
●
●
●●●
●
●
●● ●●
●● ● ●●
●
●
●●●
●
●
●●●
●
●● ●●
●
● ●●
● ●●
●
●●
●● ●●●
●●
● ●●
●●●●
●●
●●
●
●
●
●
●
●●●
●
● ●
●●
●
●●● ●●
●
●●
●
●
●●
●
● ●●
●●●
●
●
● ●
●
● ●● ●● ● ● ●●● ● ● ●
●
●
●●
●
●●
● ● ● ●●● ● ●●
●
● ●● ●●●● ● ●●●●● ●●
●
●●
●
●
●●
● ●
●●●
●
●●●●●
● ●
●●●●
●
●●
●
●
● ●● ●
● ● ● ●●
●
●●
●
●●
●
●● ●●
● ●●
●
●
●
●●●● ●
●●●
● ● ●
●●● ●
●●● ●
●●
●● ●
● ●● ●
● ●●
●
●
●
●
●● ●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
● ●
●
●● ●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●●●
● ●
●
●
●
●
●
● ●● ●
●
●●●●
●
●
●●
●●●
●
●
● ● ●
●●● ●●
●
●
● ●
●●● ●
●
● ● ●
●●●●●●●●● ●●
●● ●
●●
●
●
●●● ●●
●● ●●●●●●●
●
●●●●
●
●●
●● ●● ●● ● ●
●
●●
●
●●
●●
●●
●● ●●
●
●
● ●
●● ● ● ●
●●
●
●● ●
●● ●
● ● ●●
● ●
●●
●
●●●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●
●
●●● ●●●
●
●● ●● ●
●● ●
● ●●● ●
●
●
●● ●
●●● ●
●●
●●
●
● ●
●
● ● ●● ●●
●
●●●●● ●●●●
●●●●●
●
●●● ●
●●
●
● ●
●
●
●
●●
● ●● ●●●● ●
●
●
● ●
●
●●
●
●
●
●
●
●
●● ●● ●● ●●
●●●
●●●
●
● ●●
●●
●●●●
●
●
●●●● ● ●
●
● ●●
●●
●●●
● ● ● ●●
●
●● ●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
● ●●
●●●● ●
●●
● ●●●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●●●
● ●
●● ●
● ●●●●
●
●●
●
●●
● ●
●●●●
●●● ●
●●
●●●●
●●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●● ●●
●●
●
●
●●●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●●
●●
●● ●
●
●●●●●●●●●●●
●
●
●●●
●
●
●●
●
●
●
●●●●
●●
●
●
●
● ●●
●
● ●●
●
●
●
●
●●
●
●
●●●●
●
●●●
●
●
●
●●
●
●●●
●
●
●●●●●●
●
●
●
●
●●
●
●
●●
●
●
●●●
●●
●●
●
●
●●
●
●●●● ●●
●
●●●●●
●
●
●●
●
●
●●
●
●●
●
●●●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
● ●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
● ●●● ●●
●
●●
●
●●●
●
●
●●
●
●
●
●● ●
●
●
●
● ●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ●
●
● ●
● ●●
●
●
●●
●
●
●
●
●
● ●●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●●
●
● ●
●
● ●● ● ●
●
●
●●●● ●● ●
●
●
● ●●
●
●
●●
●●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
● ●●●● ●
●●
●●●●
● ●● ● ● ●
● ●
●
●●● ●
●
●●●
●
●
●●
●
●●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●●
●
●●●●
●● ● ●●
● ●
●
●
●
●● ●● ● ● ●
●
●● ●●
●●
●●
●●
●
●●
●●
●
● ●●
●
●
●●●●
●
●●●● ●●
● ●●● ● ●●● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●● ●●
●●●●
●●
●● ●
●
●
●● ●
●●
● ●
●
●
●
●●
●
● ●●
●● ●●
●●●
●
●
●
●●●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●●
● ●● ●
●
●
●
●
●
●
●●● ● ●● ●● ●●●
● ●
●● ●● ●● ●● ●
●
●
●
●
●●
● ● ●
●●
●
●●
● ● ●● ● ●●
●●
●● ●●●●● ●
●
●
●●
●● ●
●●
●
●
●●
● ●●●
●
● ●●
●●
●●
●● ●
●●
●
●
●●
●●●●
●
●
●
● ● ●
●
● ●
●●
●● ●
●
●
● ●
●● ●
●●●
●
●●●●●
●
●●
●
● ●
● ● ●●● ● ●
●
●
● ●
●
●●
●
●●●
● ●
●●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●●
●
●●●●
●
●
●●
●●
●
●
●
●●
●
●●
●
● ●
●
●●●
●
●
●
●
●
● ●
●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●●●●●●●●●●●●●●●●●●●●●
●●● ●
●
●●
●●
●
●●●●●●●
● ●●
●●
●●● ●●●●●
●
●
●● ●●
●● ●●
●●●
0
●
●
●
●
●●
●
●
● ●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
−2
−4
−4
−2
0
2
PC1
4
6
M3 competition data
−2
33
−4
−2
0 time
2
4
6
Evolving new
series
PC1
4
6
ata
Evolved monthly data
4
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●●●
●●
●●
●
●
●●
●
●
●●●
●
● ● ● ● ●
●●
●●
● ● ● ● ●
●
●
●
● ● ● ● ●●
●
●
● ● ● ● ● ●
●●
● ● ● ● ● ●
●
●
●
●●● ● ●● ●●●●● ● ●●
●
● ● ● ●●
●
●
●
●
●
●
●
● ●●
●●●
●●
●
●●
● ●●
● ● ●●●
●
●
●●
●
●
● ●
●● ●
●●
●● ●
●
●●
●●
●
●●
●
●● ●
●
●●●● ● ● ●●
● ●●
●●
● ● ●● ●
●●
●●●●
●
● ●●
●
●
●
● ●●●●● ● ●● ● ●
●
●
●
● ●●● ●● ●
●
●
●
● ●● ●● ● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
PC2
2
0
●
4
●
−2
● ●
●
●
●●●
●
● ●●
● ●
●●
●● ●
● ●
●
● ●●
●●
●●●●
●
●
●
●
●● ●●● ●
●
● ●●
●
● ●
● ●
●● ●●● ●
●●
●●
●
● ●●●●
●●●●
●
●
●● ●
●
●
●
●●
●
●
●
●●
●
● ●
●
●
●
● ●
●●
● ●
●
●● ●
●●
●●●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●●
●● ●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ●●
●
●
●●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●● ●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●●
●●
● ● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
● ● ● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
● ●●
●
● ●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●● ●●●
●
●
●
●●
●
●
●
●● ●
●●
●●
●
●●● ●
●●●●●●●●●
●
●
●
●
●
●
● ●
●
●
●●
●
● ●
●
● ● ● ● ●●
●
●
●●●●
●
●
●
●●
●
●
●●
● ●●
●
●
●
●●
●
●
●
●●● ●
●
●
●●
● ●●
●
●
● ●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
● ●●
●●
●
●
●
●
● ●
●
●● ●● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
● ● ● ● ●
●
● ●●●●●●●● ●●●●
● ● ●●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●●
● ●● ●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●●
●
●●●
●
●
●
●
●
●
● ●●●
●
● ●●●
● ●●●
●
●
●
●
●
●
● ●●
●● ● ● ●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●●
●
●
●
●● ●
●
●
●
●●
●●●
●
●●●● ●●●●●● ● ●
●●●
●
●●
●●
●●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●●
●
●
●
● ●● ●● ●
●●
●
●
●
●
●
●●
●●●
●
●
● ●
●
●
●
●●●●
●
●●●●
●
●
●●
● ● ● ● ●
●
●
● ● ●●●● ●
●
●
●● ● ●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●● ● ● ●● ●
●
●● ●
●
●
● ●●●●●
●
●
●
● ●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●●
●●●● ● ●
●
●
●
●
●●●
●
●
●● ●●
● ● ●●
●
●● ●
●
●
●●●
●
●
●
●
●
●
●●
●
● ●●
●
●
●●
●● ●●●
●●
● ●●
●
●●●●
●●
●●
●● ● ● ●●●●● ● ●
●●
●
●
●
●
●
● ●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●●●
●
●●
●
●●
●● ● ● ●● ●● ● ● ● ● ● ● ●
●
●
●
●●
● ● ● ●●● ● ●
●●●●●●●●●●●
● ●●● ●
●
●●●●
●●●●
●
●
●●
●
●
● ●
●
●
●
●● ●●
● ●●●●● ●
●
●●●
●●
●
●
●
●●●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●● ● ●● ●●●●●●●●●●● ● ●●●●●
●●● ●
●
● ●● ● ●
●● ●
●● ●
●●
●● ●
● ● ●
●
●
● ● ●
●●●●●●●●●●● ●●●●●●●● ●●●
● ●● ●
●
●
●
●●
● ●● ●● ●
● ● ● ● ● ●
● ● ●
●
●
●●● ●●
●
● ●
●
●● ●
●●● ● ●●
●●
●●●●
●
●●● ● ● ●
●
●●
●
● ●
●
●●
●
●●
●
●
●
●
●● ●●
●
●●● ●
●
●●
● ● ●●
●● ●
●
●● ●
●●
●
● ● ●●
●
●
● ●
●●●
●
●●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●● ●
●
●●
●● ●●●●
● ●●●
●
● ● ● ● ● ● ●
●
●
●● ●
●
●●
●● ●
● ●
●
●
● ● ● ● ●●
●●●●●●●●●●●
● ● ●
●
●
●
●● ●
●● ●●
● ●
● ●
●
●
●
●●
●
●
●
●● ●●●●●●●●● ●●●●●●●●●● ●●●
●
●
●
●● ●● ●● ●
● ●
●
●
●●●
● ●●
●
●
●
●●●● ● ●
●●
●
●● ●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●●● ●
●
●●●● ●
●
●●
●
● ●●●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●●●●
●●●
●●
●●●
●
●
●
●
●
●
●●
●● ●
●
●
●● ● ●●●●●● ●
●●
●
●
●
●
●
●●● ●
●
●
●
●●●● ●
●
●
●
●
● ● ● ● ● ● ● ● ●●
●●●●● ●
●
●
●●
●
●●
●●●
●●
●●
●
●
●● ●
●
●
●●●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
● ● ●●
●●
● ●●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
● ●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●●● ●
●●
●
●
●
●
●●●● ●●
●
● ●●●
●
●
● ●●
●
●●
●
●●●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●● ●
●●
●
●●
●
●
●●
●
●
●●
● ●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●●●
●
●●
●
●●
●
●● ●
●
●
●
● ●
●
●
●●●
●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
● ●●● ●
● ●● ● ● ●● ● ● ● ●
● ●● ●
●●
●
●
●
●
●
●●
● ●●●●
●●
●●
●
●
●
●●●●●
●
●
●
●
●
●●●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●● ●●
●
●
●
●
● ●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●
●
● ●●
●●
●●●●●
●
●
●●●●
●
●●●●
●
●
●
●
●
●●
●
●
●
● ● ●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●
●
●●
●●●●
●
●●
●●
●●
●●●
●●
●
● ● ●●
●
●
●●
● ●●●●●●●●●●●●●●●●●●●●●●●●
● ●●
●
●●
●●
●
●
●●
●
●
●
●●
●●
●
● ●● ● ●
●
●●
●
●
●●
●
●
●●
●
●
●
●
● ●●●
●
●
●●●
●
● ●●●● ●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●● ● ● ●
● ● ● ●●
●
●
●
●●●
●
● ●●●
● ●●●●
●●●●
● ●●
●
●
●
●●
●
●
●
●
● ●●
●
●●
●●
●
●●
●
●●
● ●
●
●
●●●●
●●
●
● ●● ●●●
● ●●● ● ●
●●
●
●
●
● ●
●●● ●●●●●
● ●
●● ●●●● ● ●
● ●●●
● ● ●
●
●●● ●●
● ●● ●● ● ●● ● ●
● ●● ●● ●
●
●● ●● ●●●● ●● ●● ● ● ●● ●● ●●● ●● ●●● ●● ●●●● ●
● ●
●●
●
● ●●●●
● ● ● ●● ● ●
●●●
●
● ●
●
●
●
●
●● ●●●
●
● ●●●●● ● ● ● ● ●●
●
●
●
●
● ● ● ● ●●● ●
●● ●● ●● ●● ●
●
●●
●
● ●
●
●●
●
●●●
●●●● ● ●
● ● ●
●● ●
●
●
●
●
●
●●
●
● ● ●● ● ●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●●
●●
●
●
●
● ● ●●●●
● ● ●●
●●
●
●
●
●●
●●
●●
●
●
●● ●
●
●
● ●●●● ●
● ● ●●
●●
●● ●
●●
●
● ● ●
● ●●
●
●
●
●
●●
●● ●● ●
● ●
●
●
●● ●
●● ● ●●
● ●●
●●
● ●
●
●
●
●
● ●●● ●●
●
●
●
●
●●
●
●●
●● ● ●●● ● ●●●●●●●●● ●● ●●● ● ●
●
●
●●
●
●●●●
●
●● ● ●●●
●●●
●● ● ● ● ● ●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ● ●● ●●● ● ● ● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●● ● ●● ●
●
●
●
●
●
● ●
●
●
●●
●
−4
6
−2
0
2
PC1
4
6
M3 competition data
33
Questions raised
Can SVM be used to create a forecast selection
routine to give better forecasts?
How much do M3 conclusions depend on the
particular set of time series involved?
Has the M3 data set biased forecast method
development?
What other features should we consider? What
difference does it make?
Is PCA the right approach? Perhaps we should
use multidimensional scaling? Or something
else?
Should we use more than 2 PC dimensions?
M3 competition data
34
Questions raised
development?
else?
M3 competition data
34
Questions raised
development?
else?
M3 competition data
34
Questions raised
development?
else?
M3 competition data
34
Questions raised
development?
else?
M3 competition data
34
Questions raised
development?
else?
M3 competition data
34
Outline
1 The problem
4 Yahoo web traffic
5 What next?
Yahoo web traffic
35
Yahoo web-traffic
Tens of thousands of time series collected at
one-hour intervals over one month.
Consisting of several server metrics (e.g. CPU usage
and paging views) from many server farms globally.
Aim: find unusual (anomalous) time series.
Yahoo web traffic
36
30
15000
25
40
25
0
35
600
200
70
20000
15000
10000
5000
0
50
50
date
memory484
busy369
60
memory413
25
memory147
15
busy200
20
memory429
20
busy50
60
memory460
30
value
20000
40
2014−11−09
2014−11−10
2014−11−11
2014−11−12
2014−11−13
2014−11−14
2014−11−15
2014−11−16
2014−11−17
2014−11−18
2014−11−19
2014−11−20
2014−11−21
2014−11−22
2014−11−23
2014−11−24
2014−11−25
2014−11−26
2014−11−27
2014−11−28
2014−11−29
2014−11−30
2014−12−01
2014−12−02
2014−12−03
2014−12−04
2014−12−05
2014−12−06
2014−12−07
2014−12−08
2014−12−09
2014−12−10
35
busy271
value
45
busy233
2014−11−09
2014−11−10
2014−11−11
2014−11−12
2014−11−13
2014−11−14
2014−11−15
2014−11−16
2014−11−17
2014−11−18
2014−11−19
2014−11−20
2014−11−21
2014−11−22
2014−11−23
2014−11−24
2014−11−25
2014−11−26
2014−11−27
2014−11−28
2014−11−29
2014−11−30
2014−12−01
2014−12−02
2014−12−03
2014−12−04
2014−12−05
2014−12−06
2014−12−07
2014−12−08
2014−12−09
2014−12−10
2014−12−11
2014−12−12
2014−11−09
2014−11−10
2014−11−11
2014−11−12
2014−11−13
2014−11−14
2014−11−15
2014−11−16
2014−11−17
2014−11−18
2014−11−19
2014−11−20
2014−11−21
2014−11−22
2014−11−23
2014−11−24
2014−11−25
2014−11−26
2014−11−27
2014−11−28
2014−11−29
2014−11−30
2014−12−01
2014−12−02
2014−12−03
2014−12−04
2014−12−05
2014−12−06
2014−12−07
2014−12−08
2014−12−09
2014−12−10
2014−12−11
2014−12−12
Yahoo web-traffic
10000
5000
400
1000
10
500
25000
20000
15000
10000
5000
0
date
Yahoo web traffic
date
37
Feature space
ACF1: first order autocorrelation = Corr(Yt , Yt−1 )
Strength of trend and seasonality based on STL
Trend linearity and curvature
Size of seasonal peak and trough
Spectral entropy
Lumpiness: variance of block variances (block size 24).
Spikiness: variances of leave-one-out variances of STL remainders.
Level shift: Maximum difference in trimmed means of consecutive
moving windows of size 24.
Variance change: Max difference in variances of consecutive
Flat spots: Discretize sample space into 10 equal-sized intervals.
Find max run length in any interval.
Number of crossing points of mean line.
Kullback-Leibler
score: Maximum of
R
DKL (PkQ) = P(x) ln P(x)/Q(x)dx where P and Q are estimated by
kernel density estimators applied to consecutive windows of size 48.
Change index: Time of maximum KL score
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
Feature space
Spectral entropy
Kullback-Leibler
score: Maximum of
R
Yahoo web traffic
38
●
●●
●●
●
●●●
●
●
●
● ●●
●● ●
● ●●
●
● ●●●●●●●
● ●●●
● ● ●●
● ●●
●
● ●
●●
●●
●
●
●
●
●●
●●
● ●●
●
●
●●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●● ● ●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ● ●●●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●● ● ●
●●● ●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
● ● ●●●●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●●
●
●
●●
●
●●● ●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
● ●●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●● ●●
●●
●
●●●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●●●
●
●●●
●
● ●●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
● ●●●
●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●●
●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●●
●●
●●
●●
●
●
●
● ●● ●●
●
●
●
●●●●
●● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●● ● ● ●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●●
●
●
●●●● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●●
●
●
●
●
● ●
●
● ●●● ●
●
●
●
●
●
●
●
●●● ● ● ●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●●
● ●
● ●● ● ● ●●● ● ● ●●
●
●●
●●
●
●
●●
●●●
●●●●●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●●●
●
● ●
●
●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●● ●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
● ● ●●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●● ● ●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ● ●● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●● ● ●
●●●
●●● ●
●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ● ● ●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●
●●
●●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
● ●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
● ●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
● ● ●●●
●
●
●
●
● ● ● ●
●● ●
●● ● ●
●●
● ●● ●
●●● ●●
●
● ●● ●●
●●
●
●
●●●●
●
●
●
●●
●●
● ●●
●
●●
●●
●
●● ● ●
● ●
●● ●●●●● ●
●
●
●
●
● ●●
●
●●●●
●● ●●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
● ●●
●
●
● ●
●
●●
●
●●
●
●●
●
●●●
●
●
●●
●●
●
●
●
●
●
●● ●●●
●
●●●●●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●●
●
● ●●●● ●
●
●
●
●
●●●
● ●●●
● ●
●
●●
●
●
●
● ● ● ●● ● ●
●●
●
● ●
●
●
●
●
●● ●
● ● ●●●
●
●
●
●●●●
●●
●●
●
●
●
●
●
●
●
●● ●●
● ●● ● ● ●● ● ● ●●
● ●●
●
●●
●
●
●
●● ●● ●● ●
● ● ●●●●●
●●
● ●
●
●
●
●
●
●
● ● ● ●●●
●
●
●
●●
●
● ●●●● ●
●● ● ●
● ●
● ● ●●
● ●
●
● ● ●●● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●●
●●
● ● ●● ●
●● ●
● ●● ●
● ●●●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ● ●
●●
●●
●
●
● ●
● ●
●
●
●
●●
●● ●
●
●
● ●●●
●
●
●● ● ●
●
●
● ● ●● ●
●
●
●
●
●
●●
●●●
●
●●
●
●
● ● ●●
●●
●●
● ●●
●● ●
● ● ● ● ● ● ● ●● ●
●
●
●
● ●●
● ●●
●
●
●●
● ●●●●●●●
●
●
●
●
●
● ●
●
● ●●● ●●
●
●
●
● ●
● ●●
●●
●
●
●
●
●● ● ●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●●●●●●●
●
●
●
●
●
●
●
●
●● ●
●
● ●
●
●●
●
●●
●
●
●
●●● ●
●
●
●
● ●
●
●●
●
●
●● ●●● ● ●●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
● ●●
●
● ●
●
●
●
●
●
●
● ●
●
●
−2
−4
n
sea
so
a
ch
n
x
.id
ge
ure
rvat
ckuls
core
s
eesss
n
i
ik in
spmp
u
l
h
0
troug
entro
py
ots
2
●
fsp
●
peak
s
ritynt
ea oi
lin cp
standardized PC2 (17.3% explained var.)
Principal component analysis
vlstcA
C
rhheianftnFg1e
d
●
●
●
●
−2.5
0.0
2.5
Yahoo web traffic
39
●
●●
●●
●
●●●
●
●
●
● ●●
●● ●
● ●●
●
● ●●●●●●●
● ●●●
● ● ●●
●●
● ●
●
●●
●●● ● ●
●
●
●●
●●
● ●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●● ● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●● ●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
● ●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ● ●●●●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●● ●● ●
●
●
●
●
●●●
●●●
●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●●
●
●●●
●
●●
●●
●
●
●
●● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
● ● ●●●●
● ●●●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●● ●●
●●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●
●
● ●●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
● ●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●● ●●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
● ●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
● ●● ●●
●
●
●● ●
●
●●●●● ●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●●
●● ●
●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●●●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
● ●
●
● ●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●●
●●
●
●
●
●
●●
●
●●
●●
●●●● ●●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●● ●● ● ● ●● ●●●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●●●●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●●●
●
● ●
●
●
●
●●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●
●● ●●● ●●
●
●
●
●
●●
●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●● ●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●● ●
● ● ●●
●● ●●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ● ● ●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
● ●●
●●
●
●
●●
●●
●
●
●
●
●
●●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
● ● ●
●
●
●
●●
●●
●●
●
●
●
●
●● ●●
●
●
●●
●
●
●
●
●
● ● ●● ● ●
●
●
●
●
●●
●
●
●●● ●●
●
● ●●
●
●
●●
●
●
●●●●
●
●●●
●
●
● ● ● ● ●●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
● ●● ●
●
●●
●●
● ● ●
●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●● ●●
●
● ●●
●●
●●● ●●
●
●
● ●●
●●
●●
●
●
●●●●
● ●●
●
●
●
●●●
●●
● ●●
● ●●
●
● ●● ● ●
●
●
●●●
●●
●
●●
●
●
● ●●
●
●●
●
●
●
●
● ●●
●
●●
●●
●●
●●
●
●
●● ●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●●●●●●● ● ●
●
●●●●
●●
●●●
●●
●
●
●
●●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ● ●●
●●●
● ●●
●
●
● ●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●● ●●
● ●●
●●
●
●●
●
●
●●
●● ●● ●● ●
● ● ●●
●●
● ●
●●●●
●
●
●
● ●●● ● ●●●●●●
●
●
●● ●●●● ●●●
●
●
●
●● ● ●
● ●
● ● ●●
● ●
● ●● ● ●
● ●●●
● ● ● ●● ● ● ●●
●
● ●
●
● ●●
●
●
●●
●
●
● ●
●●
●●
● ●
● ●
●● ●
● ●●●
●
●● ●● ●
●
●●
●
●
●
●
●
●
●
● ●
●● ● ●
●
●
● ●
● ●
●●●
●●
●●● ●
●●
●●
●
●
● ●
● ●
●
●
●
●●
● ●
●
●
● ●●●
●
●
●● ● ●
●
●
● ● ●● ●
●
●
●
●
●
●●
●●●
●
●●
●
●
● ●
● ● ●●
●●
●●
● ●●
●
● ● ● ● ● ● ● ●● ●
●
●
●
●
●
●
● ●●
●
●
●●
● ●●●●●●●
●
●
●
●
●
● ●
●
● ●●● ●●
●
●
●
● ●
● ●●
●●
●
●
●
●
●● ● ●
●
●●
●
● ●
●● ●
●
●
●●●
●●
● ●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
● ●
●●
●●
●
●
●●
●
●
●
●●● ●
●
●
●
● ●
●
●●
●
●
●● ●●● ● ●●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
entro
0
py
−2
−4
sea
son
x
id
e.
ng
a
ch
ture
urva e
ckl
scor
sss
ne s
ikiine
spmp
lu
ots
troug
h
2
●
fsp
●
peak
s
ritynt
ea oi
lin cp
What is “anomalous”
vlstcA
C
rhheifantnFg1e
d
●
●
●
●
−2.5
0.0
2.5
We need a measure of the “anomalousness” of a time
series.
1
Rank points based on their local density.
2
Rank points based on whether they are within
α-convex hulls of different radius.
Yahoo web traffic
40
●
●●
●●
●
●●●
●
●
●
● ●●
●● ●
● ●●
●
● ●●●●●●●
● ●●●
● ● ●●
●●
● ●
●
●●
●●● ● ●
●
●
●●
●●
● ●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●● ● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●● ●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
● ●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ● ●●●●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●● ●● ●
●
●
●
●
●●●
●●●
●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●●
●
●●●
●
●●
●●
●
●
●
●● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
● ● ●●●●
● ●●●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●● ●●
●●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●
●
● ●●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
● ●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●● ●●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
● ●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
● ●● ●●
●
●
●● ●
●
●●●●● ●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●●
●● ●
●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●●●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
● ●
●
● ●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●●
●●
●
●
●
●
●●
●
●●
●●
●●●● ●●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●● ●● ● ● ●● ●●●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●●●●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●●●
●
● ●
●
●
●
●●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●
●● ●●● ●●
●
●
●
●
●●
●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●● ●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●● ●
● ● ●●
●● ●●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ● ● ●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
● ●●
●●
●
●
●●
●●
●
●
●
●
●
●●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
● ● ●
●
●
●
●●
●●
●●
●
●
●
●
●● ●●
●
●
●●
●
●
●
●
●
● ● ●● ● ●
●
●
●
●
●●
●
●
●●● ●●
●
● ●●
●
●
●●
●
●
●●●●
●
●●●
●
●
● ● ● ● ●●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
● ●● ●
●
●●
●●
● ● ●
●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●● ●●
●
● ●●
●●
●●● ●●
●
●
● ●●
●●
●●
●
●
●●●●
● ●●
●
●
●
●●●
●●
● ●●
● ●●
●
● ●● ● ●
●
●
●●●
●●
●
●●
●
●
● ●●
●
●●
●
●
●
●
● ●●
●
●●
●●
●●
●●
●
●
●● ●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●●●●●●● ● ●
●
●●●●
●●
●●●
●●
●
●
●
●●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ● ●●
●●●
● ●●
●
●
● ●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●● ●●
● ●●
●●
●
●●
●
●
●●
●● ●● ●● ●
● ● ●●
●●
● ●
●●●●
●
●
●
● ●●● ● ●●●●●●
●
●
●● ●●●● ●●●
●
●
●
●● ● ●
● ●
● ● ●●
● ●
● ●● ● ●
● ●●●
● ● ● ●● ● ● ●●
●
● ●
●
● ●●
●
●
●●
●
●
● ●
●●
●●
● ●
● ●
●● ●
● ●●●
●
●● ●● ●
●
●●
●
●
●
●
●
●
●
● ●
●● ● ●
●
●
● ●
● ●
●●●
●●
●●● ●
●●
●●
●
●
● ●
● ●
●
●
●
●●
● ●
●
●
● ●●●
●
●
●● ● ●
●
●
● ● ●● ●
●
●
●
●
●
●●
●●●
●
●●
●
●
● ●
● ● ●●
●●
●●
● ●●
●
● ● ● ● ● ● ● ●● ●
●
●
●
●
●
●
● ●●
●
●
●●
● ●●●●●●●
●
●
●
●
●
● ●
●
● ●●● ●●
●
●
●
● ●
● ●●
●●
●
●
●
●
●● ● ●
●
●●
●
● ●
●● ●
●
●
●●●
●●
● ●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
● ●
●●
●●
●
●
●●
●
●
●
●●● ●
●
●
●
● ●
●
●●
●
●
●● ●●● ● ●●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
entro
0
py
−2
−4
sea
son
x
id
e.
ng
a
ch
ture
urva e
ckl
scor
sss
ne s
ikiine
spmp
lu
ots
troug
h
2
●
fsp
●
peak
s
ritynt
ea oi
lin cp
vlstcA
C
rhheifantnFg1e
d
●
●
●
●
−2.5
0.0
2.5
series.
1
2
Yahoo web traffic
40
●
●●
●●
●
●●●
●
●
●
● ●●
●● ●
● ●●
●
● ●●●●●●●
● ●●●
● ● ●●
●●
● ●
●
●●
●●● ● ●
●
●
●●
●●
● ●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●● ● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●● ●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
● ●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ● ●●●●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●● ●● ●
●
●
●
●
●●●
●●●
●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●●
●
●●●
●
●●
●●
●
●
●
●● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
● ● ●●●●
● ●●●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●● ●●
●●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●
●
● ●●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
● ●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●● ●●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
● ●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
● ●● ●●
●
●
●● ●
●
●●●●● ●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●●
●● ●
●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●●●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
● ●
●
● ●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●●
●●
●
●
●
●
●●
●
●●
●●
●●●● ●●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●● ●● ● ● ●● ●●●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●●●●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●●●
●
● ●
●
●
●
●●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●
●● ●●● ●●
●
●
●
●
●●
●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●● ●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●● ●
● ● ●●
●● ●●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ● ● ●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
● ●●
●●
●
●
●●
●●
●
●
●
●
●
●●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
● ● ●
●
●
●
●●
●●
●●
●
●
●
●
●● ●●
●
●
●●
●
●
●
●
●
● ● ●● ● ●
●
●
●
●
●●
●
●
●●● ●●
●
● ●●
●
●
●●
●
●
●●●●
●
●●●
●
●
● ● ● ● ●●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
● ●● ●
●
●●
●●
● ● ●
●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●● ●●
●
● ●●
●●
●●● ●●
●
●
● ●●
●●
●●
●
●
●●●●
● ●●
●
●
●
●●●
●●
● ●●
● ●●
●
● ●● ● ●
●
●
●●●
●●
●
●●
●
●
● ●●
●
●●
●
●
●
●
● ●●
●
●●
●●
●●
●●
●
●
●● ●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●●●●●●● ● ●
●
●●●●
●●
●●●
●●
●
●
●
●●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ● ●●
●●●
● ●●
●
●
● ●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●● ●●
● ●●
●●
●
●●
●
●
●●
●● ●● ●● ●
● ● ●●
●●
● ●
●●●●
●
●
●
● ●●● ● ●●●●●●
●
●
●● ●●●● ●●●
●
●
●
●● ● ●
● ●
● ● ●●
● ●
● ●● ● ●
● ●●●
● ● ● ●● ● ● ●●
●
● ●
●
● ●●
●
●
●●
●
●
● ●
●●
●●
● ●
● ●
●● ●
● ●●●
●
●● ●● ●
●
●●
●
●
●
●
●
●
●
● ●
●● ● ●
●
●
● ●
● ●
●●●
●●
●●● ●
●●
●●
●
●
● ●
● ●
●
●
●
●●
● ●
●
●
● ●●●
●
●
●● ● ●
●
●
● ● ●● ●
●
●
●
●
●
●●
●●●
●
●●
●
●
● ●
● ● ●●
●●
●●
● ●●
●
● ● ● ● ● ● ● ●● ●
●
●
●
●
●
●
● ●●
●
●
●●
● ●●●●●●●
●
●
●
●
●
● ●
●
● ●●● ●●
●
●
●
● ●
● ●●
●●
●
●
●
●
●● ● ●
●
●●
●
● ●
●● ●
●
●
●●●
●●
● ●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●● ●
●
● ●
●●
●●
●
●
●●
●
●
●
●●● ●
●
●
●
● ●
●
●●
●
●
●● ●●● ● ●●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
entro
0
py
−2
−4
sea
son
x
id
e.
ng
a
ch
ture
urva e
ckl
scor
sss
ne s
ikiine
spmp
lu
ots
troug
h
2
●
fsp
●
peak
s
ritynt
ea oi
lin cp
vlstcA
C
rhheifantnFg1e
d
●
●
●
●
−2.5
0.0
2.5
series.
1
2
Yahoo web traffic
40
Bivariate kernel density
n
f̂ (x; H) =
1X
n
KH (x − Xi )
i=1
Xi ∈ a bivariate random sample {X1 , X2 , . . . , Xn }
KH (x) is the standard normal kernel function
H estimated by minimizing the sum of AMISE
Rank points based on f̂ values in 2d PCA space.
Yahoo web traffic
41
Bivariate kernel density
n
f̂ (x; H) =
1X
n
KH (x − Xi )
i=1
Xi ∈ a bivariate random sample {X1 , X2 , . . . , Xn }
KH (x) is the standard normal kernel function
H estimated by minimizing the sum of AMISE
Rank points based on f̂ values in 2d PCA space.
Yahoo web traffic
41
−4
−2
2
4
−6
5
1
−8
pc2
0
2
4
6
Bivariate density ranking
3
−5
0
5
pc1
Yahoo web traffic
42
40000
30000
20000
10000
0
6000
4000
2000
0
50000
40000
30000
20000
10000
0
40000
30000
20000
10000
0
5000
4000
3000
2000
1000
0
S7793 S8494 S10464 S7833 S1715
2015−02−28
2015−03−01
2015−03−02
2015−03−03
2015−03−04
2015−03−05
2015−03−06
2015−03−07
2015−03−08
2015−03−09
2015−03−10
2015−03−11
2015−03−12
2015−03−13
2015−03−14
2015−03−15
2015−03−16
2015−03−17
2015−03−18
2015−03−19
2015−03−20
2015−03−21
2015−03−22
2015−03−23
2015−03−24
2015−03−25
2015−03−26
2015−03−27
2015−03−28
2015−03−29
2015−03−30
2015−03−31
2015−04−01
value
Bivariate density ranking
date
Yahoo web traffic
42
α-convex hulls
The space generated by point pairs that can be
touched by an empty disc of radius α.
α → ∞ gives a convex hull.
Points can become isolated when α is small.
We rank points based on the value of α when
they become isolated.
Yahoo web traffic
43
α-convex hulls
Yahoo web traffic
43
α-convex hulls
Yahoo web traffic
43
α-convex hulls
Yahoo web traffic
43
6
α-convex hull
●
●
●
●
●
●●●
●●
●
●
● ●
●
●● ●
●
●
●●
● ●●●●●●●
●
●
●
●
● ●●● ● ● ●
●
●
●●
●●● ● ●
●
●
●●
●●
●
●●
●
●● ●
●●●
● ●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●● ● ●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●●
●
●
●●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●●
●●
●●
●
●
●
●●
●●●●●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
● ●●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●●●● ●
●
●
●●
●
●●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●● ●
●●
●●
●●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●●
●●● ●
●●
●
●● ●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●
●
●
● ● ● ● ●●●
●●
●●
●
●
●●●
●● ● ●
●
● ●
●
●
●●
●
●
●●
●
●●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●●●● ● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●● ●●● ●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●●●●●●
●●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●●● ●●● ● ●● ●● ●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●● ●● ● ●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ● ●
●
●
●
●
●
●
●
●●
●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●● ● ●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●● ● ● ●● ● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●●
●● ●●●
●
●●● ●
●●
●●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●● ● ●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
● ●●
●●●
●
●
●
●
●
●
●●
●●
● ●●
● ● ●● ● ●
●
●
●●
●
●
●●
●●●●
●
●●●● ●● ●●
●
●
●
●
●
●
● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●●
●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●● ● ●
●●
● ●●
●●●●●
●
●
●● ●●●●●
●
●●
●
●
● ●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●●●●
●
●●●
●
●
●
●●●
● ●
●●
●
●●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●● ●●
●
●
●
●●
●●
●
●
●●●
●
●
● ●●●● ●
●
●
● ●
● ●
●●
● ●●
●●
● ● ● ●● ●●
●
●
●
● ●●
●
●●
● ●
●
●
● ● ●●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●●
●
●
●● ●● ●● ●● ●
●
●
● ●●●●
●● ●●
●
●
●●●●●●● ●
●
●●● ●●● ●
●●
● ●●
●●●●●●
●
●
●
●● ●
●
●●
●●
●
●
● ●●
●● ●●
●● ● ●
●●●●
●● ●●● ●● ●
● ●●
● ●●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ●
●
●
●● ●
● ●
●
●●
●● ●
● ●●●
●●
●
● ●●
●
●
●
●
●
● ●●
●
●
● ●
●
● ●
●●●
●
●
●●
●● ● ● ● ●
●
●
● ●●
●●
● ● ● ●
●
●
●● ●
● ● ●
● ●●
●
● ● ●●●
●
●
● ● ●● ●
● ●
●
●
●
●●
●●
●●
●
● ●
●●
●●
● ●●
●
●● ●●● ●
●●
●●
● ● ● ● ● ● ● ●● ●
●
●
●●
●
●
●
● ●●●●●●
●●
●
●
●
●
●●●● ●● ●● ●
●
●
●
●
●
●
●
●
●
● ●●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●● ●
●●●
● ●
●●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●●
●●
●
●
●●
●
●
●●● ●
●
●
●
●
● ●
●
●●
●
●● ●●● ● ●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
● ●
● ●●
●
● ●
●
●
●
●
●
●
●●
●
●
−8
−6
−4
−2
0
2
4
●
●
●
●
−5
0
Yahoo web traffic
5
44
●
●
6
α-convex hull ranking
●
●
●
●
●
●●●
●●
●
●
● ●
●
●● ●
●
●
●●
●
● ●●●●●●
● ●●
● ●●●
●●
●● ●
●
●●
●●
● ●●
●
●
●●
●●
●
●●
●
●● ●
●●●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●● ● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
● ● ●●●●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●●●
●●● ●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●●
●●
●●
●
●
●
●●
●●●●●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
● ●●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
● ●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●● ●
●●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●●●●●●
●
● ●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●●●
●●●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●●
●
●
●
●
●
●
●
● ● ●●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●●●●
●
●
●
●
●●
●
●●
●●
● ●● ●●●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●●●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●● ●●● ● ●●● ●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●● ●● ● ●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ● ●
●
●
●
●
●
●
●
●●
● ●●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●● ●
●
●●
●
●
●●
●
●
●
●
●
●
●● ● ● ●● ● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●●●
●● ●●●
●
●●● ●
●●
●●
●
●
●
●
●
●
●
●● ●
●
●●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●● ● ●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
● ●●
●
●●
●
●
●
● ●
●
●
●
●
●
●●● ●● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ● ●● ● ●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●●
●
●●●●●
●
● ●● ●●
● ●●
●● ● ●
●
●● ●● ● ●
● ● ●● ● ●
●●●●
●
●
●●
●
● ●●
●
●●
●
●●
●●
● ●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
● ●●●●●●●
●●
●
●● ● ●
●
●
●
●
●●
●
●●●
●
●
●
● ●●
●
●●●
●
●
●
● ●
● ●●
●●
●●
●
●●
●●
●
●●
●●
●
●
●
●
●
●● ●●
●
●
●
●
●● ●●
●
●
●
●●
●●
●
●●
●●●
●
●
● ●●●● ●
●
● ●
● ●
●●
● ●●
●●
● ● ● ●● ●●
●
●
●
● ●●
●
●●
●
● ●
●
●
● ● ●●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●●
●
●
●● ●● ●● ●● ●
●
●
● ●●●●
●● ●●
●
●
●●●●●●● ●
●
●●● ●●● ●
●●
● ●●
●●●●●●
●
● ●●
●
●● ●
●
●
●
●●
●
●
●●●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ●
●
●
● ●● ● ● ● ● ●
●
●●
●●
● ●
●● ●
●
●● ●
● ●
● ●●
●●
● ●●●
●●
●
●●
● ●
●
●
●
●
● ●●
●
●
● ●
●
● ●
●●●
●
●
●●
●● ● ● ● ●
●
●
● ●●
●●
● ● ● ●
●
●
●● ●
● ● ●
● ●●
●
● ● ●●●
●
●
● ● ●● ●
● ●
●
●
●
●●
●●
●●
●
● ●
●●
●●
● ●●
●
●● ●●● ●
●●
●●
● ● ● ● ● ● ● ●● ●
●
●
●●
●
● ●●
●
● ●●●●●●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
● ●
●
●
● ●
● ●
●
● ● ● ●
●●
●
●
●●
●
●
●
●
●●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●●●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●●
●
●●
●
●
●●● ●
●
●
●
●
● ●
●
●●
●
●● ●●● ● ●●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●● ●
● ●
● ●●
●
● ●
●
●
●
●
●
●
●●
●
●
0
2
4
●
●
−4
−2
●
3
−6
●
−8
●
●
●
5
●
●
1
4
2
−5
0
Yahoo web traffic
5
45
●
●
50000
40000
30000
20000
10000
0
40000
30000
20000
10000
0
6000
4000
2000
0
5000
4000
3000
2000
1000
0
30000
20000
10000
0
S10464 S7793 S8494 S1715 S7826
2015−02−28
2015−03−01
2015−03−02
2015−03−03
2015−03−04
2015−03−05
2015−03−06
2015−03−07
2015−03−08
2015−03−09
2015−03−10
2015−03−11
2015−03−12
2015−03−13
2015−03−14
2015−03−15
2015−03−16
2015−03−17
2015−03−18
2015−03−19
2015−03−20
2015−03−21
2015−03−22
2015−03−23
2015−03−24
2015−03−25
2015−03−26
2015−03−27
2015−03−28
2015−03−29
2015−03−30
2015−03−31
2015−04−01
value
α-convex hull ranking
date
Yahoo web traffic
45
HDR versus α-convex hull
HDR boxplot
6
6
α-convex hull
●
●
●
●
●
●●●
●●
●
●
● ●
●
●● ●
● ●●
●
● ●●●●●●●
● ●●
● ●●●
● ● ●●
●●
●●
●●
● ●●
●
●●
●●
●●●
●
●
●●
●●●
● ●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●● ● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●●●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
● ●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
● ●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
● ●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●● ●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●●
●
●
●●●
●●
●
●●● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●●
●●●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●●
● ●●●
●
●●
●
●
●
●●
●●●
●
●
●●
●
●
●●
●
●
●
●
●●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
● ●●● ●
●●
●
●
● ●
●
●●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●●●
●●
●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●● ●●● ● ●●● ●● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●● ●● ● ●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ● ●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ● ●
●
●
●
●
●
●
●
●●
●●
●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●● ●
●
●● ●●●●
●●
●●
●
●
●
●
●
●●
●●
●
●● ●
●
●●
● ● ●●
●●●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●● ● ●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
● ●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
● ●
●●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ● ●● ● ●
●
●
●
●
●
●
●
●
●
●●●●
●
●●●●●●
●●
●
●
●
●
●
●
● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
● ● ●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●●● ● ●
●
●●● ●
●
●●●
●●
● ●●
●●●●
●●●
●
●● ● ●
●●
●●●
●
●
●● ●●●●● ●
●
●
● ●
●●
●
●
●
●
● ●
●●
●●●
●
●●
●●
● ●●
●
●
●
●
●●●
● ●
● ●●
●
●●
●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●● ●●
●
●
●
●
●
●● ●●
●
●
●
●●
●
●●●
●
●
●●●
●
●
● ●●●● ●
●
● ●
● ●
● ●●
●●
● ● ● ●● ●●
●
●
●
● ●●
●
●
●
●
● ● ●●
●
●
●
●
●
●●●●
●
●●
●●●●
●
●
●●
●
● ●● ●
●
●
● ●●
●
●
● ●●
●● ●● ●● ●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
● ● ●● ●
● ●● ●
●
●● ●
●●
●
●
● ● ●●●
●● ●●● ●● ●
● ●
● ●●●●
● ●
● ● ● ●
●
●
●
●●
●●
● ●
●
●● ●
● ●
● ●●
●●
●● ●● ● ● ●
● ●●●
●●
●
●●
●
●
●
●
●●
●
●● ● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●● ●
●● ● ●
● ●●
●
● ● ●●●
●
●
● ● ●● ●
● ●
●
●
●
●●
●●
●●
●
● ●
●●
●●
● ●●
●
●● ●●● ●
●●
●●
● ● ● ● ● ● ● ●● ●
●
●
●●
●
● ●●
●
● ●●●●●●
●
●
●
●
●
● ●
●●● ●
●
●
●
●
● ●
●
●
● ●
● ●
●
●
● ● ● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●●●
●●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●●
●
●●
●
●
●●● ●
●
●
●
●
● ●
●
●●
●
●● ●●● ● ●●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●● ●
● ●
● ●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
4
−4
−4
−2
−2
2
−6
5
3
−6
●
1
3
●
−5
0
pc1
5
−8
−8
pc2
0
0
2
2
4
4
●
●
●
●
5
●
●
1
4
2
−5
Yahoo web traffic
0
5
46
●
●
HDR
value
50000
40000
30000
20000
10000
0
40000
30000
20000
10000
0
6000
α-convex hull
4000
2000
0
5000
4000
3000
2000
1000
0
30000
20000
10000
0
Yahoo web traffic
S10464 S7793 S8494 S1715 S7826
S7793 S8494 S10464 S7833 S1715
2015−02−28
2015−03−01
2015−03−02
2015−03−03
2015−03−04
2015−03−05
2015−03−06
2015−03−07
2015−03−08
2015−03−09
2015−03−10
2015−03−11
2015−03−12
2015−03−13
2015−03−14
2015−03−15
2015−03−16
2015−03−17
2015−03−18
2015−03−19
2015−03−20
2015−03−21
2015−03−22
2015−03−23
2015−03−24
2015−03−25
2015−03−26
2015−03−27
2015−03−28
2015−03−29
2015−03−30
2015−03−31
2015−04−01
40000
30000
20000
10000
0
6000
4000
2000
0
50000
40000
30000
20000
10000
0
40000
30000
20000
10000
0
5000
4000
3000
2000
1000
0
2015−02−28
2015−03−01
2015−03−02
2015−03−03
2015−03−04
2015−03−05
2015−03−06
2015−03−07
2015−03−08
2015−03−09
2015−03−10
2015−03−11
2015−03−12
2015−03−13
2015−03−14
2015−03−15
2015−03−16
2015−03−17
2015−03−18
2015−03−19
2015−03−20
2015−03−21
2015−03−22
2015−03−23
2015−03−24
2015−03−25
2015−03−26
2015−03−27
2015−03−28
2015−03−29
2015−03−30
2015−03−31
2015−04−01
value
Top 5 anomalous time series
date
date
47
Outline
1 The problem
4 Yahoo web traffic
5 What next?
What next?
48
What next?
Develop a more comprehensive set of features
that are reliable measures and fast to compute.
e.g., for finance data.
Consider other dimension reduction methods
and more than 2 dimensions.
Develop dynamic and interactive visualization
tools.
Make methods available in an R package.
Some of the methods are already available in the
anomalous package for R on github.
What next?
49
What next?
tools.
What next?
49
What next?
tools.
What next?
49
What next?
tools.
What next?
49
What next?
tools.
What next?
49
Further information
å
å
å
å
Papers and R packages: robjhyndman.com
Blog: robjhyndman.com/hyndsight
Code: github.com/robjhyndman
Email: Rob.Hyndman@monash.edu
What next?
50

[-0.2cm]Visualisation of big time series data

Transcription

Similar documents

PDF Document

Your partner for multimedia events at Kursaal Bern

A Knowledge Dashboard for Manufacturing Industries - CEUR

Visualisation?

The tools and expertise to get software right first time

Flyer SMARTCOLLECT - Camille Bauer Metrawatt AG

stenograph

Presentation slides

Visual Thinking Assignment USA TV Network Renewal Decision