Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
A contamination model for approximate stochastic order
Eustasio del Barrio
Universidad de Valladolid. IMUVA.
3rd Workshop on Analysis, Geometry and Probability - Universitat Ulm
28th September - 2dn October 2015, Ulm
Eustasio del Barrio Testing approximate stochastic order 1 / 37
Outline
Outline
1 Stochastic order, testability and relaxed versions of s. o.
2 Inference for approximate stochastic order
3 Implementation, simulation & data example
Eustasio del Barrio Testing approximate stochastic order 2 / 37
Stochastic order, testability and relaxed versions of s. o. The stochastic order model
130 140 150 160
0.0
0.2
0.4
0.6
0.8
1.0
π(Fn,Gm) = 0.128409 π(Gm,Fn) = 0.009342
heights
ED
F's
Gm Age 10 Boys Fn Age 10 Girls
Data: National Health and Nutrition Examination SurveyEmpirical d.f.’s for boys and girls at age 10.
Are girls taller than boys?
Stochastic order (Lehmann, 1955): P,Q probs. on R with d.f.’s F , G
P ≤st Q if F (x) ≥ G(x), x ∈ R
For NHANES data, P10 ≤st Q10?Eustasio del Barrio Testing approximate stochastic order 3 / 37
Stochastic order, testability and relaxed versions of s. o. Testing stochastic order
Common testing problems in literature (P ≤st Q ≡ F ≤st G)
a) H0: F = G vs Ha: F <st G
b) H0: F ≤st G vs Ha: F 6≤st G
c) H0: F 6≤st G vs Ha: F ≤st G
Problem a)
focus on statistical evidence for strict relation
assumes stochastic order holds
both H0 and Ha can be false (here focus on b), c))
Problem b) ‘testing for stochastic dominance’(McFadden, 1989; Mosler, 1995; Anderson, 1996, Davidson & Duclos, 2000;Linton et al., 2005, 2010,. . . )
goodness-of-fit problem
absence of evidence against s. o. as minimal requirement for a)
lack of evidence against H0 not evidence for F ≤st G
Eustasio del Barrio Testing approximate stochastic order 4 / 37
Stochastic order, testability and relaxed versions of s. o. Testing stochastic order
Common testing problems in literature (P ≤st Q ≡ F ≤st G)
a) H0: F = G vs Ha: F <st G
b) H0: F ≤st G vs Ha: F 6≤st G
c) H0: F 6≤st G vs Ha: F ≤st G
Problem a)
focus on statistical evidence for strict relation
assumes stochastic order holds
both H0 and Ha can be false (here focus on b), c))
Problem b) ‘testing for stochastic dominance’(McFadden, 1989; Mosler, 1995; Anderson, 1996, Davidson & Duclos, 2000;Linton et al., 2005, 2010,. . . )
goodness-of-fit problem
absence of evidence against s. o. as minimal requirement for a)
lack of evidence against H0 not evidence for F ≤st G
Eustasio del Barrio Testing approximate stochastic order 4 / 37
Stochastic order, testability and relaxed versions of s. o. Testing stochastic order
Problem c) H0: F 6<st G vs Ha: F <st G: assessing stochastic order
rejection provides convincing evidence of F <st G
Unfortunately, no good test for b) exists:
Assume X1, . . . , Xn i.i.d. F <st G
Φ an α-level test (EHΦ(X1, . . . , Xn) ≤ α, H ∈ H0)
Take xm s.t. G(xm) > 1− 1m , Hm s.t. Hm(xm) = 0
Set Fm = (1− 1m )F + 1
mHm; Fm 6<st G
α ≥ EFmΦ(X1, . . . , Xn) ≥ (1− 1m )nEFΦ(X1, . . . , Xn)
Take m→∞
‘no data’ test (reject H0 with prob α regardless data) is UMP!
Berger, 1988 (one-sample); Davidson & Duclos, 2013 (two-sample)
Eustasio del Barrio Testing approximate stochastic order 5 / 37
Stochastic order, testability and relaxed versions of s. o. Testing stochastic order
Problem c) H0: F 6<st G vs Ha: F <st G: assessing stochastic order
rejection provides convincing evidence of F <st G
Unfortunately, no good test for b) exists:
Assume X1, . . . , Xn i.i.d. F <st G
Φ an α-level test (EHΦ(X1, . . . , Xn) ≤ α, H ∈ H0)
Take xm s.t. G(xm) > 1− 1m , Hm s.t. Hm(xm) = 0
Set Fm = (1− 1m )F + 1
mHm; Fm 6<st G
α ≥ EFmΦ(X1, . . . , Xn) ≥ (1− 1m )nEFΦ(X1, . . . , Xn)
Take m→∞
‘no data’ test (reject H0 with prob α regardless data) is UMP!
Berger, 1988 (one-sample); Davidson & Duclos, 2013 (two-sample)
Eustasio del Barrio Testing approximate stochastic order 5 / 37
Stochastic order, testability and relaxed versions of s. o. Uniformly consistent tests
Uniformly consistent tests
X1, X2, . . . i.i.d. P with values in X
A0,n (A1,n) acceptance (critical) set for Hn against Kn based on X1, . . . , Xn
Test uniformly (exponentially) consistent if for some r, r′ > 0
supP∈Hn
Pn(A1,n) ≤ e−rn, supP∈Kn
Pn(A0,n) ≤ e−r′n
Consider the testing problem
H : P = P0 vs K : d(P, P0) > δ
If d dominates dTV and P0 not discrete, no uniformly consistent test of H vs K(Barron, 1989)
Eustasio del Barrio Testing approximate stochastic order 6 / 37
Stochastic order, testability and relaxed versions of s. o. Uniformly consistent tests
Here we propose
A relaxed version of stochastic order for which we can expect to getstatistical evidence
A consistent procedure for gathering that evidence
Which is exponentially uniformly consistent (with due corrections)
Some of our relaxations does hold
Deviation from stochastic order measured through required level ofrelaxation
Easy interpretation
Eustasio del Barrio Testing approximate stochastic order 7 / 37
Stochastic order, testability and relaxed versions of s. o. Relaxations of stochastic order
A relaxation of stochastic order (Arcones et al., 2002)
θ(P,Q) := P[X ≤ Y ]
X,Y independent r.v.’s with laws P,Q, resp.
P ≤sp Q (stochastically precedes) if θ(P,Q) ≥ 12
Stochastic ordering implies stochastic precedence: if P ≤st Q
P(X ≤ Y ) =
∫(1−G(x−))dF (x) ≥
∫(1− F (x−))dF (x) = P(X ≤ X ′) ≥ 1
2,
X ′ independent copy of X
Stochastic precedence a less restrictive assumption
But P ≤sp Q equivalent to median(Y −X) ≥ 0, roughly change in location(different, but similar nature as E(Y −X) ≥ 0)
Eustasio del Barrio Testing approximate stochastic order 8 / 37
Stochastic order, testability and relaxed versions of s. o. Relaxations of stochastic order
A relaxation of stochastic order (Arcones et al., 2002)
θ(P,Q) := P[X ≤ Y ]
X,Y independent r.v.’s with laws P,Q, resp.
P ≤sp Q (stochastically precedes) if θ(P,Q) ≥ 12
Stochastic ordering implies stochastic precedence: if P ≤st Q
P(X ≤ Y ) =
∫(1−G(x−))dF (x) ≥
∫(1− F (x−))dF (x) = P(X ≤ X ′) ≥ 1
2,
X ′ independent copy of X
Stochastic precedence a less restrictive assumption
But P ≤sp Q equivalent to median(Y −X) ≥ 0, roughly change in location(different, but similar nature as E(Y −X) ≥ 0)
Eustasio del Barrio Testing approximate stochastic order 8 / 37
Stochastic order, testability and relaxed versions of s. o. Relaxations of stochastic order
A relaxation of stochastic order (Arcones et al., 2002)
θ(P,Q) := P[X ≤ Y ]
X,Y independent r.v.’s with laws P,Q, resp.
P ≤sp Q (stochastically precedes) if θ(P,Q) ≥ 12
Stochastic ordering implies stochastic precedence: if P ≤st Q
P(X ≤ Y ) =
∫(1−G(x−))dF (x) ≥
∫(1− F (x−))dF (x) = P(X ≤ X ′) ≥ 1
2,
X ′ independent copy of X
Stochastic precedence a less restrictive assumption
But P ≤sp Q equivalent to median(Y −X) ≥ 0, roughly change in location
(different, but similar nature as E(Y −X) ≥ 0)
Eustasio del Barrio Testing approximate stochastic order 8 / 37
Stochastic order, testability and relaxed versions of s. o. Relaxations of stochastic order
A relaxation of stochastic order (Arcones et al., 2002)
θ(P,Q) := P[X ≤ Y ]
X,Y independent r.v.’s with laws P,Q, resp.
P ≤sp Q (stochastically precedes) if θ(P,Q) ≥ 12
Stochastic ordering implies stochastic precedence: if P ≤st Q
P(X ≤ Y ) =
∫(1−G(x−))dF (x) ≥
∫(1− F (x−))dF (x) = P(X ≤ X ′) ≥ 1
2,
X ′ independent copy of X
Stochastic precedence a less restrictive assumption
But P ≤sp Q equivalent to median(Y −X) ≥ 0, roughly change in location(different, but similar nature as E(Y −X) ≥ 0)
Eustasio del Barrio Testing approximate stochastic order 8 / 37
Stochastic order, testability and relaxed versions of s. o. Tolerance zones around false models
False model assessment
Assume model F is false (X ∼ P , P /∈ F)
Is model F an adequate approximation for the data? for P?
Pθ ∈ F is an adequate approximation for the data, X1, . . . , Xn, if a typicalsample of size n from Pθ looks like the data
Data features (Davies, 1995)Credibility indices (Lindsay & Liu, 2009)
Pθ ∈ F gives an adequate description of P if d(P, Pθ) ≤ τd = χ2− distance (Hodges & Lehmann, 1954)d = Euclidean distance (Dette & Munk, 2003)d = smallest π such that P = (1− π)Pθ + πR (Rudas et al. (1994);Ae-dB-C-M, 2008, 2010, 2011, 2012; Liu & Lindsay, 2009; Cerioli etal., 2012)
Choice of τ a hard issue
Interpretation of τ simpler for the π-index
Eustasio del Barrio Testing approximate stochastic order 9 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation
Essential model validation
Observe data X ∼ P , test H1
P = (1− α)R+ αP for some R ∈ FObserve ind. samples X ∼ P , Y ∼ Q, test H2
P = (1− α)R+ αP
Q = (1− α)S + αQ for some (R,S) ∈ FRelated problem of interest: Find α0 = minimal α s.t. null model holds
Example: the similarity model (AE-dB-C-M, 2012)
P and Q α-similar, α ∈ [0, 1) if ∃ prob, R, s.t.{P = (1− α)R+ αP
Q = (1− α)R+ αQ
P , Q α-similar, ⇔ dTV (P,Q) ≤ α (dTV (P,Q) = supA |P (A)−Q(A)|)
Here F = {(R,R)} and
α0,sim(P,Q) = dTV (P,Q)
Eustasio del Barrio Testing approximate stochastic order 10 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation
Essential model validation
Observe data X ∼ P , test H1
P = (1− α)R+ αP for some R ∈ FObserve ind. samples X ∼ P , Y ∼ Q, test H2
P = (1− α)R+ αP
Q = (1− α)S + αQ for some (R,S) ∈ FRelated problem of interest: Find α0 = minimal α s.t. null model holds
Example: the similarity model (AE-dB-C-M, 2012)
P and Q α-similar, α ∈ [0, 1) if ∃ prob, R, s.t.{P = (1− α)R+ αP
Q = (1− α)R+ αQ
P , Q α-similar, ⇔ dTV (P,Q) ≤ α (dTV (P,Q) = supA |P (A)−Q(A)|)
Here F = {(R,R)} and
α0,sim(P,Q) = dTV (P,Q)
Eustasio del Barrio Testing approximate stochastic order 10 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation
Essential model validation
Observe data X ∼ P , test H1
P = (1− α)R+ αP for some R ∈ FObserve ind. samples X ∼ P , Y ∼ Q, test H2
P = (1− α)R+ αP
Q = (1− α)S + αQ for some (R,S) ∈ FRelated problem of interest: Find α0 = minimal α s.t. null model holds
Example: the similarity model (AE-dB-C-M, 2012)
P and Q α-similar, α ∈ [0, 1) if ∃ prob, R, s.t.{P = (1− α)R+ αP
Q = (1− α)R+ αQ
P , Q α-similar, ⇔ dTV (P,Q) ≤ α (dTV (P,Q) = supA |P (A)−Q(A)|)
Here F = {(R,R)} and
α0,sim(P,Q) = dTV (P,Q)
Eustasio del Barrio Testing approximate stochastic order 10 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation
Essential model validation
Observe data X ∼ P , test H1
P = (1− α)R+ αP for some R ∈ FObserve ind. samples X ∼ P , Y ∼ Q, test H2
P = (1− α)R+ αP
Q = (1− α)S + αQ for some (R,S) ∈ FRelated problem of interest: Find α0 = minimal α s.t. null model holds
Example: the similarity model (AE-dB-C-M, 2012)
P and Q α-similar, α ∈ [0, 1) if ∃ prob, R, s.t.{P = (1− α)R+ αP
Q = (1− α)R+ αQ
P , Q α-similar, ⇔ dTV (P,Q) ≤ α (dTV (P,Q) = supA |P (A)−Q(A)|)
Here F = {(R,R)} and
α0,sim(P,Q) = dTV (P,Q)
Eustasio del Barrio Testing approximate stochastic order 10 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation
Approximate stochastic order: P ≤st,α Q if
P = (1− α)R+ αP
Q = (1− α)S + αQ for some R ≤st S
(F = {(R,S) : R ≤st S})
(maybe s. o. too restrictive, but core of distribution fits model)
Interest on minimal contamination level s.t. stochastic order model holds
α0(P,Q) := inf{α : P ≤st,α Q}
Eustasio del Barrio Testing approximate stochastic order 11 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation
Approximate stochastic order: P ≤st,α Q if
P = (1− α)R+ αP
Q = (1− α)S + αQ for some R ≤st S
(F = {(R,S) : R ≤st S})
(maybe s. o. too restrictive, but core of distribution fits model)
Interest on minimal contamination level s.t. stochastic order model holds
α0(P,Q) := inf{α : P ≤st,α Q}
Eustasio del Barrio Testing approximate stochastic order 11 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation
Approximate stochastic order: P ≤st,α Q if
P = (1− α)R+ αP
Q = (1− α)S + αQ for some R ≤st S
(F = {(R,S) : R ≤st S})
(maybe s. o. too restrictive, but core of distribution fits model)
Interest on minimal contamination level s.t. stochastic order model holds
α0(P,Q) := inf{α : P ≤st,α Q}
Eustasio del Barrio Testing approximate stochastic order 11 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation
Approximate stochastic order: P ≤st,α Q if
P = (1− α)R+ αP
Q = (1− α)S + αQ for some R ≤st S
(F = {(R,S) : R ≤st S})
(maybe s. o. too restrictive, but core of distribution fits model)
Interest on minimal contamination level s.t. stochastic order model holds
α0(P,Q) := inf{α : P ≤st,α Q}
Eustasio del Barrio Testing approximate stochastic order 11 / 37
Stochastic order, testability and relaxed versions of s. o. Trimming methods in essential model validation
Trimmed Distributions
(X , β) measurable space; P(X , β) prob. measures on (X , β), P ∈ P(X , β)
Rα(P ) =
{R ∈ P(X , β) : R� P,
dR
dP≤ 1
1− αP -a.s.
}
Proposition
(a) Rα(P ) is a convex set; α1 ≤ α2 ⇒ Rα1(P ) ⊂ Rα2
(P )
(b) If α < 1 and (X , β) complete separable metric space then Rα(P ) compactfor weak convergence.
(c) R ∈ Rα(P ) iff P = (1− α)R+ αP
Eustasio del Barrio Testing approximate stochastic order 12 / 37
Stochastic order, testability and relaxed versions of s. o. Essential model validation & trimming
Null models in essential model validation expressable in terms of trimmings
Observe X ∼ P , test H1
P = (1− α)R+ αP for some R ∈ F
H1 holds iff Rα(P ) ∩ F 6= ∅
Observe indep. X ∼ P , Y ∼ Q test H2
P = (1− α)R+ αP
Q = (1− α)S + αQ for some (R,S) ∈ F
H2 holds iff (Rα(P )×Rα(Q)) ∩ F 6= ∅
If R(P ),F closed for metric d
H1 holds iff d(Rα(P ),F) = 0
H2 holds iff d(Rα(P )×Rα(Q),F) = 0
Eustasio del Barrio Testing approximate stochastic order 13 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Trimmings mix well with stochastic order:
For any P ∃ Pα, Pα in Rπ(P ) s. t.
Pα ≤st R ≤st Pα for every R ∈ Rα(P )
Pα, Pα easily computable
Recall P ≤st,α Q iff ∃ P ∈ Rα(P ), Q ∈ Rα(Q), s.t. P ≤st Q
Hence P ≤st,α Q iff Pα ≤st Qα
Conclude from thisα0(P,Q) = sup
x(G(x)− F (x))
Equivalently,P ≤st,α Q ⇔ α0(P,Q) ≤ α
Eustasio del Barrio Testing approximate stochastic order 14 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Trimmings mix well with stochastic order:
For any P ∃ Pα, Pα in Rπ(P ) s. t.
Pα ≤st R ≤st Pα for every R ∈ Rα(P )
Pα, Pα easily computable
Recall P ≤st,α Q iff ∃ P ∈ Rα(P ), Q ∈ Rα(Q), s.t. P ≤st Q
Hence P ≤st,α Q iff Pα ≤st Qα
Conclude from thisα0(P,Q) = sup
x(G(x)− F (x))
Equivalently,P ≤st,α Q ⇔ α0(P,Q) ≤ α
Eustasio del Barrio Testing approximate stochastic order 14 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Trimmings mix well with stochastic order:
For any P ∃ Pα, Pα in Rπ(P ) s. t.
Pα ≤st R ≤st Pα for every R ∈ Rα(P )
Pα, Pα easily computable
Recall P ≤st,α Q iff ∃ P ∈ Rα(P ), Q ∈ Rα(Q), s.t. P ≤st Q
Hence P ≤st,α Q iff Pα ≤st Qα
Conclude from thisα0(P,Q) = sup
x(G(x)− F (x))
Equivalently,P ≤st,α Q ⇔ α0(P,Q) ≤ α
Eustasio del Barrio Testing approximate stochastic order 14 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Trimmings mix well with stochastic order:
For any P ∃ Pα, Pα in Rπ(P ) s. t.
Pα ≤st R ≤st Pα for every R ∈ Rα(P )
Pα, Pα easily computable
Recall P ≤st,α Q iff ∃ P ∈ Rα(P ), Q ∈ Rα(Q), s.t. P ≤st Q
Hence P ≤st,α Q iff Pα ≤st Qα
Conclude from thisα0(P,Q) = sup
x(G(x)− F (x))
Equivalently,P ≤st,α Q ⇔ α0(P,Q) ≤ α
Eustasio del Barrio Testing approximate stochastic order 14 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Trimmings mix well with stochastic order:
For any P ∃ Pα, Pα in Rπ(P ) s. t.
Pα ≤st R ≤st Pα for every R ∈ Rα(P )
Pα, Pα easily computable
Recall P ≤st,α Q iff ∃ P ∈ Rα(P ), Q ∈ Rα(Q), s.t. P ≤st Q
Hence P ≤st,α Q iff Pα ≤st Qα
Conclude from thisα0(P,Q) = sup
x(G(x)− F (x))
Equivalently,P ≤st,α Q ⇔ α0(P,Q) ≤ α
Eustasio del Barrio Testing approximate stochastic order 14 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Trimmings mix well with stochastic order:
For any P ∃ Pα, Pα in Rπ(P ) s. t.
Pα ≤st R ≤st Pα for every R ∈ Rα(P )
Pα, Pα easily computable
Recall P ≤st,α Q iff ∃ P ∈ Rα(P ), Q ∈ Rα(Q), s.t. P ≤st Q
Hence P ≤st,α Q iff Pα ≤st Qα
Conclude from thisα0(P,Q) = sup
x(G(x)− F (x))
Equivalently,P ≤st,α Q ⇔ α0(P,Q) ≤ α
Eustasio del Barrio Testing approximate stochastic order 14 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Example 1. P = N(µ, σ), Q = N(ν, σ), µ > ν
Now Q ≤st P and,
α0(P,Q) = (supx
(G(x)− F (x)) = 2Φ(µ−ν2σ
)− 1.
µ− ν = 0.1σ ⇒ P ≤st,0.04 Q
µ− ν = 0.25σ ⇒ P ≤st,0.0987 Q
µ− ν = 0.5σ ⇒ P ≤st,0.1915 Q
µ− ν = σ ⇒ P ≤st,0.3413 Q
Example 2. P = N(µ, σ), Q = N(ν, τ)
Here α0(P,Q) depends on µ, ν, σ, τ
θ(P,Q) = µ− ν (Arcones et al.,2002)
For µ = ν, P ≤sp Q regardless σ, τ
But N(0, σ) takes values greater than N(0, 0) 50% of times!
Eustasio del Barrio Testing approximate stochastic order 15 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Example 1. P = N(µ, σ), Q = N(ν, σ), µ > ν
Now Q ≤st P and,
α0(P,Q) = (supx
(G(x)− F (x)) = 2Φ(µ−ν2σ
)− 1.
µ− ν = 0.1σ ⇒ P ≤st,0.04 Q
µ− ν = 0.25σ ⇒ P ≤st,0.0987 Q
µ− ν = 0.5σ ⇒ P ≤st,0.1915 Q
µ− ν = σ ⇒ P ≤st,0.3413 Q
Example 2. P = N(µ, σ), Q = N(ν, τ)
Here α0(P,Q) depends on µ, ν, σ, τ
θ(P,Q) = µ− ν (Arcones et al.,2002)
For µ = ν, P ≤sp Q regardless σ, τ
But N(0, σ) takes values greater than N(0, 0) 50% of times!
Eustasio del Barrio Testing approximate stochastic order 15 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Example 1. P = N(µ, σ), Q = N(ν, σ), µ > ν
Now Q ≤st P and,
α0(P,Q) = (supx
(G(x)− F (x)) = 2Φ(µ−ν2σ
)− 1.
µ− ν = 0.1σ ⇒ P ≤st,0.04 Q
µ− ν = 0.25σ ⇒ P ≤st,0.0987 Q
µ− ν = 0.5σ ⇒ P ≤st,0.1915 Q
µ− ν = σ ⇒ P ≤st,0.3413 Q
Example 2. P = N(µ, σ), Q = N(ν, τ)
Here α0(P,Q) depends on µ, ν, σ, τ
θ(P,Q) = µ− ν (Arcones et al.,2002)
For µ = ν, P ≤sp Q regardless σ, τ
But N(0, σ) takes values greater than N(0, 0) 50% of times!
Eustasio del Barrio Testing approximate stochastic order 15 / 37
Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming
Example 1. P = N(µ, σ), Q = N(ν, σ), µ > ν
Now Q ≤st P and,
α0(P,Q) = (supx
(G(x)− F (x)) = 2Φ(µ−ν2σ
)− 1.
µ− ν = 0.1σ ⇒ P ≤st,0.04 Q
µ− ν = 0.25σ ⇒ P ≤st,0.0987 Q
µ− ν = 0.5σ ⇒ P ≤st,0.1915 Q
µ− ν = σ ⇒ P ≤st,0.3413 Q
Example 2. P = N(µ, σ), Q = N(ν, τ)
Here α0(P,Q) depends on µ, ν, σ, τ
θ(P,Q) = µ− ν (Arcones et al.,2002)
For µ = ν, P ≤sp Q regardless σ, τ
But N(0, σ) takes values greater than N(0, 0) 50% of times!
Eustasio del Barrio Testing approximate stochastic order 15 / 37
Inference for approximate stochastic order Estimation & testing
Inference in approximate stochastic order models
Assume X1, ..., Xn i.i.d. P ; Y1, ..., Ym i.i.d. Q, independent samples
Goals
(a) For a fixed α, test H0 : P ≤st,α Q vs. H0 : P 6≤st,α Q
(b) For a fixed α, test H0 : P 6≤st,α Q vs. H0 : P ≤st,α Q
(c) Estimation/confidence intervals/confidence bounds for α0(P,Q)
Recall P ≤st,α Q⇔ α0(P,Q) ≤ α; reformulate (a), (b) as
(a) H0 : α0(P,Q) ≤ α vs. Ha : α0(P,Q) > α (testing against approximate s.o.)
(b’) H0 : α0(P,Q) ≥ α vs. Ha : α0(P,Q) < α (testing for approximate s.o.)
Eustasio del Barrio Testing approximate stochastic order 16 / 37
Inference for approximate stochastic order Estimation & testing
Inference in approximate stochastic order models
Assume X1, ..., Xn i.i.d. P ; Y1, ..., Ym i.i.d. Q, independent samples
Goals
(a) For a fixed α, test H0 : P ≤st,α Q vs. H0 : P 6≤st,α Q
(b) For a fixed α, test H0 : P 6≤st,α Q vs. H0 : P ≤st,α Q
(c) Estimation/confidence intervals/confidence bounds for α0(P,Q)
Recall P ≤st,α Q⇔ α0(P,Q) ≤ α; reformulate (a), (b) as
(a) H0 : α0(P,Q) ≤ α vs. Ha : α0(P,Q) > α (testing against approximate s.o.)
(b’) H0 : α0(P,Q) ≥ α vs. Ha : α0(P,Q) < α (testing for approximate s.o.)
Eustasio del Barrio Testing approximate stochastic order 16 / 37
Inference for approximate stochastic order Estimation & testing
Inference in approximate stochastic order models
Assume X1, ..., Xn i.i.d. P ; Y1, ..., Ym i.i.d. Q, independent samples
Goals
(a) For a fixed α, test H0 : P ≤st,α Q vs. H0 : P 6≤st,α Q
(b) For a fixed α, test H0 : P 6≤st,α Q vs. H0 : P ≤st,α Q
(c) Estimation/confidence intervals/confidence bounds for α0(P,Q)
Recall P ≤st,α Q⇔ α0(P,Q) ≤ α; reformulate (a), (b) as
(a) H0 : α0(P,Q) ≤ α vs. Ha : α0(P,Q) > α (testing against approximate s.o.)
(b’) H0 : α0(P,Q) ≥ α vs. Ha : α0(P,Q) < α (testing for approximate s.o.)
Eustasio del Barrio Testing approximate stochastic order 16 / 37
Inference for approximate stochastic order Asymptotic theory
Assume F and G continuous; n = m→∞
Fn, Gn empirical d.f.’s
Theorem
α0(Fn, Gn) →a.s.
α0(F,G),
√n(α0(Fn, Gn)− α0(F,G))→
wB(F,G)
withB(F,G) = sup
x∈Γ(F,G)
(B1(F (x))−B2(G(x))),
B1, B2 independent Brownian Bridges;
Γ(F,G) := {x ∈ R : F (x)−G(x) = α0(F,G)}
A bootstrap version also available, but slow approximation (support estimation)
Eustasio del Barrio Testing approximate stochastic order 17 / 37
Inference for approximate stochastic order Asymptotic theory
Quantiles of B(F,G) depend on F,G in a complex way
Define Bα = B(U(α, 1 + α), U(0, 1)), 0 ≤ α ≤ 1
Bα = supα≤t≤1
(B1(t)−B2(t− α))
P (B0 >√
2t) = e−t2/2, α = 0, 0.1, . . . , 0.5
Eustasio del Barrio Testing approximate stochastic order 18 / 37
Inference for approximate stochastic order Asymptotic theory
Quantiles of B(F,G) depend on F,G in a complex way
Define Bα = B(U(α, 1 + α), U(0, 1)), 0 ≤ α ≤ 1
Bα = supα≤t≤1
(B1(t)−B2(t− α))
P (B0 >√
2t) = e−t2/2, α = 0, 0.1, . . . , 0.5
Eustasio del Barrio Testing approximate stochastic order 18 / 37
Inference for approximate stochastic order Asymptotic theory
Bounds for asymptotic quantiles
Kβ(F,G) (resp. Kβ(α)) β-quantile of B(F,G) (resp. Bα)
Kβ(F,G) ≤ Kβ(α(F,G)), β ∈ (0, 1)
If β ∈ (0, 12 ]
σ(F,G, α(F,G))Φ−1(β) ≤ Kβ(F,G),
where σ(F,G, α(F,G)) = mint∈T (F,G,α(F,G)) σt,
T (F,G, α(F,G)) = {t : ∃x s.t. F (x) = t, G(x) = t− α(F,G)} and
σ2t = t(1− t) + (t− α(F,G))(1− t+ α(F,G))
Eustasio del Barrio Testing approximate stochastic order 19 / 37
Inference for approximate stochastic order Testing against essential stochastic order
Testing against essential stochastic order
ConsiderH0 : α0(F,G) ≤ α, vs. Ha : α0(F,G) > α
(equivalently, H0 : F ≤st,α G vs. Ha : F 6≤st,α G)
Reject H0 if √n(α0(Fn, Gn)− α) > K1−β(α),
K1−β(α) = 1− β quantile of B(α)
Theorem
limn→∞
sup(F,G)∈H0
PF,G(√n(α0(Fn, Gn)− α) > K1−β(α))
= limn→∞
PF0,G0(√n(α0(Fn, Gn)− α) > K1−β(α)) = β,
F0 ≡ U(α, 1 + α), G0 ≡ U(0, 1)
Eustasio del Barrio Testing approximate stochastic order 20 / 37
Inference for approximate stochastic order Testing against essential stochastic order
Testing against essential stochastic order
ConsiderH0 : α0(F,G) ≤ α, vs. Ha : α0(F,G) > α
(equivalently, H0 : F ≤st,α G vs. Ha : F 6≤st,α G)
Reject H0 if √n(α0(Fn, Gn)− α) > K1−β(α),
K1−β(α) = 1− β quantile of B(α)
Theorem
limn→∞
sup(F,G)∈H0
PF,G(√n(α0(Fn, Gn)− α) > K1−β(α))
= limn→∞
PF0,G0(√n(α0(Fn, Gn)− α) > K1−β(α)) = β,
F0 ≡ U(α, 1 + α), G0 ≡ U(0, 1)
Eustasio del Barrio Testing approximate stochastic order 20 / 37
Inference for approximate stochastic order Testing against essential stochastic order
Testing against essential stochastic order
ConsiderH0 : α0(F,G) ≤ α, vs. Ha : α0(F,G) > α
(equivalently, H0 : F ≤st,α G vs. Ha : F 6≤st,α G)
Reject H0 if √n(α0(Fn, Gn)− α) > K1−β(α),
K1−β(α) = 1− β quantile of B(α)
Theorem
limn→∞
sup(F,G)∈H0
PF,G(√n(α0(Fn, Gn)− α) > K1−β(α))
= limn→∞
PF0,G0(√n(α0(Fn, Gn)− α) > K1−β(α)) = β,
F0 ≡ U(α, 1 + α), G0 ≡ U(0, 1)
Eustasio del Barrio Testing approximate stochastic order 20 / 37
Inference for approximate stochastic order Testing against essential stochastic order
Testing against essential stochastic order
ConsiderH0 : α0(F,G) ≤ α, vs. Ha : α0(F,G) > α
(equivalently, H0 : F ≤st,α G vs. Ha : F 6≤st,α G)
Reject H0 if √n(α0(Fn, Gn)− α) > K1−β(α),
K1−β(α) = 1− β quantile of B(α)
Theorem
limn→∞
sup(F,G)∈H0
PF,G(√n(α0(Fn, Gn)− α) > K1−β(α))
= limn→∞
PF0,G0(√n(α0(Fn, Gn)− α) > K1−β(α)) = β,
F0 ≡ U(α, 1 + α), G0 ≡ U(0, 1)
Eustasio del Barrio Testing approximate stochastic order 20 / 37
Inference for approximate stochastic order Testing against essential stochastic order
Nonasymptotic bounds
Theorem
If α0(F,G) < α and K1−β(α) ≥ 0 then
PF,G(√n(α0(Fn, Gn)− α) > K1−β(α)) ≤ 2e−n(α−α0(F,G))2 .
If α0(F,G) > α then
PF,G(√n(α0(Fn, Gn)− α) ≤ K1−β(α)) ≤ e−2(
√n(α−α0(F,G))−K1−β(α))2 .
Test is u.e.c. for H ′0 : α0(F,G) ≤ α′ vs. H ′a : α0(F,G) > α′′ if α′ < α < α′′
Compute sample sizes to guarantee given power against fixed alternatives
Eustasio del Barrio Testing approximate stochastic order 21 / 37
Inference for approximate stochastic order Testing against essential stochastic order
Nonasymptotic bounds
Theorem
If α0(F,G) < α and K1−β(α) ≥ 0 then
PF,G(√n(α0(Fn, Gn)− α) > K1−β(α)) ≤ 2e−n(α−α0(F,G))2 .
If α0(F,G) > α then
PF,G(√n(α0(Fn, Gn)− α) ≤ K1−β(α)) ≤ e−2(
√n(α−α0(F,G))−K1−β(α))2 .
Test is u.e.c. for H ′0 : α0(F,G) ≤ α′ vs. H ′a : α0(F,G) > α′′ if α′ < α < α′′
Compute sample sizes to guarantee given power against fixed alternatives
Eustasio del Barrio Testing approximate stochastic order 21 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Testing for essential stochastic order
ConsiderH0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
(equivalently, H0 : F 6≤st,α′ G if α′ < α vs. Ha : F ≤st,α′ G for some α′ < α)
Reject H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
where σ2α = 1−α2
2 , (assume β < 12 )
Theorem
limn→∞
sup(F,G)∈H0
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β))
= limn→∞
PF0,G0(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) = β,
F0 ≡ 1−α2 U(0, 1+α
2 ) + 1+α2 U( 1+α
2 , 1 + α(1−α)2 ), G0 ≡ U(0, 1)
Eustasio del Barrio Testing approximate stochastic order 22 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Testing for essential stochastic order
ConsiderH0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
(equivalently, H0 : F 6≤st,α′ G if α′ < α vs. Ha : F ≤st,α′ G for some α′ < α)
Reject H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
where σ2α = 1−α2
2 , (assume β < 12 )
Theorem
limn→∞
sup(F,G)∈H0
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β))
= limn→∞
PF0,G0(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) = β,
F0 ≡ 1−α2 U(0, 1+α
2 ) + 1+α2 U( 1+α
2 , 1 + α(1−α)2 ), G0 ≡ U(0, 1)
Eustasio del Barrio Testing approximate stochastic order 22 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Testing for essential stochastic order
ConsiderH0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
(equivalently, H0 : F 6≤st,α′ G if α′ < α vs. Ha : F ≤st,α′ G for some α′ < α)
Reject H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
where σ2α = 1−α2
2 , (assume β < 12 )
Theorem
limn→∞
sup(F,G)∈H0
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β))
= limn→∞
PF0,G0(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) = β,
F0 ≡ 1−α2 U(0, 1+α
2 ) + 1+α2 U( 1+α
2 , 1 + α(1−α)2 ), G0 ≡ U(0, 1)
Eustasio del Barrio Testing approximate stochastic order 22 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Testing for essential stochastic order
ConsiderH0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
(equivalently, H0 : F 6≤st,α′ G if α′ < α vs. Ha : F ≤st,α′ G for some α′ < α)
Reject H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
where σ2α = 1−α2
2 , (assume β < 12 )
Theorem
limn→∞
sup(F,G)∈H0
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β))
= limn→∞
PF0,G0(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) = β,
F0 ≡ 1−α2 U(0, 1+α
2 ) + 1+α2 U( 1+α
2 , 1 + α(1−α)2 ), G0 ≡ U(0, 1)
Eustasio del Barrio Testing approximate stochastic order 22 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Nonasymptotic bounds
Theorem
If α0(F,G) > α then
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) ≤ e−2n(α−α0(F,G))2
If α0(F,G) < α and n(α− α0(F,G))2 ≥ log 2 −σαΦ−1(β),
PF,G(√n(α0(F,G)− α) ≥ σαΦ−1(β)) ≤ 2e−(σαΦ−1(β)+
√n(α−α0(F,G)))2
Test is u.e.c. for H ′0 : α0(F,G) ≥ α′ vs. H ′a : α0(F,G) < α′′ if α′′ < α < α′
Compare to case H0 : F 6≤st G vs. Ha : F ≤st G
Try to assess F ≤st G up to α = 0.05 contamination; β = 0.05
Want to detect alternatives with α0(F,G) ≤ 0.01 with power 0.9
Take n = 8143
Eustasio del Barrio Testing approximate stochastic order 23 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Nonasymptotic bounds
Theorem
If α0(F,G) > α then
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) ≤ e−2n(α−α0(F,G))2
If α0(F,G) < α and n(α− α0(F,G))2 ≥ log 2 −σαΦ−1(β),
PF,G(√n(α0(F,G)− α) ≥ σαΦ−1(β)) ≤ 2e−(σαΦ−1(β)+
√n(α−α0(F,G)))2
Test is u.e.c. for H ′0 : α0(F,G) ≥ α′ vs. H ′a : α0(F,G) < α′′ if α′′ < α < α′
Compare to case H0 : F 6≤st G vs. Ha : F ≤st G
Try to assess F ≤st G up to α = 0.05 contamination; β = 0.05
Want to detect alternatives with α0(F,G) ≤ 0.01 with power 0.9
Take n = 8143
Eustasio del Barrio Testing approximate stochastic order 23 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Nonasymptotic bounds
Theorem
If α0(F,G) > α then
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) ≤ e−2n(α−α0(F,G))2
If α0(F,G) < α and n(α− α0(F,G))2 ≥ log 2 −σαΦ−1(β),
PF,G(√n(α0(F,G)− α) ≥ σαΦ−1(β)) ≤ 2e−(σαΦ−1(β)+
√n(α−α0(F,G)))2
Test is u.e.c. for H ′0 : α0(F,G) ≥ α′ vs. H ′a : α0(F,G) < α′′ if α′′ < α < α′
Compare to case H0 : F 6≤st G vs. Ha : F ≤st G
Try to assess F ≤st G up to α = 0.05 contamination; β = 0.05
Want to detect alternatives with α0(F,G) ≤ 0.01 with power 0.9
Take n = 8143
Eustasio del Barrio Testing approximate stochastic order 23 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Nonasymptotic bounds
Theorem
If α0(F,G) > α then
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) ≤ e−2n(α−α0(F,G))2
If α0(F,G) < α and n(α− α0(F,G))2 ≥ log 2 −σαΦ−1(β),
PF,G(√n(α0(F,G)− α) ≥ σαΦ−1(β)) ≤ 2e−(σαΦ−1(β)+
√n(α−α0(F,G)))2
Test is u.e.c. for H ′0 : α0(F,G) ≥ α′ vs. H ′a : α0(F,G) < α′′ if α′′ < α < α′
Compare to case H0 : F 6≤st G vs. Ha : F ≤st G
Try to assess F ≤st G up to α = 0.05 contamination; β = 0.05
Want to detect alternatives with α0(F,G) ≤ 0.01 with power 0.9
Take n = 8143
Eustasio del Barrio Testing approximate stochastic order 23 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Nonasymptotic bounds
Theorem
If α0(F,G) > α then
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) ≤ e−2n(α−α0(F,G))2
If α0(F,G) < α and n(α− α0(F,G))2 ≥ log 2 −σαΦ−1(β),
PF,G(√n(α0(F,G)− α) ≥ σαΦ−1(β)) ≤ 2e−(σαΦ−1(β)+
√n(α−α0(F,G)))2
Test is u.e.c. for H ′0 : α0(F,G) ≥ α′ vs. H ′a : α0(F,G) < α′′ if α′′ < α < α′
Compare to case H0 : F 6≤st G vs. Ha : F ≤st G
Try to assess F ≤st G up to α = 0.05 contamination; β = 0.05
Want to detect alternatives with α0(F,G) ≤ 0.01 with power 0.9
Take n = 8143
Eustasio del Barrio Testing approximate stochastic order 23 / 37
Inference for approximate stochastic order Testing for essential stochastic order
Nonasymptotic bounds
Theorem
If α0(F,G) > α then
PF,G(√n(α0(Fn, Gn)− α) < σαΦ−1(β)) ≤ e−2n(α−α0(F,G))2
If α0(F,G) < α and n(α− α0(F,G))2 ≥ log 2 −σαΦ−1(β),
PF,G(√n(α0(F,G)− α) ≥ σαΦ−1(β)) ≤ 2e−(σαΦ−1(β)+
√n(α−α0(F,G)))2
Test is u.e.c. for H ′0 : α0(F,G) ≥ α′ vs. H ′a : α0(F,G) < α′′ if α′′ < α < α′
Compare to case H0 : F 6≤st G vs. Ha : F ≤st G
Try to assess F ≤st G up to α = 0.05 contamination; β = 0.05
Want to detect alternatives with α0(F,G) ≤ 0.01 with power 0.9
Take n = 8143
Eustasio del Barrio Testing approximate stochastic order 23 / 37
Inference for approximate stochastic order Confidence bounds
Confidence bounds
Instead of testing for/against contaminated stochastic order, report upper/lowerbounds for true contamination level, α0(F,G)
For β < 12 ,
α0(Fn, Gn)−√nσnΦ−1(β)
σ2n = mint:Fn(t)−Gn(t)=α0(Fn,Gn)) σ
2t ,
σ2t = t(1− t) + (t− α0(Fn, Gn))(1− t+ α0(Fn, Gn))
is an upper bound with asymptotic confidence level at least 1− β
Better use bias corrected α0(Fn, Gn)BOOT
α0(Fn, Gn)−√nK1−β(α0(Fn, Gn))
is a lower confidence bound for α0(F,G) with asymptotic confidence level 1− βQuantiles K1−β(α0(Fn, Gn)) numerically approximated
Eustasio del Barrio Testing approximate stochastic order 24 / 37
Inference for approximate stochastic order Paired sampling
Dependent data
Often X = pre-treatment, Y = post-treatment measurement
(X,Y ) ∼ H with marginals F and G
Has patient improved with treatment? ⇔ F ≤st G?
As before, H0 : F 6≤st G vs. Ha : F ≤st G not testable
Consider instead H0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
(X1, Y1), . . . , (Xn, Yn) i.i.d. random vectors with common joint d.f. H
H(x, y) = C(F (x), G(y)), C copula
α0(F,G) depends only on marginals; α0(Fn, Gn) consistent estimator
Distribution of α0(Fn, Gn) does depend on C:
√n (α0(Fn, Gn)− α0(F,G))→w sup
x:G(x)−F (x)=α0(F,G)
BC(G(x), F (x)),
BC centered Gaussian on [0, 1]2, covarianceKC((s, t), (s′, t′)) = s ∧ s′ + t ∧ t′ − (s− t)(s′ − t′)− C(t′, s)− C(t, s′)
Eustasio del Barrio Testing approximate stochastic order 25 / 37
Inference for approximate stochastic order Paired sampling
Testing for approximate stochastic order, dependent data
Now
α(1− α) ≤ Var(BC(t, t− α))≤1− α2 − 2|t− 1+α2 |, t ∈ [α, 1]
equality for antimonotone coupling C(s, t) = (s+ t− 1)+
KC,β(F,G) β-quantile of rhs in limit distribution; for β ∈ (0, 12 )
KC,β(F,G) ≥ (1− α0(F,G)2)1/2Φ−1(β)
Similar to independent case, set σ2α = 1− α2; reject α0(F,G) ≥ α if
√n(α0(Fn, Gn)− α) < σαΦ−1(β)
Eustasio del Barrio Testing approximate stochastic order 26 / 37
Inference for approximate stochastic order Paired sampling
Testing for approximate stochastic order, dependent data
As before, test uniformly asymptotically consistent:
limn→∞
supH∈H0
PH[√n(α0(Fn, Gn)− α) < σαΦ−1(β)
]= lim
n→∞PH[√n(α0(Fn, Gn)− α) < σπ0
Φ−1(β)]
= β,
H joint d.f. with marginals F ≡ 1−α2 U(0, 1+α
2 ) + 1+α2 U( 1+α
2 , 1 + α(1−α)2 ),
G ≡ U(0, 1) and copula C(s, t) = (s+ t− 1)+.
Eustasio del Barrio Testing approximate stochastic order 27 / 37
Inference for approximate stochastic order Paired sampling
Nonasymptotic bounds: if α0(F,G) > α then
PH[√n(α0(Fn, Gn)− α < σαΦ−1(β)
]≤ e−n2 (α−α0(Fn,Gn)2 ,
If α0(F,G) < α and n(α− α0(F,G))2 ≥ 2 log 2− σαΦ−1(β),
PH[√n(α0(Fn, Gn)− α) ≥ σαΦ−1(β)
]≤ 2e
−n2 [(α−α0(F,G))+√
2σα√n
Φ−1(β)]2.
Independent vs. dependent setup
In independent setup rejection of H0 : α0(F,G) ≥ α when
√n(α0(Fn, Gn)− α) < σα√
2Φ−1(β).
Under dependence, the extra√
2 factor allows tocontrol uniformly type I error probability
Eustasio del Barrio Testing approximate stochastic order 28 / 37
Inference for approximate stochastic order Paired sampling
Nonasymptotic bounds: if α0(F,G) > α then
PH[√n(α0(Fn, Gn)− α < σαΦ−1(β)
]≤ e−n2 (α−α0(Fn,Gn)2 ,
If α0(F,G) < α and n(α− α0(F,G))2 ≥ 2 log 2− σαΦ−1(β),
PH[√n(α0(Fn, Gn)− α) ≥ σαΦ−1(β)
]≤ 2e
−n2 [(α−α0(F,G))+√
2σα√n
Φ−1(β)]2.
Independent vs. dependent setup
In independent setup rejection of H0 : α0(F,G) ≥ α when
√n(α0(Fn, Gn)− α) < σα√
2Φ−1(β).
Under dependence, the extra√
2 factor allows tocontrol uniformly type I error probability
Eustasio del Barrio Testing approximate stochastic order 28 / 37
Implementation, simulation & data example Implementation issues
Testing for essential stochastic order: finite sample performance
Consider again H0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
Rejection of H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
σ2α = 1−α2
2 , asympt. of level β; type I-type II error probs. exponentially → 0
σα from worst case choice; possible improvement from estimated σn
A more important improvement: E(α0(Fn, Gn)) ≥ α0(F,G); estimate bias by
biasBOOT(α0(Fn, Gn)) := E∗(α0(F ∗n , G∗n))− α0(Fn, Gn)
Defineαn,BOOT := α0(Fn, Gn)− biasBOOT(α0(Fn, Gn)).
Reject H0 if √n(αn,BOOT − α) < σnΦ−1(β),
Test asympt. of level β
Eustasio del Barrio Testing approximate stochastic order 29 / 37
Implementation, simulation & data example Implementation issues
Testing for essential stochastic order: finite sample performance
Consider again H0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
Rejection of H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
σ2α = 1−α2
2 , asympt. of level β; type I-type II error probs. exponentially → 0
σα from worst case choice; possible improvement from estimated σn
A more important improvement: E(α0(Fn, Gn)) ≥ α0(F,G); estimate bias by
biasBOOT(α0(Fn, Gn)) := E∗(α0(F ∗n , G∗n))− α0(Fn, Gn)
Defineαn,BOOT := α0(Fn, Gn)− biasBOOT(α0(Fn, Gn)).
Reject H0 if √n(αn,BOOT − α) < σnΦ−1(β),
Test asympt. of level β
Eustasio del Barrio Testing approximate stochastic order 29 / 37
Implementation, simulation & data example Implementation issues
Testing for essential stochastic order: finite sample performance
Consider again H0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
Rejection of H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
σ2α = 1−α2
2 , asympt. of level β; type I-type II error probs. exponentially → 0
σα from worst case choice; possible improvement from estimated σn
A more important improvement: E(α0(Fn, Gn)) ≥ α0(F,G); estimate bias by
biasBOOT(α0(Fn, Gn)) := E∗(α0(F ∗n , G∗n))− α0(Fn, Gn)
Defineαn,BOOT := α0(Fn, Gn)− biasBOOT(α0(Fn, Gn)).
Reject H0 if √n(αn,BOOT − α) < σnΦ−1(β),
Test asympt. of level β
Eustasio del Barrio Testing approximate stochastic order 29 / 37
Implementation, simulation & data example Implementation issues
Testing for essential stochastic order: finite sample performance
Consider again H0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
Rejection of H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
σ2α = 1−α2
2 , asympt. of level β; type I-type II error probs. exponentially → 0
σα from worst case choice; possible improvement from estimated σn
A more important improvement: E(α0(Fn, Gn)) ≥ α0(F,G); estimate bias by
biasBOOT(α0(Fn, Gn)) := E∗(α0(F ∗n , G∗n))− α0(Fn, Gn)
Defineαn,BOOT := α0(Fn, Gn)− biasBOOT(α0(Fn, Gn)).
Reject H0 if √n(αn,BOOT − α) < σnΦ−1(β),
Test asympt. of level β
Eustasio del Barrio Testing approximate stochastic order 29 / 37
Implementation, simulation & data example Implementation issues
Testing for essential stochastic order: finite sample performance
Consider again H0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
Rejection of H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
σ2α = 1−α2
2 , asympt. of level β; type I-type II error probs. exponentially → 0
σα from worst case choice; possible improvement from estimated σn
A more important improvement: E(α0(Fn, Gn)) ≥ α0(F,G); estimate bias by
biasBOOT(α0(Fn, Gn)) := E∗(α0(F ∗n , G∗n))− α0(Fn, Gn)
Defineαn,BOOT := α0(Fn, Gn)− biasBOOT(α0(Fn, Gn)).
Reject H0 if √n(αn,BOOT − α) < σnΦ−1(β),
Test asympt. of level β
Eustasio del Barrio Testing approximate stochastic order 29 / 37
Implementation, simulation & data example Implementation issues
Testing for essential stochastic order: finite sample performance
Consider again H0 : α0(F,G) ≥ α, vs. Ha : α0(F,G) < α
Rejection of H0 if √n(α0(Fn, Gn)− α) < σαΦ−1(β),
σ2α = 1−α2
2 , asympt. of level β; type I-type II error probs. exponentially → 0
σα from worst case choice; possible improvement from estimated σn
A more important improvement: E(α0(Fn, Gn)) ≥ α0(F,G); estimate bias by
biasBOOT(α0(Fn, Gn)) := E∗(α0(F ∗n , G∗n))− α0(Fn, Gn)
Defineαn,BOOT := α0(Fn, Gn)− biasBOOT(α0(Fn, Gn)).
Reject H0 if √n(αn,BOOT − α) < σnΦ−1(β),
Test asympt. of level βEustasio del Barrio Testing approximate stochastic order 29 / 37
Implementation, simulation & data example Simulation setup
Fα,a ≡ U(a, 1 + a); Fα,b ≡ 1−α2 U(0, 1+α
2 ) + 1+α2 U( 1+α
2 , 1 + α(1−α)2 )
G ≡ U(0, 1)
Fα,a Fα,b
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
0.0
0.2
0.4
0.6
0.8
1.0
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Fα,a, G worst choice in test against essential s.o.
Fα,b, G worst choice in test for essential s.o.
Eustasio del Barrio Testing approximate stochastic order 30 / 37
Implementation, simulation & data example Simulation results
Testing for essential stochastic order
Table : Observed rejection frequencies. H0 : α0(F,G) ≥ α vs. Ha : α0(F,G) < αG = U(0, 1), m = n; reject if
√n(α0(Fn, Gn)− α) < σαΦ−1(0.05)
α n F0.1,a F0.1,b F0.05,a F0.05,b F0.01,a F0.01,b F0
0.01
50 0.000 0.000 0.000 0.000 0.000 0.000 0.000100 0.000 0.000 0.000 0.000 0.000 0.000 0.000500 0.000 0.000 0.000 0.000 0.000 0.000 0.000
1000 0.000 0.000 0.000 0.000 0.000 0.000 0.0005000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.05
50 0.000 0.000 0.000 0.000 0.000 0.000 0.000100 0.000 0.000 0.000 0.000 0.000 0.000 0.000500 0.000 0.000 0.000 0.000 0.000 0.000 0.000
1000 0.000 0.000 0.000 0.000 0.016 0.071 0.1835000 0.000 0.000 0.000 0.008 0.924 0.939 0.995
0.1
50 0.000 0.000 0.000 0.000 0.000 0.000 0.000100 0.000 0.000 0.000 0.000 0.000 0.000 0.000500 0.000 0.001 0.010 0.152 0.532 0.589 0.698
1000 0.000 0.011 0.155 0.491 0.946 0.952 0.9795000 0.000 0.025 0.996 1.000 1.000 1.000 1.000
Nonasymptotic estimate n = 8143
Eustasio del Barrio Testing approximate stochastic order 31 / 37
Implementation, simulation & data example Simulation results
Testing for essential stochastic order
Table : Observed rejection frequencies. H0 : α0(F,G) ≥ α vs. Ha : α0(F,G) < αG = U(0, 1), m = n; reject if
√n(αn,BOOT − α) < σnΦ−1(0.05)
α n F0.1,a F0.1,b F0.05,a F0.05,b F0.01,a F0.01,b F0
0.01
50 0.000 0.003 0.001 0.012 0.019 0.034 0.038100 0.000 0.001 0.000 0.005 0.015 0.027 0.028500 0.000 0.000 0.000 0.001 0.016 0.028 0.049
1000 0.000 0.000 0.000 0.000 0.013 0.030 0.0635000 0.000 0.000 0.000 0.000 0.019 0.038 0.136
0.05
50 0.000 0.009 0.015 0.042 0.062 0.067 0.093100 0.000 0.006 0.008 0.038 0.065 0.098 0.106500 0.000 0.003 0.007 0.058 0.208 0.220 0.332
1000 0.000 0.001 0.009 0.039 0.415 0.426 0.5665000 0.000 0.000 0.003 0.057 0.978 0.987 1.000
0.1
50 0.003 0.030 0.054 0.089 0.137 0.138 0.134100 0.007 0.052 0.076 0.121 0.246 0.250 0.266500 0.007 0.040 0.337 0.387 0.801 0.830 0.876
1000 0.005 0.056 0.589 0.661 0.976 0.985 0.9975000 0.003 0.057 0.999 1.000 1.000 1.000 1.000
Eustasio del Barrio Testing approximate stochastic order 32 / 37
Implementation, simulation & data example Simulation results
Testing against essential stochastic order
Table : Observed rejection frequencies. H0 : α0(F,G) ≤ α vs. Ha : α0(F,G) > αG = U(0, 1), m = n; reject if
√n(α0(Fn, Gn)− α) > K0.95(α)
α n F0 F0.01,b F0.01,a F0.05,b F0.05,a F0.1,b F0.1,a
0.01
50 0.045 0.039 0.052 0.060 0.111 0.159 0.302100 0.022 0.031 0.040 0.066 0.107 0.199 0.450500 0.021 0.026 0.045 0.210 0.477 0.951 1.000
1000 0.011 0.028 0.047 0.441 0.774 1.000 1.0005000 0.002 0.017 0.049 1.000 1.000 1.000 1.000
0.05
50 0.015 0.010 0.016 0.023 0.052 0.056 0.142100 0.004 0.007 0.009 0.014 0.031 0.069 0.186500 0.000 0.001 0.002 0.009 0.047 0.211 0.664
1000 0.000 0.000 0.000 0.001 0.060 0.606 0.9545000 0.000 0.000 0.000 0.001 0.040 1.000 1.000
0.1
50 0.001 0.002 0.005 0.004 0.007 0.009 0.027100 0.000 0.003 0.000 0.001 0.002 0.010 0.031500 0.000 0.000 0.000 0.000 0.001 0.002 0.056
1000 0.000 0.000 0.000 0.000 0.000 0.001 0.0355000 0.000 0.000 0.000 0.000 0.000 0.000 0.056
Eustasio del Barrio Testing approximate stochastic order 33 / 37
Implementation, simulation & data example Simulation results
Testing for essential stochastic order, dependent case
Table : Observed rejection frequencies. H0 : α0(F,G) ≥ α vs. Ha : α0(F,G) < αG = U(0, 1), m = n; reject if
√n(α0(Fn, Gn − α) < σnΦ−1(0.05)
α n H0.1,a H0.1,b H0.05,a H0.05,b H0.01,a H0.01,b H0
0.01
50 0.000 0.006 0.000 0.026 0.003 0.084 0.019100 0.000 0.001 0.000 0.006 0.003 0.042 0.009500 0.000 0.000 0.000 0.000 0.000 0.022 0.009
1000 0.000 0.000 0.000 0.000 0.000 0.028 0.0125000 0.000 0.000 0.000 0.000 0.000 0.043 0.049
0.05
50 0.000 0.006 0.000 0.031 0.008 0.090 0.025100 0.000 0.010 0.000 0.034 0.020 0.140 0.028500 0.000 0.001 0.000 0.061 0.094 0.210 0.167
1000 0.000 0.000 0.000 0.076 0.209 0.295 0.3525000 0.000 0.000 0.000 0.047 0.956 0.839 0.999
0.1
50 0.000 0.044 0.013 0.128 0.071 0.191 0.085100 0.001 0.053 0.028 0.130 0.116 0.240 0.153500 0.000 0.040 0.155 0.263 0.640 0.567 0.783
1000 0.000 0.044 0.357 0.397 0.956 0.869 0.9915000 0.000 0.048 0.999 0.973 1.000 1.000 1.000
Hπ,a independent marginals U(π, 1 + π), U(0, 1); H0 = H0,a
Hπ,b marginals F ≡ 1−π2 U(0, 1+π
2 ) + 1+π2 U( 1+π
2 , 1 + π(1−π)2 ), G ≡ U(0, 1),
copula C(s, t) = (s+ t− 1)+
Eustasio del Barrio Testing approximate stochastic order 34 / 37
Implementation, simulation & data example Case study
Data: National Health and Nutrition Examination Survey
Evolution with age of the heights of boys and girls
Sample sizes by age (boys, top)2 3 4 5 6 7 8 9 10 11 12 13 14 15
796 632 633 563 557 582 579 543 556 556 735 728 704 716776 563 620 567 542 564 572 579 536 587 733 757 764 665
130 140 150 160
0.0
0.2
0.4
0.6
0.8
1.0
π(Fn,Gm) = 0.128409 π(Gm,Fn) = 0.009342
heights
ED
F's
Gm Age 10 Boys Fn Age 10 Girls
Eustasio del Barrio Testing approximate stochastic order 35 / 37
Implementation, simulation & data example Case study
95%-Upper bounds by age for α0(F a, Ga) (top row) and α0(Ga, F a) (bottom)
2 3 4 5 6 7 8 9 10 11 12 13 140.00 0.00 0.00 0.01 0.00 0.04 0.02 0.04 0.16 0.16 0.15 0.05 0.010.17 0.09 0.14 0.18 0.15 0.11 0.13 0.08 0.03 0.01 0.07 0.28 0.47
5 10 15
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Upper 95% confidence bounds for the stochastic dominance levels
age
Statistical evidence that girls are taller than boys at 10-11
Eustasio del Barrio Testing approximate stochastic order 36 / 37
Conclusions
Conclusions
Trimmed stochastic order models capture adequately deviations from exactstochastic order
Provided valid inference models/methods
Valid testing procedures with controlled error probabilities
Nonasymptotic bounds; uniformly exponentially consistent tests
Good finite sample performance through bootstrap correction
Thanks for your attention!
Eustasio del Barrio Testing approximate stochastic order 37 / 37
Conclusions
Conclusions
Trimmed stochastic order models capture adequately deviations from exactstochastic order
Provided valid inference models/methods
Valid testing procedures with controlled error probabilities
Nonasymptotic bounds; uniformly exponentially consistent tests
Good finite sample performance through bootstrap correction
Thanks for your attention!
Eustasio del Barrio Testing approximate stochastic order 37 / 37