On Sensitivity Value of Pair-MatchedObservational Studies

Qingyuan Zhao

Department of Statistics, University of Pennsylvania

August 2nd, JSM 2017

Manuscript and slides are available athttp://www-stat.wharton.upenn.edu/~qyzhao/.

1/12

Sensitivity Analysis

I Observational studies = Treatment is not randomized.

I The core but unverifiable assumption: treatment ignorability,aka no unmeasured confounding. (Fisher’s criticism of“smoking causes lung cancer”.)

I Sensitivity analysis: what if this assumption is violated (in acontrolled way captured by one or a few sensitivityparameters)?

I There is a long list of approaches of sensitivity analysis. I willconsider Rosenbaum’s sensitivity analysis for a pair-matchedstudy. (Cornfield’s response to Fisher.)

2/12

What does a sensitivity analysis look like?

I Sensitivity parameter Γ ≥ 1 in Rosenbaum’s model: withineach matched pair,

1/Γ ≤ odds ratio(1st unit treated, 2nd unit treated) ≤ Γ.

I Γ = 1 corresponds to ignorable treatment. Can test the sharpnull hypothesis by e.g. Wilcoxon’s signed rank test.

I When Γ > 1: Rosenbaum obtained lower and upper bounds ofthe p-value of any signed score test.

I An example:

probe set sensitivity analysis sensitivity value

37583 atΓ 1 2 3 5 7 10 1.84 2.44pΓ 0.00 0.02 0.13 0.60 1.00 1.00 0.01 0.05

3/12

Sensitivity value

Definition (“Truncated” sensitivity value)

Γ∗∗α = inf

{Γ ≥ 1 | pΓ > α

}.

Interpretation: If the unmasured confounder changes thewithin-pair odds ratio of treatment by more than Γ∗∗

α , then thesharp null hypothesis could be not significant.

I Γ∗∗α > 1 iff p1 = p1 ≤ α. What if the null hypothesis is not

significant even when Γ = 1?

I Mathematically, pΓ can be defined for 0 < Γ < 1 as well. It ismore convenient to work with Γ∗

α that takes the infimum overΓ > 0.

4/12

Some motivations I

I Sensitivity value is a concise summary of the study’s“sensitivity to measured confounding”.

I A value vs. A table.

I Analogy: p-value for a randomized experiment vs. sensitivityvalue for an observational study.

5/12

Some motivations II

I Asymptotics of sensitivity value ↔ Power of sensitivityanalysis:

P(Γ∗α ≥ Γ) = P(pΓ ≤ α).

I The “favorable situation”: no unmeasured confounding andnonzero causal effect.

I Fixed Γ asymptotics: Rosenbaum [2015] considered theBahadur efficiency of a sensitivity analysis by studying howfast pΓ → 0.

I Fixed α asymptotics: examine the distribution of Γ∗α.

6/12

Background

I A general and common strategy is to use the signed score test(Yi is the within-pair difference)

T =

∑Ii=1 sgn(Yi )qi∑I

i=1 qi

, qi = ψ

(rank(|Yi |)

I + 1

).

I Rosenbaum found bounding variable T̄Γ in the sense that

P(T ≥ t|F) ≤ P(T Γ ≥ t|F) = pΓ,

I CLT for T̄Γ:

√I · T Γ − Γ/(1 + Γ)√

Γ/(1 + Γ)2σ2q,I

d→ N(0, 1), σ2q,I =

I−1∑I

i=1 q2i(

I−1∑I

i=1 qi

)2.

7/12

CLT for sensitivity value

Theorem (Z, 2017)Suppose Yi

i.i.d.∼ F and√

I · (T − µF )/σFd→ N(0, 1), the transformed

sensitivity value κ∗α = Γ∗α/(1 + Γ∗

α) for fixed 0 < α < 1 satisfies

√I ·[κ∗α − µF

]d→ N

(− σqΦ̄−1(α)

√µF (1− µF ), σ2

F

).

κ*

µFc(µF, σq) I

σF I

8/12

Design sensitivity

κ*

µFc(µF, σq) I

σF I

I Rosenbaum [2004] noticed a phase transition at µF :I If κ > µF , sensitivity analysis has no asymptotic power.I If κ < µF , power → 1 as I →∞.I He calls the value µF/(1− µF ) “design sensitivity”.

I The CLT for sensitivity value separates the contribution bydesign and by obtaining more sample.

8/12

Design sensitivity

κ*

µF

I Rosenbaum [2004] noticed a phase transition at µF :I If κ > µF , sensitivity analysis has no asymptotic power.I If κ < µF , power → 1 as I →∞.I He calls the value µF/(1− µF ) “design sensitivity”.

I The CLT for sensitivity value separates the contribution bydesign and by obtaining more sample.

8/12

Design sensitivity

κ*

µF

I Rosenbaum [2004] noticed a phase transition at µF :I If κ > µF , sensitivity analysis has no asymptotic power.I If κ < µF , power → 1 as I →∞.I He calls the value µF/(1− µF ) “design sensitivity”.

I The CLT for sensitivity value separates the contribution bydesign and by obtaining more sample.

9/12

Implication: select statistics

I The goal: maximize Γ∗α stochastically.

√I ·[κ∗α − µF

]d→ N

(− σqΦ̄−1(α)

√µF (1− µF ), σ2

F

).

I The expecation of Γ∗α is determined by two quantities:

1. µF =< ψ, gF >;2. σq = ‖ψ‖2

2/‖ψ‖21.

I Tradeoff: want to maximize µF without making σq too large.

I Manuscript has detailed comparison of different choices of ψunder different F .

I Practically, can estimate µF by sample splitting and thendecide.

10/12

Implication: select subgroupsI Hsu, Small, and Rosenbaum [2013] noticed a dilemma.

Suppose there are two subgroups with unequal µF .Heuristically,

I If I →∞, should use the subgroup with larger µF .I If I is small, should combine the samples.

I Can use the asymptotics to compute the critical sample size.I For example, for Noether’s statistics ψ(u) ≡ 1 (ratio of

sample sizes below: 1:1 and 3:1).

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●●

●●●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2

prefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer both

10

30

100

300

1000

3000

10000

−0.2 −0.1 0.0 0.1 0.2µF1 − µF2

criti

cal s

ampl

e si

ze

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1

prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2

prefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer both

10

30

100

300

1000

3000

10000

−0.2 −0.1 0.0 0.1 0.2µF1 − µF2

criti

cal s

ampl

e si

ze

10/12

Implication: select subgroupsI Hsu, Small, and Rosenbaum [2013] noticed a dilemma.

Suppose there are two subgroups with unequal µF .Heuristically,

I If I →∞, should use the subgroup with larger µF .I If I is small, should combine the samples.

I Can use the asymptotics to compute the critical sample size.I For example, for Noether’s statistics ψ(u) ≡ 1 (ratio of

sample sizes below: 1:1 and 3:1).

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●●

●●●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2

prefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer both

10

30

100

300

1000

3000

10000

−0.2 −0.1 0.0 0.1 0.2µF1 − µF2

criti

cal s

ampl

e si

ze

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1prefer group 1

prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2prefer group 2

prefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer bothprefer both

10

30

100

300

1000

3000

10000

−0.2 −0.1 0.0 0.1 0.2µF1 − µF2

criti

cal s

ampl

e si

ze

11/12

Implication: select hypotheses

I One treatment and hundreds of outcomes that are susceptibleto unmeasured confounding.

I Can use sensitivity value to screen causal hypotheses.

I The manuscript has an application to genomics screening.Wilcoxon (8,7,8) (8,6,7)

−4 −2 0 2 4 −4 −2 0 2 4 −4 −2 0 2 4

0.00

0.25

0.50

0.75

1.00

Quantile of N(0, 1)

Sam

ple

quan

tile

of κ

*

histogram

0 5 10 15

0.00

0.25

0.50

0.75

1.00

density

Wilcoxon

(8,7,8)

(8,6,7)

I Zhao, Small, and Rosenbaum [2017] proposed a relatedmethod called “Cross Screening”.

12/12

References

I Manuscript:I Q. Zhao. On sensitivity value of pair-matched observational

studies. arXiv:1702.03442.I Additional references:

J. Y. Hsu, D. S. Small, and P. R. Rosenbaum. Effect modification and designsensitivity in observational studies. Journal of the American StatisticalAssociation, 108(501):135–148, 2013.

P. R. Rosenbaum. Design sensitivity in observational studies. Biometrika, 91(1):153–164, 2004.

P. R. Rosenbaum. Bahadur efficiency of sensitivity analyses in observationalstudies. Journal of the American Statistical Association, 110(509):205–217,2015.

Q. Zhao, D. S. Small, and P. P. Rosenbaum. Cross-screening in observationalstudies that test many hypotheses. arXiv:1703.02078, 2017.