Embed Size (px)
Citation preview
Robust Learning under Uncertain Test Distributions:Relating Covariate Shift to Model Misspecification
Junfeng Wen1 [email protected] Yu2 [email protected] Greiner1 [email protected] of Computing Science, University of Alberta, Edmonton, AB T6G 2E8 CANADA2Bell Labs, Alcatel-Lucent, 600 Mountain Avenue, Murray Hill, NJ 07974 USA
AbstractMany learning situations involve learning theconditional distribution ppy|xq when the traininginstances are drawn from the training distribu-tion ptrpxq, even though it will later be used topredict for instances drawn from a different testdistribution ptepxq. Most current approaches fo-cus on learning how to reweigh the training ex-amples, to make them resemble the test distribu-tion. However, reweighing does not always help,because (we show that) the test error also de-pends on the correctness of the underlying modelclass. This paper analyses this situation by view-ing the problem of learning under changing dis-tributions as a game between a learner and an ad-versary. We characterize when such reweighingis needed, and also provide an algorithm, robustcovariate shift adjustment (RCSA), that providesrelevant weights. Our empirical studies, on UCIdatasets and a real-world cancer prognostic pre-diction dataset, show that our analysis applies,and that our RCSA works effectively.
1. IntroductionTraditional machine learning often explicitly or implicitlyassumes that the data used for training a model come fromthe same distribution as that of the test data. However, thisassumption is violated in many real-world applications. Forexample, biostatisticians often try to collect a large and di-verse training set, perhaps for building prognostic predic-tors for patients with different diseases. When cliniciansdeploy these predictors, they do not know whether the lo-cal test patient population will be even close to that trainingpopulation. Sometimes we can collect a small sample fromthe target test population, but in most cases we have noth-
Proceedings of the 31 st International Conference on MachineLearning, Beijing, China, 2014. JMLR: W&CP volume 32. Copy-right 2014 by the author(s).
ing more than weak prior knowledge about how the testdistribution may shift, such as anticipated changes in gen-der ratio or age distribution. It is useful to build predictorsthat are robust against such changes in test distributions.
In this work, we investigate the problem of distribu-tion change under covariate shift assumption (Shimodaira,2000), in which both training and test distributions sharethe same conditional distribution ppy|xq, while theirmarginal distributions, ptrpxq and ptepxq, are different. Tocorrect the shifted distribution, major efforts have beendedicated to importance reweighing (Quionero-Candelaet al., 2009; Sugiyama & Kawanabe, 2012). However,reweighing methods will not necessarily improve the per-formance in test set, as prediction accuracy under covariateshift is also dependent on model misspecification (White,1981). Fig. 1 shows three examples of misspecified mod-els, where we are considering the model class of straightlines of the form y“ax`b, for xPr´1.5, 2.5s. In Fig. 1(a),no straight line is a good fit for the cubic curve acrossthe whole interval, but Model 2 fits the curve reasonablywell in the small interval r´0.5, 0.5s. If training data isspread all over r´1.5, 2.5s while test data concentrates onr´0.5, 0.5s, improvement via reweighing could be signif-icant. The situation in Fig. 1(b) is different: although thetrue model is a curve and not a straight line, the best linearfit is no more than ε away from the value of the true model.In this case, no matter what test distributions we see in theinterval r´1.5, 2.5s, the regression loss of the best linearmodel will never be more than ε from the Bayes optimalloss. In Fig. 1(c), the true model is a straight line except atx “ 0; perhaps this outlier is a cancer patient whose tumourspontaneously disappeared on its own. Unless the test dis-tribution concentrates most of its mass at x “ 0, the straightline fit learned from the training data over the interval willstill be a very good predictor. Sometimes we can rule outthis type of covariate shift through prior knowledge. If suchoutliers are extremely rare during training time, we wouldnot expect the test population to have many such patients.Reweighing will not help much in cases 1(b) and 1(c).
Robust Learning under Uncertain Test Distributions
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5−2
0
2
4
6
8
10
12
14
16
Input
Outp
ut
True modelModel 1Model 2
(a) Large misspecification.
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5−2
−1
0
1
2
3
4
5
Input
Outp
ut
↑
↓ε
True modelBest linear fit
(b) Small misspecification.
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5−0.5
0
0.5
1
1.5
2
2.5
3
3.5
Input
Ou
tpu
t
True model
(c) Single point misspecification.
Figure 1. Three different scenarios of model misspecifications.
In this paper, we relate covariate shift to model misspecifi-cation and investigate when reweighing can help a learnerdeal with covariate shift. We introduce a game between alearner and an adversary that performs robust learning. Thelearner chooses a model θ from a set Θ to minimize theloss, while the adversary chooses a reweighing function αfrom a set A to create new test distributions to maximize theloss. There are two major contributions in this paper: First,we provide an improved understanding of the relation be-tween covariate shift and model misspecification throughthis game analysis. If the learner can find a θ that min-imizes the loss against any possible α that the adversarycan play, then it is not necessary to perform reweighingagainst covariate shift scenarios represented by A. Sec-ond, we provide a systematic method for checking a modelclass Θ against different covariate shift scenarios, such aschanging gender ratio and age distributions in the prognos-tic predictor example, to help user decide whether impor-tance reweighing would be beneficial.
For practical use, our method can be used to decide if themodel class is sufficient against shifts that are close to a testsample; or robust against a known range of potential shiftsif test sample is unavailable. If the model class is insuffi-cient, we can consider different ways to deal with covariateshifts, such as reweighing using unlabelled test samples, orexploring a different model class for the problem.
2. Related WorkOur work is inspired by Grunwald & Dawid (2004), whointerpret maximum entropy as a game between an adver-sary and a learner on minimizing the worst case expectedlog loss. Teo et al. (2008) and Globerson & Roweis (2006)also consider an adversarial scenario under changing testset conditions, but they are concerned with corruption ordeletion of features rather than covariate shift.
Many results on covariate shift correction involve densityratio estimation. Shimodaira (2000) showed that, given co-variate shift and model misspecification, reweighing eachinstance with ptepxqptrpxq is asymptotically optimal for
log-likelihood estimation, where ptrpxq and ptepxq are as-sumed to be known or estimated in advance. Sugiyama& Muller (2005) extended this work by proposing an(almost) unbiased estimator for L2 generalization error.There are several works focusing on minimizing differ-ent types of divergence between distributions in the liter-ature (Kanamori et al., 2008; Sugiyama et al., 2008; Ya-mada et al., 2011). Kernel mean matching (KMM) (Huanget al., 2007) reweighs instances to match means in aRKHS (Scholkopf & Smola, 2002). Our work and someother approaches (Pan et al., 2009) adapt the idea of match-ing means of the datasets to correct shifted distribution,but we extend their approaches from a two-step optimiza-tion to a game framework that jointly learns a modeland weights with covariate shift correction. Some otherapproaches (Zadrozny, 2004; Bickel et al., 2007; 2009;Storkey & Sugiyama, 2007) consider different generativemodels for special cases of covariate shift mechanisms.
Besides these approaches, there are many other works fo-cusing on the theoretical analysis of statistical learningbounds for covariate shift. Ben-David et al. (2007) gavea bound on L1 generalization error given the presence ofmismatched distributions. Analyses on other forms of errorwere also introduced in the literature (Shimodaira, 2000;Sugiyama & Muller, 2005; Cortes et al., 2010). However,most of these analyses neglect the effect of model misspeci-fication (White, 1981). Apart from Shimodaira (2000) whopointed out a link between covariate shift and model mis-specification with some quantitative evidence, and Huanget al. (2007) who observed that simpler models tend to ben-efit more from density ratio correction, few have addressedthe question of determining when reweighing helps versuswhen it is not needed. In this paper, we show the relation-ship between covariate shift and model misspecification.
3. Learning Under Uncertain TestDistributions as a Game
Suppose we are given an i.i.d. (independent and identi-cally distributed) training sample tpx1, y1q, ¨ ¨ ¨ , pxn, ynqu
Robust Learning under Uncertain Test Distributions
drawn from a joint distribution ptrpx, yq, and know that thetest distribution ptepx, yq is the same as ptrpx, yq. The mostcommon and well-established method to learn a predictionfunction f : X ÞÑ Y is through solving the following em-pirical risk minimization (ERM) problem:
minθPΘ
1
n
ÿn
i“1lpfθpxiq, yiq ` λ Ωpθq, (1)
where the prediction function fθp¨q is parametrized by avector θ, lp¨, ¨q is a loss function, Ωp¨q is a regularizer on θto control overfitting and λ P R is regularization parameter.
When there is covariate shift, the feature distribution ptepxqis different from ptrpxq but the conditional distributionppy|xq representing the classification/regression rule re-mains the same. In this scenario, one of the most com-mon approach to correct for the effect of covariate shift isto reweigh the training instances in the ERM problem toreflect their true proportions on the test set:
minθPΘ
1
n
ÿn
i“1wpxiq lpfθpxiq, yiq ` λ Ωpθq, (2)
wherewpxiq is a reweighing function that approximates thedensity ratio ptepxiqptrpxiq. There are different methodsfor estimating the density ratio wpxq using unlabelled testdata (Quionero-Candela et al., 2009; Sugiyama & Kawan-abe, 2012). This suggests viewing the learning problemas a two-step estimation problem, where the density ratiowpxq is estimated first before estimating θ.
Interestingly, in econometrics the phenomenon of covari-ate shift has been used to detect model misspecification.White (1981) considered the problem of detecting modelmisspecification in non-linear least squares regression formodels yi“fθpxiq`εi, where εi is i.i.d. noise. He showedthat under certain assumptions, when there is no misspeci-fication, the objective and solution θ‹ of the problem
minθ
1
n
ÿn
i“1pyi ´ fθpxiqq
2
converge to the same limits as the reweighed problem:
minθ
ÿn
i“1wipyi ´ fθpxiqq
2,
for any fixed set of non-negative weights wi such thatř
i wi “ 1. He then derived several misspecification testsbased on asymptotic approximation of the difference of thesolutions. The key idea in his work is to detect model mis-specification by creating his own covariate shift wi, so thatcorrect inference on the effects of different variables in re-gression can be performed.
In this paper, we explore White’s main insight further bymodelling the reweighing functions wpxiq as an adversaryin a game against the learner. Instead of detecting modelmisspecification, we want to tell whether density ratio cor-rection is needed under a set of potential distribution shifts.
The rest of this section will introduce our game formula-tion. Section 4 will then explain how it can be used to de-tect whether density ratio correction is needed or not.
We tie the two problems of density ratio estimation andlearning a predictor together through the robust Bayesframework (Grunwald & Dawid, 2004). The learner triesto minimize the loss by selecting a model θ P Θ, while theadversary tries to maximize the loss by selecting a reweigh-ing function w P W . Formally, we model the learningproblem as a (regularized) minimax game:
minθPΘ
maxwPW
1
n
ÿn
i“1wpxiq lpfθpxiq, yiq ` λ Ωpθq. (3)
The learner can be seen as minimizing the worst case lossover the set of test distributions W produced by the ad-versary. The definition of the strategy set W used by theadversary is important in our approach, as it determinesthe extent to which any model misspecification can be ex-ploited by the adversary to increase the loss. Depending onthe application scenario, it can be defined using our priorknowledge on how the test distributions could change, orbased on unlabelled test data if they are available.
To refine this formulation, we assume the reweighing func-tions wpxq are linearly parametrized:
wαpxq “ÿk
j“1αjkjpxq, (4)
where α contains the mixing coefficients and kjpxq arenon-negative basis functions. For example, each kjpxqcould be a non-negative kernel function, say, the Gaussiankernel
Kpbj , xq “ exp`
´||bj ´ x||22σ2
˘
(5)
with basis bj , or it could be Ijpxq, the indicator function forthe jth disjoint group of the data, representing groups fromdifferent genders, age ranges, or k-means clusters, etc. Itcould be viewed as the conditional probability ppx|jq ofobserving x given class j in a mixture model. As for α,it is generally constrained to lie in some compact subspaceA of the non-negative quadrant of Euclidean space. Thislinear formulation is flexible enough to capture many dif-ferent types of uncertainties in the test distributions, andyet simple enough to be solved efficiently as a convex op-timization problem. Therefore, we consider uncertain testdistributions and optimize the following minimax game:
minθPΘ
maxαPRk
1
n
ÿn
i“1wαpxiq lpfθpxiq, yiq ` λ Ωpθq
s.t.1
n
ÿn
i“1wαpxiq “ 1, 0 ď αj ď B.
(6)
The sum-to-one normalization constraint ensures thatwαpxq behaves like a Radon-Nikodym derivative that prop-erly reweighs the training distribution to a potential test dis-tribution (Shimodaira, 2000; Sugiyama et al., 2008). The
Robust Learning under Uncertain Test Distributions
boundB P R on the parameters αj ensure that the reweigh-ing function wαpxq is bounded, which naturally controlsthe capacity of the adversary. In this formulation, the strat-egy set1 An of the adversary is the intersection of a hyper-cube and an affine subspace:
An “
"
α
ˇ
ˇ
ˇ
ˇ
1
n
ÿn
i“1wαpxiq “ 1, 0 ď αj ď B
*
, (7)
which is closed and convex. For the games defined abovebetween the learner and the adversary, a minimax solutionpθ˚,α˚q exists (Rockafellar, 1996, Corollary 37.3.2).
3.1. Solving the Training Problem
We first define the empirical adversarial loss as
LAnpθq “ maxαPAn
1
n
ÿn
i“1wαpxiq lpfθpxiq, yiq. (8)
The training problem in Eq. (6) can be solved efficientlyfor loss functions lpfθp¨q, ¨q that are convex in θ. NoticeEq. (8) is a convex function in θ if lpfθp¨q, ¨q is convex inθ, as we are taking the maximum over a set of convex func-tions. A subgradient of LAnpθ
1q at a point θ1 is:
B
BθLAnpθ
1q “1
n
ÿn
i“1wα1pxiq
B
Bθlpfθ1pxiq, yiq, (9)
where α1 is the solution of the problem with θ1 fixed:
α1 “ argmaxαPAn
1
n
ÿn
i“1wαpxiq lpfθ1pxiq, yiq. (10)
Since the strategy set An is linearly constrained and theobjective is also linear, we can easily use linear program-ming to solve for α1 in Eq. (10). Knowing how to computethe subgradient, we can treat the robust training problemas a convex ERM problem with the adversarial loss. Theoptimization problem can be solved efficiently with sub-gradient methods or bundle methods (Kiwiel, 1990).
3.2. Incorporating Unlabelled Test Data
If unlabelled test data txn`1, . . . , xn`mu are available, wewould require the reweighing functions wαpxq used by theadversary to produce test distributions that are close to theunlabelled data, especially when covariate shift occurs. Inthis case we can further restrict the strategy set An of theadversary via moment matching constraints (MMC):
minθPΘ
maxαPRk
1
n
ÿn
i“1wαpxiq lpfθpxiq, yiq ` λΩpθq
s.t.1
n
ÿn
i“1wαpxiq “ 1, 0 ď αj ď B
1
n
ÿn
i“1wαpxiqφpxiq “
1
m
ÿn`m
i“n`1φpxiq, (11)
1We use the subscript n to denote its dependence on the sam-ple tx1, ¨ ¨ ¨ , xnu.
where φp¨q are feature functions similar to those usedin maximum entropy models (Berger et al., 1996). LetKnα “ φte represent the linear constraint of Eq. (11),then the strategy set of the adversary becomes the closedconvex set:
AMMCn “
"
α
ˇ
ˇ
ˇ
ˇ
1
n
ÿn
i“1wαpxiq“1, 0ďαjďB,Knα“ φte
*
.
In practice, it might not be feasible to satisfy all the momentmatching constraints. It is also unwise to enforce these ashard constraints, as the small test sample might not be rep-resentative of the true test distribution. We can incorporatean L1 or L2 penalty on the constraint violations similar toAltun & Smola (2006) while retaining convexity of the op-timization problem. The details are not shown here due tospace constraints. We refer to Eq. (11) as robust covariateshift adjustment (RCSA).
4. Relating Covariate Shift to ModelMisspecification
This section relates covariate shift to model misspecifica-tion and describes a procedure for testing whether correct-ing for covariate shift could be needed, assuming the testdistribution comes from the strategy set A of the adversary.We will also state and discuss several theoretical results tojustify our test. Their proofs are in Appendix A. We startwith a definition:Definition 1 (Pointwise Domination). A parameter θ‹ issaid to pointwisely dominate all θ1 P Θ over the loss func-tion lp¨, ¨q if, for all x P X and for all θ1 P Θ,ż
lpfθ‹pxq, yqppy|xqdy ď
ż
lpfθ1pxq, yqppy|xqdy. (12)
This condition means there is a single θ‹ that pointwiselyminimizes the loss l for any x P X . It is easy to see thatthis pointwise domination condition is implied by the tra-ditional definition of correctly specified model class whenlp¨, ¨q is the log loss, ´ log pθpy|xq. With log loss, thepointwise domination condition then becomes:
´
ż
ppy|xq log pθ‹py|xqdy ď ´
ż
ppy|xq log pθ1py|xqdy.
This inequality always holds because pθ‹py|xq “ ppy|xqminimizes the entropy on the left hand side. Therefore, acorrectly specified model always implies the existence ofa pointwise dominator θ‹. However, the converse is notalways true, as the underlying model class Θ might be tooweak (e.g., if Θ contains only a single model θ).
Note that pointwise domination condition does not dependon the marginal distribution ppxq. If we can find such a
Robust Learning under Uncertain Test Distributions
pointwise dominator, then the test distribution can be arbi-trarily shifted without damaging the performance of point-wise dominator θ‹. However, this condition is too strin-gent and is almost never true on real data. This motivatesus to consider the game formulation in Section 3: Insteadof requiring θ to minimize the loss at every single point x,we require θ to minimize the loss against every reweighingfunction wαpxq that the adversary can play. We define
A8 “"
α P AS
ˇ
ˇ
ˇ
ˇ
ż
wαpxq dF px, yq “ 1
*
,
where AS is the support of the strategy set of the adver-sary that does not depend on training samples (e.g., the hy-percube 0 ď αj ď B), and F px, yq is the joint trainingdistribution of px, yq. Now we define dominant strategy:Definition 2 (Dominant Strategy). We say that θ: P Θ is adominant strategy for the learner if, for all α P A8, for allθ1 P Θ,ż
wαpxqlpfθ:pxq, yqdF px, yqď
ż
wαpxqlpfθ1pxq, yqdF px, yq.
It is easy to show that the pointwise domination conditionimplies the existence of dominant strategy.Theorem 3. Suppose a pointwise dominator θ‹ exists, thenθ‹ is also a dominant strategy for the learner, against anybounded adversarial set A.
The existence of a dominant strategy of the learner is thekey criterion in deciding whether density ratio correctionis necessary. If such a dominator θ: exists, then it giveslower or equal loss compared to other models θ1, no mat-ter which reweighing function wαpxq is used. Thus if onecan find θ:, no density ratio correction is needed in ex-pectation, since θ: is asymptotically optimal as long as thetraining and test distributions come from the given adver-sarial set. However, if no such dominator exists, then forany model θ, there exists another model θ1 and a reweigh-ing functionwα1pxq such that θ1 has strictly lower loss thanθ on wα1pxq. This means that a reweighing wα1pxq and itscorresponding model θ1 are preferable. As a result, densityratio correction will be helpful if the test set is drawn fromwα1pxq while the training set is not, provided that we canestimate wα1pxq accurately.
How do we know if such a dominant strategy exists for agame between Θ and A? The robust solution of the gamecan help us in finding out. Let sθ be the solution of theunweighed loss minimization problem
sθ “ argminθPΘ
ż
lpfθpxq, yq dF px, yq, (13)
and pθ be the solution of the reweighed adversarial loss min-imization problem
pθ “ argminθPΘ
maxαPA8
ż
wαpxqlpfθpxq, yq dF px, yq. (14)
Our main observation is that, if a dominant strategy θ: ex-ists, then under suitable assumptions on the adversary, theunweighed solution sθ is also a dominant strategy.Theorem 4. Suppose the reweighing functionwαpxq is lin-ear inα, and the constant reweighingα0 withwα0pxq “ 1is in the relative interior of A8. If a dominant strategy θ:
of the learner exists, then the unweighed solution sθ is alsoa dominant strategy for the learner.
Thm. 4 suggests a way to test for the existence of dominantstrategy. Consider the adversarial loss
LA8pθq “ maxαPA8
ż
wαpxqlpfθpxq, yq dF px, yq.
By definition, any dominant strategy θ: minimizes the ad-versarial loss, so Thm. 4 implies that the unweighed solu-tion sθ will also minimize the adversarial loss. Therefore,by comparing the value of the minimax solution LA8ppθq(which by definition minimizes the adversarial loss) againstLA8psθq, we can tell if a dominant strategy exists. If theyare not equal, then we are certain that no such dominantstrategy exists, and density ratio correction can be helpful,depending on the choice of training and test distributionsfrom A. On the other hand, if they are equal, we cannotconclude that a dominant strategy exists, as it is possiblethat the reweighed adversarial distribution matches the uni-form unweighed distribution arbitrarily closely. However,such examples are rather contrived and we never encoun-tered such a situation in any of our experiments.
The final question left is how to compare LA8ppθq andLA8psθq in practice with a finite sample. Let pθn be a solu-tion of the robust game:
pθn “ argminθPΘ
maxαPAn
1
n
nÿ
i“1
wαpxiqlpfθpxiq, yiq, (15)
and let sθn be a solution of the unweighed ERM problem:
sθn “ argminθPΘ
1
n
nÿ
i“1
lpfθpxiq, yiq. (16)
Our convergence results below states that, instead ofLA8ppθq and LA8psθq, we can compare the empirical ad-versarial losses LAnppθnq and LAnpsθnq.Theorem 5. Suppose the support AS forα and Θ for θ areeach closed, convex, and bounded. Suppose also wαpxqand lpfθpxq, yq are bounded continuous functions inα andθ for each px, yq pair. If the set satisfying the normalizationconstraint tα P AS |
ş
wαpxqdF px, yq “ 1u is non-emptyin the relative interior of AS , then we have, for all θ P Θ,
LAnpθq Ñ LA8pθq
in probability, i.e., for all ε, δ ą 0, we can find m P N suchthat for all n ě m, we have
|LAnpθq ´ LA8pθq| ă ε
Robust Learning under Uncertain Test Distributions
with probability at least 1´ δ.
For simplicity, we do not consider the moment matchingconstraints Knα “ sφte, but this can be handled in theproof with techniques similar to the normalization con-straint. We use t-test with cross-validation to comparethese quantities in the experiments.
The chain of implications can be summarized as follows:
No model misspecification for Θ
ñPointwise dominator exists for Θ
ñDominant strategy against any bounded adversary A exists
We can see that “no model misspecification” is a verystrong condition, as it requires a dominator against anybounded adversary A, including pathologically spiky testdistributions with tall spikes and small support. Also, thereis an implicit assumption in using density ratio correctionthat covariate shifts on the test set are not represented by ar-bitrarily complex functions. Otherwise estimation of den-sity ratio cannot take place and there is no way to correctcovariate shift. Therefore, instead of focusing on densityratio correction alone, we look at another way of dealingwith covariate shift, by performing model checking againsta set of restricted changes in test distributions representedby the adversary A. In the next section we will provideempirical evaluations of this particular approach.
Our analysis is different from Shimodaira (2000). We donot start with a strong condition like “no model misspec-ification”, which is equivalent to requiring the learner tohave a dominant strategy against any adversaries. Instead,given weak prior knowledge about how the distributionscan shift, we provide a method that can determine whetherreweighing can be helpful. This analysis is more practicalthan deciding whether reweighing is needed based on thestrong notion of no model misspecification, as almost allmodels are misspecified on real datasets.
5. Empirical Studies5.1. Experiments on Toy Datasets
We first present two toy examples to show the effectivenessof our test. We construct a linear model, f1pxq “ x`1`ε,and a non-linear (cubic) model, f2pxq “ x3 ´ x ` 1 ` ε,where ε „ N p0, 0.12q is additive Gaussian noise (adaptedfrom Shimodaira (2000)). For both, we learn a linear re-gressor fθpxq “ θ1 ¨ x` θ0 from data to minimize squaredloss lpfθpxiq, yiq “ ||θT xi ´ yi||
2 with L2 regularizer onθ: Ωpθq “ 1
2 ||θ||22.
To show how to detect whether a dominant strategy ex-ists with various adversarial sets A, we generate 500 datapoints uniformly in the interval r´1.5, 2s, which we par-
tition into training and test sets via 10-fold cross valida-tion. To construct reasonable adversaries, we use Eq.(4)with Gaussian kernel as our reweighing function. As wementioned earlier, the adversarial set is determined by priorknowledge of how the test distribution might change. Inthis toy example, we use a large range of σ, based on theaverage distance from an instance to its n
c -nearest neigh-bours, where n is the number of training points and c Pt2, 4, 8, 16, ¨ ¨ ¨ u. The smaller σ is, the more powerful theadversary can be, i.e., the more possible test distributions itcan generate. The bases, bj , are chosen to be the trainingpoints. B is set to be 5, a bound that is rarely reached inpractice due to the normalization constraint. Therefore, thisbound does not significantly limit the adversary’s power,as it allows the adversary to put as much importance ona single kernel as it wants. We tune the parameter λ via10-fold cross validation.2 Figure 2(a) shows that LAnppθnqand LAnpsθnq (mean and one standard deviation as errorbar) are very close for all σ in the linear example, indicat-ing that the adversary cannot exploit the weakness of linearlearner. Figure 2(b) shows that, for the non-linear example,even with moderate σ, there is a noticeable difference be-tween LAnppθnq against LAnpsθnq, strongly suggesting thatno dominant strategy exists in this case, which suggests thatcovariate shift correction is necessary if the test distributionis shifted here. The experiments showing covariate shiftscenarios are in Appendix B due to space limits.
To see how the adversary creates different empirical adver-sarial losses in a non-linear example, we fix the σ to theaverage distance from an instance to its n
5 -nearest neigh-bour and illustrate a concrete trial in Figure 2(c). It is obvi-ous that the adversary puts more weights at the test pointswhere the loss of the classifier learned from training datais large. Our robust formulation incorporates the adversaryand prevents any point from having too large a loss. Asa result, the adversary cannot undermine the robust learnerseverely, which leads to the gap of the empirical adversariallosses of robust and regular learners in Figure 2(b).
5.2. Experiments on Real-world Datasets
This section presents the experimental results on real worlddatasets to show how our formulation determines whetherdominant strategy exists against some adversaries and if so,how to correct such covariate shifts. We investigate bothregression problems using squared loss, and classificationproblems using hinge loss. A linear model is learned fromthe dataset unless otherwise specified.
We obtain some classification datasets from UCI reposi-
2Here, as there is no covariate shift, we just use simple crossvalidation. Whenever test distribution is shifted in the experi-ment, parameters are tuned via importance weighted cross vali-dation (Sugiyama et al., 2007).
Robust Learning under Uncertain Test Distributions
1.08 0.48 0.22 0.11 0.05 0.020.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
σ
Advers
arial Loss
Regular
Robust
(a) Linear example.
1.06 0.49 0.23 0.11 0.05 0.020
2
4
6
8
10
12
14
σ
Ad
ve
rsa
ria
l L
oss
Regular
Robust
(b) Cubic example.
−1.5 −1 −0.5 0 0.5 1 1.5 2−1
0
1
2
3
4
5
6
7
8
9
Ou
tpu
t
Input
−1.5 −1 −0.5 0 0.5 1 1.5 20
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
−1.5 −1 −0.5 0 0.5 1 1.5 20
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
We
igh
t
Training PointTest PointRegular RegressorRobust RegressorTest Reweigh for RegularTest Reweigh for Robust
(c) Adversarial reweighing.
Figure 2. Toy examples. Adversarial test losses are shown in Figures 2(a) and 2(b), where the x-axis shows the values of σ. Figure 2(c)provides a non-linear example to show how the adversary attacks the regressors by reweighing the test points, with output on the lefty-axis and weight on the right y-axis.
Table 1. Dataset SummaryDATASET SIZE DIM TYPE
AUSTRALIAN 690 14 CLASSIFICATIONBREAST CANCER 683 10 CLASSIFICATIONGERMAN NUMER 1000 24 CLASSIFICATION
HEART 270 13 CLASSIFICATIONIONOSPHERE 351 34 CLASSIFICATION
LIVER DISORDER 345 6 CLASSIFICATIONSONAR 208 60 CLASSIFICATIONSPLICE 1000 60 CLASSIFICATION
AUTO-MPG 392 6 REGRESSIONCANCER 1523 40 REGRESSION
tory3. All are binary classification problems. For regres-sion task, we use Auto-mpg dataset, which admits a nat-ural covariate shift scenario, as it contains data collectedfrom 3 different cities. We also have a set of cancer patientsurvival time data provided by our medical collaborators,containing 1523 uncensored patients with 40 features, in-cluding gender, stage of cancer, and various measurementsobtained at the time of diagnosis. Table 1 shows the sum-mary of the datasets we used in the experiments.
5.2.1. DOMINANT STRATEGY DETECTION
We first detect the existence of dominant strategy as we didin the toy example. To construct reasonable adversaries,Gaussian kernel is applied to Eq.(4), setting σ to be the av-erage distance from an instance to its n
5 -nearest neighbour,the bases bj to be the training points and B to be 5. 4
Figure 3(a) shows the experimental results. Auto-mpg12
3http://archive.ics.uci.edu/ml/index.html4We allow the user to set up other adversaries, as the appro-
priate adversary depends on user’s belief about how the test dis-tribution may change. We use this medium power adversary todifferentiate between datasets under linear models. If the adver-sary is too weak, no correction is needed for all datasets. If theadversary is too strong, all datasets require correction, as on realdata there is no “correct” model. We omit these less interestingcases due to space limits.
explores when the training data comes from city 1 andtest data is from city 2, while Auto-mpg13 exploreswhen training data comes from city 1 and test data comesfrom city 3. Here we focus on the empirical adversariallosses of robust versus regular models. A significant dif-ference indicates that there is no dominant strategy andthus, the linear model is vulnerable to our reweighing ad-versary. For classification datasets and the cancer dataset,we apply 10-fold cross validation to obtain training andtest sets. For Auto-mpg, we fix the test set and apply10-fold cross validation to obtain training set. Figure 3(a)presents these losses over the datasets (mean and one stan-dard deviation as error bar). t-test at significance level 0.05indicates that two losses are significantly different for theLiver disorders and Auto-mpg datasets. As a re-sult, the linear model is vulnerable for these sets.
To further substantiate the incapability of the linear model,we attempted to detect dominant strategy for a Gaussianmodel set Θ (i.e., changing from linear kernel to Gaussiankernel where the learner use internal cross-validation tochose the kernel width). Results are shown in Figure 3(b).Compared to Figure 3(a), the gap of empirical adversariallosses between robust and regular models shrinks signif-icantly in Figure 3(b). Our result indicates that t-test nolonger claims a significant difference between these losses,suggesting that the adversary cannot severely underminethe performance of regular learning. Therefore, model re-vision can be a good alternative to performing covariateshift correction.
5.2.2. REWEIGHING ALGORITHM FOR COVARIATESHIFT SCENARIOS
As previously mentioned, the reweighing mechanism couldimprove the performance if the model is vulnerable to thereweighing adversary. For the covariate shift correctiontask, we set the test points as the reference bases bj of theweight function (Eq. (4)), because they are more informa-
Robust Learning under Uncertain Test Distributions
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Advers
arial Loss
Australian
Breast_cancer
Germ
an_numer
Heart
Ionosphere
Liver_disorders
Sonar
Splice
Auto−mpg12
Auto−mpg13
Cancer
Robust
Regular
(a) Linear learner.
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Advers
arial Loss
Australian
Breast_cancer
Germ
an_numer
Heart
Ionosphere
Liver_disorders
Sonar
Splice
Auto−mpg12
Auto−mpg13
Cancer
Robust
Regular
(b) Gaussian learner.
0
0.2
0.4
0.6
0.8
1
1.2
Test Loss
Australian
Breast_cancer
Germ
an_numer
Heart
Ionosphere
Liver_disorders
Sonar
Splice
Auto−mpg12
Auto−mpg13
Cancer−gender
Cancer−stage
UnweighedClustKLIEPRuLSIFRCSA
(c) Reweighed linear model performance.
Figure 3. Experimental results for dominant strategy detection and covariate shift correction. Figure 3(a) and Figure 3(b) show theadversarial test losses of robust and regular learners.
tive than training points about the test distribution, as sug-gested by Sugiyama et al. (2008). The reweighing set (σand B) is chosen as in Section 5.2.1.
To create covariate shift scenarios in the classificationdatasets, we apply the following scheme to obtain shiftedtest set. We first randomly pick 75% of the whole set forrobust training (6) to attain a model θ. Then we evaluatethe empirical adversarial loss (8) of this model on the 25%hold-out test set, and record all the weights of these testpoints. The probability that a test instance x will remain inthe test set is min
´
1, wαpxq1m
¯
, where m is the number oftest points at the moment (25% of the set). About 10% ofthe whole dataset remain after filtering; these instances willserve as the final test set with shifted distribution. The intu-ition is: we want some test points with large errors even forrobust learner. These points are more likely to undermineany model, meaning covariate shift will have more signif-icant impact on the performance. The procedure is per-formed 10 times, leading to the average test losses reportedin Figure 3(c). As Auto-mpg has a natural covariate shiftscenario, we do not artificially partition the dataset. Weapplied 10-fold cross validation to obtain the training set.We consider two shifted scenarios in cancer survival timeprediction:
1. Gender split. The dataset contains about 60% maleand 40% female patients. In gender split, we randomly take20% of the male and 80% of the female patients into train-ing set, while the rest goes to test set. That is, the trainingset is dominated by male patients while the test set is dom-inated by female patients.
2. Cancer stage split. Approximately 70% of the datasetare of stage-4. In cancer stage split, we randomly take 20%of stage-1-to-3 and 80% of stage-4 patients to training set,while the rest goes to test set. The training set is dominatedby stage-4 patients while the test set is dominated by stage-1-to-3 patients.
Figure 3(c) compares the test losses of RCSA with
the regular unweighed learning algorithm, the clustering-based reweighing algorithm (Cortes et al., 2008),KLIEP (Sugiyama et al., 2008) and RuLSIF (Yamada et al.,2011). Recall that our analysis in Section 5.2.1 shows thatlinear model is insufficient for the Liver disordersand Auto-mpg datasets, which suggests that reweighingmay help. This is confirmed in Figure 3(c): by puttingmore weights on the training instances that are similar totest instances, the reweighing algorithms can produce mod-els with smaller test losses for these datasets. Although ourrobust game formulation is mainly designed to detect dom-inant strategy, our RCSA algorithm can correct shifted dis-tribution using moment matching constraints described inSection 3.2. As shown in Figure 3(c), our method performson par with state-of-the-art algorithms when covariate shiftcorrection is required. For the datasets that appear linear(i.e., where the linear model performs relatively well), wefound that the reweighing algorithms did not significantlyreduce the test losses. In some cases, reweighing actuallyincreased the test losses due to the presence of noise.
6. ConclusionsWe have provided a method for determining if covariateshift correction is needed, given a pre-defined set of po-tential changes in the test distribution. This is useful forensuring the learned predictor will still perform well whenthere are uncertainties about the test distribution in the de-ployment environment, such as changes in gender ratio andcase mix in the cancer prognostic predictor example. It canalso be used to decide if a model class revision of Θ is nec-essary. Experimental results show that our detection testis effective on several UCI datasets and a real-world can-cer patient dataset. This analysis shows the importance ofstudying the interaction of covariate shift and model mis-specification, because the final test set error depends onboth factors.
Robust Learning under Uncertain Test Distributions
ReferencesAltun, Y. and Smola, A. Unifying divergence minimization
and statistical inference via convex duality. In Learningtheory, pp. 139–153. Springer, 2006.
Ben-David, S., Blitzer, J., Crammer, K., and Pereira, F.Analysis of representations for domain adaptation. InAdvances in Neural Information Processing Systems,volume 19, pp. 137, 2007.
Berger, A., Pietra, V., and Pietra, S. A maximum entropyapproach to natural language processing. Computationallinguistics, 22(1):39–71, 1996.
Bickel, S., Bruckner, M., and Scheffer, T. Discriminativelearning for differing training and test distributions. InInternational Conference on Machine Learning, pp. 81–88, 2007.
Bickel, S., Bruckner, M., and Scheffer, T. Discriminativelearning under covariate shift. The Journal of MachineLearning Research, 10:2137–2155, 2009.
Cortes, C., Mohri, M., Riley, M., and Rostamizadeh, A.Sample selection bias correction theory. AlgorithmicLearning Theory, 5254:38–53, 2008.
Cortes, C., Mansour, Y., and Mohri, M. Learning boundsfor importance weighting. Advances in Neural Informa-tion Processing Systems, 23:442–450, 2010.
Globerson, Amir and Roweis, Sam. Nightmare at test time:robust learning by feature deletion. In International Con-ference on Machine learning, pp. 353–360. ACM, 2006.
Grunwald, P. D. and Dawid, A. P. Game theory, maximumentropy, minimum discrepancy and robust bayesian de-cision theory. The Annals of Statistics, 32(4):1367–1433,2004.
Huang, J., Smola, A. J., Gretton, A., Borgwardt, K. M.,and Scholkopf, B. Correcting sample selection bias byunlabeled data. In Advances in Neural Information Pro-cessing Systems, volume 19, pp. 601, 2007.
Kanamori, T., Hido, S., and Sugiyama, M. Efficient directdensity ratio estimation for non-stationarity adaptationand outlier detection. In Advances in Neural InformationProcessing Systems, 2008.
Kiwiel, K. C. Proximity control in bundle methods for con-vex nondifferentiable minimization. Mathematical Pro-gramming, 46(1):105–122, 1990.
Pan, S. J., Tsang, I. W., Kwok, J. T., and Yang, Q. Domainadaptation via transfer component analysis. In Inter-national Joint Conference on Artifical Intelligence, pp.1187–1192, 2009.
Quionero-Candela, J., Sugiyama, M., Schwaighofer, A.,and Lawrence, N. D. Dataset shift in machine learning.The MIT Press, 2009.
Rockafellar, R. T. Convex analysis. Princeton UniversityPress, 1996.
Scholkopf, B. and Smola, A. J. Learning with kernels: sup-port vector machines, regularization, optimization andbeyond. The MIT Press, 2002.
Shimodaira, H. Improving predictive inference under co-variate shift by weighting the log-likelihood function.Journal of Statistical Planning and Inference, 90(2):227–244, 2000.
Storkey, A. J. and Sugiyama, M. Mixture regression forcovariate shift. In Advances in Neural Information Pro-cessing Systems, 2007.
Sugiyama, M. and Kawanabe, M. Machine learning in non-stationary environments: introduction to covariate shiftadaptation. The MIT Press, 2012.
Sugiyama, M. and Muller, K. Model selection under co-variate shift. In Artificial Neural Networks: FormalModels and Their Applications, pp. 235–240. Springer,2005.
Sugiyama, M., Krauledat, M., and Muller, K. Covariateshift adaptation by importance weighted cross valida-tion. Journal of Maching Learning Research, 8:985–1005, 2007.
Sugiyama, M., Nakajima, S., Kashima, H., Von Buenau,P., and Kawanabe, M. Direct importance estimationwith model selection and its application to covariate shiftadaptation. In Advances in Neural Information Process-ing Systems, 2008.
Teo, C. H., Globerson, A., Roweis, S., and Smola, A. Con-vex learning with invariances. In Advances in NeuralInformation Processing Systems, 2008.
White, H. Consequences and detection of misspecifiednonlinear regression models. Journal of the AmericanStatistical Association, 76(374):419–433, 1981.
Yamada, M., Suzuki, T., Kanamori, T., Hachiya, H., andSugiyama, M. Relative density-ratio estimation for ro-bust distribution comparison. In Advances in Neural In-formation Processing Systems, pp. 594–602, 2011.
Zadrozny, B. Learning and evaluating classifiers undersample selection bias. In International Conference onMachine Learning, pp. 114. ACM, 2004.
Robust Learning under Uncertain Test Distributions
Appendix AProof of Theorem 3
Proof. By the definition of a pointwise dominatorż
lpfθ‹pxq, yqdF py | xq ´
ż
lpfθ1pxq, yqdF py | xq ď 0
for all θ1 P Θ. Given any bounded adversarial set A, forany α P A, wαpxq is a non-negative function of x. There-fore integrating with respect to dF pxq givesż
wαpxq
„ż
lpfθ‹pxq, yqdF py |xq´
ż
lpfθ1pxq, yqdF py |xq
dF pxqď0
ż
wαpxqlpfθ‹pxq, yqdF px, yqď
ż
wαpxqlpfθ1pxq, yqdF px, yq.
Thus θ‹ is also a dominant strategy against the adversarialset A.
Proof of Theorem 4
Proof. We use hpθq from Eq. (19) to denote the cost vec-tor for expected adversarial loss, with the extra argumentθ to emphasize its dependence on θ. As θ: is a dominantstrategy, we have
hpθ:qTα ď hpsθqTαñ phpθ:q ´ hpsθqqTα ď 0 (17)
for all α P A8. By definition, sθ minimizes the adversarialloss for the constant unweighed strategy α0 of the adver-sary, so we have
phpθ:q ´ hpsθqqTα0 “ 0. (18)
Let α1 P A8. As α0 is in the relative interior of A8 andA8 is convex, there exists ε ą 0 such that
α2 “ α1 ` p1` εqpα0 ´α1q
is in A8. Now by Eq. (17) and (18), we have three colinearpoints such that
phpθ:q ´ hpsθqqTα1 ď 0
phpθ:q ´ hpsθqqTα0 “ 0
phpθ:q ´ hpsθqqTα2 ď 0.
So phpθ:q´hpsθqqTαmust be identically 0 on the intervalrα1,α2s, as it is a linear function in α.
This shows hpsθqTα1 “ hpθ:qTα1. As α1 is arbitrary, theunweighed solution sθ is also a dominant strategy for thelearner Θ.
Proof of Theorem 5
Notice the reweighed loss is linear in α for fixed θ:
1
n
nÿ
i“1
wαpxiqlpfθpxiq, yiq “kÿ
j“1
αj1
n
nÿ
i“1
kjpxiqlpfθpxiq, yiq
“ hTnα,
where
phnqj “1
n
nÿ
i“1
kjpxiqlpfθpxiq, yiq.
Therefore we can write LAnpθq as:
LAnpθq “ maxαPAn
hTnα
Similarly, define the corresponding cost vector h for theexpected adversarial loss such that
phqj “
ż
kjpxq lθpfpxq, yqdF px, yq, (19)
and we haveLA8pθq “ max
αPA8hTα
Similarly, define for the normalization constraint:
1
n
nÿ
i“1
wαpxiq “kÿ
j“1
αj1
n
nÿ
i“1
kjpxiq “ gTnα,
where
pgnqj “1
n
nÿ
i“1
kjpxiq.
Define the corresponding constraint vector g such that
pgqj “
ż
kjpxq dF px, yq.
Translating into the new notations, we want to proveˇ
ˇ
ˇ
ˇ
maxαPAS :gTα“1
hTα´ maxαPAS :gTnα“1
hTnα
ˇ
ˇ
ˇ
ˇ
ă ε
with probability at least 1´ δ, for all sufficiently large n.
To prove the result we need two lemmas, whose proofs ap-pear after the main proof. The first lemma states that thesample cost vector hn converges in the infinite limit to h.The second lemma states that near the feasible solutionsof gTα “ 1, there are feasible solutions of finite sampleconstraint gTnα “ 1 for large n, and also vice versa.Lemma 6. Assume the basis kjpxq for the reweighing func-tion wpxq are bounded above by Bk, and the loss functionl bounded above by Bl. We then have
Prphn ´ h2 ě εq ď 2k exp
ˆ
´2nε2
B2kB
2l k
2
˙
.
Lemma 7. Suppose ε, δ ą 0 are given and let α˚ P A8.Then there exists m P N such that, for all n ě m, withprobability at least 1´ δ, we can find αn P An such that
α˚ ´αn ď ε
Similarly, suppose ε, δ ą 0 are given. Then there existsm P N such that for all n ě m, for any αn P An, withprobability at least 1´ δ, we can find α˚ P A8 such that
αn ´α˚ ď ε.
Robust Learning under Uncertain Test Distributions
PROOF OF MAIN THEOREM
By Lemma 6, there exists n1 P N such that for all n ě n1,h´hn ď εp2Bαq with probability 1´δ3. [condition 1]
Let hTα˚ “ maxαPA8 hTα. By Lemma 7, there exists
n2 P N such that for all n ě n2, we can find α1n P An withα˚´α1n ď εp2hq with probability 1´δ3 [condition2]. Conditions 1 and 2 give
maxαPA8
hTα´ maxαPAn
hTnα
ď hTα˚ ´ hTnα1n
“ hTα˚ ´ hTα1n ` hTα1n ´ h
Tnα
1n
“ hT pα˚ ´α1nq ` ph´ hnqTα1n
ď hα˚ ´α1n ` h´ hnα1n
ď hε
2h`
ε
2BαBα “ ε
Similarly, let hTnα˚n “ maxαPAn h
Tnα. By Lemma 7,
there exists n3 P N such that for each n ě n3, we canfind α1 P A8 with α˚n ´ α
1 ď ε2h with probability1´ δ3 [condition 3]. Conditions 1 and 3 give
maxαPAn
hTnα´ maxαPA8
hTα
ď hTnα˚n ´ h
Tα1
ď hTnα˚n ´ h
Tα˚n ` hTα˚n ´ h
Tα1
“ phn ´ hqTα˚n ` h
T pα˚n ´α1q
ď hn ´ hα˚n ` hα
˚n ´α
1
ďε
2BαBα ` h
ε
2h“ ε
Therefore when n ě maxtn1, n2, n3u, with probability atleast 1´ δ (by union bound), we have
| maxαPAn
hTnα´ maxαPA8
hTα| ď ε ˝
PROOF OF LEMMA 6
By Hoeffding’s inequality, we have
Prp|phnqj ´ phqj | ąε
kq ď 2 exp
ˆ
´2nε2
B2kB
2l k
2
˙
.
By union bound, we have
Prphn ´ h1 ě εq ď 2k exp
ˆ
´2nε2
B2kB
2l k
2
˙
.
As hn ´ h2 ď hn ´ h1, we have
Prphn ´ h2 ě εq ď 2k exp
ˆ
´2nε2
B2kB
2l k
2
˙
. ˝
E! : gT!=1+"
E0 : gT!=1
f(!)
g(!)
A
Figure 4. Definition of fpηq and gpηq
PROOF OF LEMMA 7
Using Hoeffding’s inequality and union bound (similar tothe proof of Lemma 6), we have
Prpgn ´ g2 ě εq ď 2k exp
ˆ
´2nε2
B2kk
2
˙
. (20)
DefineEη “ tα P AS |gTα “ 1` ηu
for each η P R. This is the set of subspaces parallel togTα “ 1 (E0). Define also fpηq as the maximum dis-tance of any points in Eη to E0, and gpηq as the maximumdistance of any points in E0 to Eη (see Fig. 4), i.e.,
fpηq “ maxαPEη
minα1PE0
α´α1,
gpηq “ maxαPE0
minα1PEη
α´α1.
Suppose ε, δ ą 0 are given. Using Lemma 8 below,fpηq Ñ 0 as η Ñ 0, so we can find η0 ą 0 such thatfpηq ă ε whenever |η| ă η0. From Eq. (20), we can findm P N such that for all n ě m, gn ´ g ă η0Bα withprobability at least 1´ δ.
Letting αn P An for n ě m, we have
|gTαn´1|“|gTαn´gTnαn|ďg´gnαnď
η0
BαBα“η0
(21)with probability at least 1´ δ. Hence the subspace gTnα “1, i.e. An, lies between Eη0 and E´η0 with probability1 ´ δ. Specifically for a fixed αn P An, it lies on Eη forsome η with |η| ă η0. Therefore
minα1PE0
αn ´α1 ď max
αPEηminα1PE0
α´α1 “ fpηq ď ε
with probability 1´ δ.
For the second part, let α˚ P A8p“ E0q, and ε, δ ą 0 begiven. Using Lemma 8 below, gpηq Ñ 0 as η Ñ 0, so wecan find η0 ą 0 such that gpηq ă ε whenever |η| ă η0.
Robust Learning under Uncertain Test Distributions
E0
E!0
gTn! = 1
< !
!!
!n
Figure 5. Illustration for proof of Lemma 7
By Eq. (21) above, we can find m P N such that An liesentirely between E´η0 and Eη0 with probability at least1´ δ. By definition
minα1PEη0
α˚ ´α1 ď gpη0q ď ε.
Let αη0 be a point on Eη0 minimizing the distance to α˚,then the line joining αη0 and α˚ has to intersect with thesubspace gTnα “ 1 at some αn (see Fig. 5). This holdsfor all n ě m and we have αn ´ α˚ ď ε. The sameargument applies to the case when gTnα “ 1 lies betweenE0 and E´η0 . Thus
minα1PAn
α1 ´α˚ ď ε
for all n ě m, with probability at least 1´ δ.Lemma 8.
fpηq “ maxαPEη
minα1PE0
α´α1
gpηq “ maxαPE0
minα1PEη
α´α1
converge to 0 as η Ñ 0.
Proof. We want to show fpηq Ñ 0 as η Ñ 0. If not, thenthere exists f0 ą 0 and a sequence tηtu8t“1 with ηt Ñ 0,such that fpηtq ě f0 infinitely often. We collect all thoseindices tn such that fpηtnq ě f0, and form a new sequenceµn “ ηtn . Let
αn “ argmaxαPEµn
minα1PE0
α´α1.
As αn lies in a compact set AS , there exist a convergentsubsequence, say βn. Let the subsequence βn converge tosome β, and by continuity we know gTβ “ 1, so β P E0.
The function
spαq “ minα1PE0
α´α1
is a continuous function in α (minimum of a bivariate con-tinuous function over a compact set).
A
E1
E0
E!
d1d2
!j
d1 = min!!!E1
!!j " !"!
d2 ! min!!!E!
"!j # !""d2 = !d1
Figure 6. Illustration for the proof of Lemma 8
We have spβnq ě f0 and βn Ñ β, so spβnq convergesto some f 10 ě f0 as s is continuous. However, since β PE0, we have spβq “ 0. This creates a contradiction andtherefore fpηq Ñ 0.
Next we want to show gpηq Ñ 0 as η Ñ 0. Given γ ą 0,as E0 is compact, we can cover E0 with at most k balls ofradius γ2 for some finite k. We label the centres of theseballs as αj , 1 ď j ď k.
We consider the case where η ą 0. The case for η ă 0 issymmetric. By the assumption of the theorem the set tα PAS | gTα “ 1u is non-empty in the relative interior of AS .So there exists η ą 0 such that Eη is non-empty. Withoutloss of generality, assume E1 non-empty (can rescale withany positive constant other than 1), define
dj “ minα1PE1
αj ´α1.
By convexity (see Fig. 6), for 0 ă η ď 1,
minα1PEη
αj ´α1 ď η min
α1PE1
αj ´α1 “ ηdj
For any α P E0, it lies within one of the k balls, say αj .We have
minα1PEη
α´α1 ď minα1PEη
rα´αj ` αj ´α1s
“ α´αj ` minα1PEη
αj ´α1
ďγ
2` ηdj
Since the k balls altogether cover E0, for all α P E0, whenη ď γ
2 max1ďjďk dj,
minα1PEη
α´α1 ďγ
2` η max
1ďjďkdj ď
γ
2`γ
2“ γ
HencemaxαPE0
minα1PEη
α´α1 ď γ
whenever η ď minp1, γp2 max1ďjďk djqq. The argumentfor η ă 0 is symmetric. Therefore gpηq Ñ 0 as η Ñ 0.
Robust Learning under Uncertain Test Distributions
Unweighed Clust KLIEP RuLSIF RCSA0
0.002
0.004
0.006
0.008
0.01
0.012
0.014Test losses of different algorithms for linear example
(a) Linear example.Unweighed Clust KLIEP RuLSIF RCSA
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35Test losses of different algorithms for cubic example
(b) Cubic example.
Figure 7. Test losses of different reweighing algorithms for linear and cubic models.
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5−1
0
1
2
3
4
Ou
tpu
t
Input
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.50
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
We
igh
t
Training PointTest PointTrue ModelUnweighed RegressorRCSAReweighing Function
(a) Linear example.
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5−1
0
1
2
3
4
5
6
7
8
9
Ou
tpu
t
Input
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.50
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
We
igh
t
Training PointTest PointTrue ModelUnweighed RegressorRCSAReweighing Function
(b) Cubic example.
Figure 8. Reweighing for different models. The model class is correctly specified in (a), while is noticeably misspecified in (b).
Appendix BThis appendix shows the experimental results of covariateshift scenario for the synthetic data in Section 5.1. Fol-lowing the setting in Section 5.1, we generate 100 train-ing points where xtr „ N p0.5, 0.52q and 100 test pointswhere xte „ N p0, 0.32q. This scenario is adapted fromShimodaira (2000). The performance of both linear modeland cubic model are investigated for this covariate shiftscenario. Figure 7 reports the test losses (mean and onestandard deviation as errorbar) of different reweighing al-gorithms over 10 trials. Obviously, reweighing is relativelyeffective when there is no dominant strategy in the under-lining model class (Figure 7(b)), compared to the relativelywell specified case (Figure 7(a)), where reweighing doesnot reduce the test loss significantly. To see how reweigh-ing behaves for different model classes, Figure 8 provideone trial of the experiment. Although in both cases, thelearning procedure focus on a subset of training points,reweighing is more influential in the cubic example. As themodel class is well specified in Figure 8(a), learning froma subset of training points can still recover the global struc-
ture of the true model. However, in Figure 8(b), focusingon a subset of training points is more likely to recover localstructure of the true model in the test region, and thus thereweighed model performs better in the test region com-pared to the unweighed model.
