Dependent Random Variables
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Relative Change
• Influence of variables
• Influence is based on the change in value
• We need to determine the reference value to assess
whether a feature matters.
• Data type dependent, but means are a good way to start (images, text, tabular data, audio, etc.)
Δx
x0
i
f(x) = w⊤x + b =
n
∑i=1
wixi + bx0 Δx
Δx
x0
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Distributions, joint, marginal, expectations
• Want to know the influence of on
• Direct influence via function (that’s what we’re after)
• Confounded by indirect influence on other features
(from on )
• Options
• Use conditional expectation
• Use marginal
xi f(x)
xi x−i
p(x−i |xi)
p(x−i)
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Backdoor Problem
• are conditionally indep.
• Observed distribution
• Conditional is dependent
x1, x2
p(x1, x2) ≠ p(x1)p(x2)
x1 x2
zLatent
Observed
p(x1, x2) = ∫ dp(z)p(x1 |z)p(x2 |z)
trustworthy?
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
• Credit rating based on observed attributes
• Changing observed attributes will not change underlying condition.
Approve Credit?
x1 x2
zLatent
Observed
Fix thisDraw
independently
f (x)
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Causal Connection
x1 x2
f (x)
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
More Context
• Pearl’s do operator
• Use samples from marginals rather than conditional distribution. This is easier and usually more correct.
E[y |do(X1 = x1)] = ∫ dp(x2, x3)E[y |x1, x2, x3]
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
The Parliament of Micronesia
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
The Parliament of Micronesia
The parliament of Micronesia is made up of four political parties, A, B, C, and D, which have 45, 25, 15, and 15 representatives, respectively. They are to vote on whether to pass a $1M spending bill and how much of this amount should be controlled by each of the parties. A majority vote, that is, a minimum of 51 votes, is required in order to pass any legislation, and if the bill does not pass then every party gets zero to spend.
Winning coalitions are (A,B), (A,C), (A, D), (B, C, D) and any
superset of these. Payoff function for set .v(S) S
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Why do we care?
• Replace parties with features • Replace payoff with influence
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Axioms
• Symmetry
For any payoff function not containing or with
we need the reward to satisfy .
• Dummy Player
For any without with we
need the reward to satisfy .
v(S) i j
ϕ(i) = ϕ( j)
S i v(S ∪ {i}) = v(S) + v({i})
ϕ(i) = v({i})
v(S ∪ {i}) = v(S ∪ {j}) for all S
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Axioms
• Additivity
For any payoff function the rewards are also
additive, i.e.
• Shapley Value Theorem The payoff function satisfying symmetry, dummy and additivity that divides the payoff the grand coalition (everyone contributes) is uniquely defined by the Shapley value of the game.
v = v1 + v2
ϕ(i) = ϕ1(i) + ϕ2(i)
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Shapley Value Function
• Definition
• Properties
• Average over all set sizes and over all ways to
choose . Normalized by cardinality.
• Add up all incremental contributions.
• Easy to check that it satisfies the axioms. Uniqueness is a bit more work.
|N |
S ∈ N \{i}
ϕ(i, N ) = ∑S∈N \{i}
1
|N | (|N | − 1
|S | )−1
[v(S ∪ {i}) − v(S)]
Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p
Back to Micronesia
• A (45), B (25), C (15), or D (15) have individual payoff 0.
• Any coalition including A succeeds (B, C, D are interchangeable)
• The only successful coalition without A is (B, C, D).
• Going over the definition yields payoffs
A =1
2 and B = C = D =
1
6
Back to Feature Scoring - use this for f(x)