Embed Size (px)
Citation preview
/
CSE 559A: Computer Vision
Fall 2020: T-R: 11:30-12:50pm @ Wri”hton 300 / Zoom
Instru‘tor: Ayan Chakrabarti ([email protected]’u).Course Staff: A’ith Boloor, Patri‘k Williams
O‘t 15, 2020
http://www.‘se.wustl.e’u/~ayan/‘ourses/‘se559a/
1
/
GENERALGENERALPSET 1 Solutions will be poste’ a er Fri’ay.PSET 2 Out!There will be a re‘itation “or PSET 2 next Fri’ay (O‘tober 23r’).
2
/
NORMALSNORMALS
3
/
NORMALSNORMALS
4
/
NORMALSNORMALS
5
/
NORMALSNORMALS
6
/
NORMALSNORMALS
7
/
NORMALSNORMALS
8
/
ANGLESANGLES
9
/
ANGLESANGLES
10
/
ANGLESANGLES
11
/
ANGLESANGLES
12
/
ANGLESANGLES
13
/
ANGLESANGLES
14
/
ANGLESANGLES
15
/
ANGLESANGLES
16
/
ANGLESANGLES
17
/
ANGLESANGLES
18
/
ANGLESANGLES
19
/
RADIANCERADIANCE
20
/
RADIANCERADIANCE
21
/
RADIANCERADIANCE
22
/
RADIANCERADIANCE
23
/
RADIANCERADIANCE
24
/
RADIANCERADIANCE
25
/
RADIANCERADIANCE
26
/
RADIANCERADIANCE
27
/
RADIANCERADIANCE
28
/
RADIANCERADIANCE
29
/
RADIANCERADIANCE
30
/
RADIANCERADIANCE
31
/
RADIANCERADIANCE
32
/
RADIANCERADIANCE
33
/
RADIANCERADIANCE
34
/
RADIANCERADIANCE
35
/
RADIANCERADIANCE
36
/
RADIANCERADIANCE
37
/
RADIANCERADIANCE
38
/
RADIANCERADIANCE
39
/
RADIANCERADIANCE
40
/
RADIANCERADIANCE
41
/
RADIANCERADIANCE
42
/
RADIANCERADIANCE
43
/
RADIANCERADIANCE
44
/
RADIANCERADIANCE
45
/
RADIANCERADIANCE
46
/
RADIANCERADIANCE
47
/
RADIANCERADIANCE
48
/
RADIANCERADIANCE
49
/
RADIANCERADIANCE
50
/
RADIANCERADIANCERecall
BRDF =
Ratio o“ out”oin” ra’ian‘e in one ’ire‘tion to in‘omin” (in“initesimal) irra’ian‘e “rom another
Total Out”oin” Ra’ian‘e in a spe‘i“i‘ ’ire‘tion will inte”rate over ‘ontributions “rom in‘omin” irra’ian‘e “romall ’ire‘tions.
ρ( , , , )θ
i
ϕ
i
θ
o
ϕ
o
( , ) =
∫
ρ( , , , ) ( , ) cos dL
o
θ
o
ϕ
o
θ
i
ϕ
i
θ
o
ϕ
o
L
i
θ
i
ϕ
i
θ
i
ω
i
51
/
RADIANCERADIANCE
52
/
RADIANCERADIANCE
53
/
RADIANCERADIANCE
54
/
RADIANCERADIANCE
55
/
RADIANCERADIANCE
56
/
RADIANCERADIANCE
57
/
RADIANCERADIANCE
58
/
RADIANCERADIANCEBi-directional Reflectance Distribution Function
Properties
Positivity:
Helmholtz Re‘ipro‘ity:
Total Ener”y leavin” sur“a‘e is less than total ener”y arrivin”
ρ( , , , ) ≥ 0θ
i
ϕ
i
θ
o
ϕ
o
ρ( , , , ) = ρ( , , , )θ
i
ϕ
i
θ
o
ϕ
o
θ
o
ϕ
o
θ
i
ϕ
i
∫
( , ) cos d ≥
∫
[
∫
ρ( , , , ) ( , ) cos d
]
cos dL
i
θ
i
ϕ
i
θ
i
ω
i
θ
i
ϕ
i
θ
o
ϕ
o
L
i
θ
i
ϕ
i
θ
i
ω
i
θ
o
ω
o
59
/
RADIANCE, IRRADIANCE, BRDFSRADIANCE, IRRADIANCE, BRDFSA’’itional Re“eren‘e: Forsyth & Pon‘e: Chapters 4 & 5Less Detaile’ / Qui‘k: Szeliski Se‘ 2.2
60
/
LIGHTSLIGHTS
61
/
LIGHTSLIGHTS
62
/
LIGHTSLIGHTS
63
/
LIGHTSLIGHTS
64
/
LIGHTSLIGHTS
65
/
LIGHTSLIGHTS
66
/
LIGHTSLIGHTS
67
/
LIGHTSLIGHTS
68
/
LIGHTSLIGHTS
69
/
LIGHTSLIGHTS
70
/
LIGHTSLIGHTS
71
/
LIGHTSLIGHTS
72
/
LIGHTSLIGHTS
73
/
LIGHTSLIGHTS
74
/
LIGHTSLIGHTS
75
/
LIGHTSLIGHTS
76
/
LIGHTSLIGHTS
77
/
LIGHTSLIGHTS
78
/
LIGHTSLIGHTS
79
/
LIGHTSLIGHTS
80
/
LIGHTSLIGHTS
81
/
LIGHTSLIGHTS
82
/
LIGHTSLIGHTS
83
/
LIGHTSLIGHTS
84
/
LIGHTSLIGHTS
85
/
LIGHTSLIGHTS
86
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
87
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
88
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
89
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
90
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
91
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
92
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
93
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
94
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
95
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
96
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
97
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
98
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
99
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
100
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
101
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
102
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
For ea‘h point, let the set o“ intensities be observe’ un’er li”hts .
I”nore ‘olor, assume is s‘alar (‘onvert the ima”es to ”rays‘ale / R+G+B).
Three observations o“ with ’ifferent, linearly in’epen’ent, will ”ive us . Three linear equations in three variables.
Given , we ‘an “a‘tor into an’ : is len”th o“ , an’ .
{ }I
i
{ }ℓ
i
I
i
= ρ ⟨ , ⟩ = ρ = (ρ ) = nI
i
n
ℓ
i
ℓ
T
i
n
ℓ
T
i
n
ℓ
T
i
I
i
ℓ
i
n
n ρ n
ρ ‖n‖ = n/‖n‖n
103
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
But usin” only three ima”es is unstable: there will be noise, et‘. We solve in the least squares sense.
Given ima”es un’er ’ifferent li”hts, “or ea‘h pixel:
where is a matrix, is a ve‘tor, an’ is a three-ve‘tor.
K K
n = ⇒ L n = I
⎡
⎣
⎢
⎢
⎢
⎢
⎢
⎢
ℓ
T
1
ℓ
T
2
ℓ
T
3
⋮
ℓ
T
K
⎤
⎦
⎥
⎥
⎥
⎥
⎥
⎥
⎡
⎣
⎢
⎢
⎢
⎢
⎢
⎢
I
1
I
2
I
3
⋮
I
K
⎤
⎦
⎥
⎥
⎥
⎥
⎥
⎥
L K × 3 I K × 1 n
104
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
Take ”ra’ient, set to 0:
This is a‘tually a equation. Solve usin” np.lingalg.solve to use Cholesky.
n = arg ‖L n − I = arg ( L) n − 2( I n + Imin
n
‖
2
min
n
n
T
L
T
L
T
)
T
I
T
( L) n = ( I)L
T
L
T
3 × 3
105
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
106
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
107
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
108
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREO
109
/
PHOTOMETRIC STEREOPHOTOMETRIC STEREOSome Pra‘ti‘al Issues:
Even thou”h we assume li”ht at in“inity, works well in pra‘ti‘e “or just “ar away li”hts.
Calibrate li”ht ve‘tor by lookin” at an ima”e o“ some known shape an’ albe’o (typi‘ally a matte sphere)
Te‘hni‘ally only works “or Lambertian obje‘ts. But o en, ‘an make obje‘ts Lambertian with polarizers.Also, estimate’ normals are typi‘ally well-’e“ine’ “or a vali’ set o“ pixels in the ima”e.You ll ‘reate / be ”iven a mask o“ these vali’ pixels.
ℓ
110
/
NORMALS TO DEPTHNORMALS TO DEPTH
111
/
NORMALS TO DEPTHNORMALS TO DEPTH
112
/
NORMALS TO DEPTHNORMALS TO DEPTH
113
/
NORMALS TO DEPTHNORMALS TO DEPTH
114
/
NORMALS TO DEPTHNORMALS TO DEPTH
115
/
NORMALS TO DEPTHNORMALS TO DEPTH
116
/
NORMALS TO DEPTHNORMALS TO DEPTH
117
/
NORMALS TO DEPTHNORMALS TO DEPTH
118
/
NORMALS TO DEPTHNORMALS TO DEPTH
119
/
NORMALS TO DEPTHNORMALS TO DEPTH
120
/
NORMALS TO DEPTHNORMALS TO DEPTH
121
/
NORMALS TO DEPTHNORMALS TO DEPTHZ = arg ‖ − ∗ Z + ‖ − ∗ Zmin
Z
g
x
f
x
‖
2
g
y
f
y
‖
2
122
/
NORMALS TO DEPTHNORMALS TO DEPTH
We ll use as:
Z = arg ‖ − ∗ Z + ‖ − ∗ Z + λR(Z)min
Z
g
x
f
x
‖
2
g
y
f
y
‖
2
R(Z)
R(Z) = (Z ∗ )[n for =
∑
n
f
r
]
2
f
r
−1/9
−1/9
−1/9
−1/9
8/9
−1/9
−1/9
−1/9
−1/9
123
/
NORMALS TO DEPTHNORMALS TO DEPTH
Version 1: Do it in the Fourier Domain (‘alle’ Frankot-Chellappa)
Assume that in the maske’ out re”ions, .
is FT o“ ’epth map, is FT o“ , is (‘ir‘ular pa’’e’) FT o“ , an’ so on.
In ”eneral, shoul’ a’’ some very small number (e.”., ) to ’enominator “or stability.
In parti‘ular, what is the ’enominator “or ?
Both numerator an’ ’enominator are 0, be‘ause normals tell us nothin” about avera”e ’epth / offset.
Expli‘itly set to 0.
Z = arg ‖ − ∗ Z + ‖ − ∗ Z + λR(Z)min
Z
g
x
f
x
‖
2
g
y
f
y
‖
2
= = 0g
x
g
y
(Z)[u, v] =
[u, v] [u, v] + [u, v] [u, v]F
¯
x
G
x
F
¯
y
G
y
| [u, v] + | [u, v] + λ| [u, v]F
x
|
2
F
y
|
2
F
r
|
2
(Z) G
x
g
x
F
x
f
x
10
−12
[u, v] = [0, 0]
(Z)[0, 0]
124
/
NORMALS TO DEPTHNORMALS TO DEPTH
Version 2: Use ‘onju”ate ”ra’ient.
Allows us to use ’ifferent wei”hts “or ’ifferent pixels.
Z = arg ‖ − ∗ Z + ‖ − ∗ Z + λR(Z)min
Z
g
x
f
x
‖
2
g
y
f
y
‖
2
125
![,*3*7&1 (8* & (TRUZYJW;NXNTS 1&899.2* 74':89+.99.3,ayan/courses/cse559a/PDFs/lec...2JYJWXY T5 N]JQX 1THFYNTST SX JSXTWU QFSJN SR JYJWX 1JYIXX F^J FHMX JSXTWU N]JQN X R JYJWX\ NIJ 1THFYNTSN](https://img.pdfslide.net/doc/110x75/60ad80eaa512aa28a81f81cd/371-8-truzyjwnxnts-18992-7489993-ayancoursescse559apdfslec.jpg)
