20
Wasserstein GAN

Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

  • Upload
    others

  • View
    8

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Wasserstein GAN

Page 2: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Generativni modeli- Predviđamo distribuciju

verovatnoća nad uzoračkim prostorom.

- Centralno mesto u pristupu zauzima nekakva koncepcija distance.

Page 3: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Generative Adversarial Networks

Page 4: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Generative Adversarial Networks

Page 5: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Ključni problemi- Update postaje gori što je diskriminator bolji.- Treniranje GAN modela je kranje nestabilno.

Page 6: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Disjunktni nosači gustina

Šta se dešava ako su nosači disjunktni?

Page 7: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Poklapanje mnogostrukosti

Page 8: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Distribucije čiji se nosači ne poklapaju

Page 9: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Cost funkcije

Page 10: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

logD trik

Page 11: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Noisy generatorJedno od rešenja ovog problema je dodavanje normalno raspodeljenog šuma generatoru i podacima. U tom slučaju loss ima stabilne gradijente i konvergira simetričnom izrazu:

Sa druge strane, intenzitet šuma koji je potreban da bi se postiglo stabilno treniranje je krajnje veliki i drastično obara kvalitet generisanih vrednosti.

Page 12: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Alternativna metrika/divergencija

Page 13: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Wasserstein u odnosu na druge divergencije

Page 14: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Wasserstein dual i treniranje

Page 15: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Algoritam

Page 16: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Praktični rezultati- Batchnorm nema nikakvog efekta na proces treniranja- Treniranje postaje praktično neosetljivo na arhitekturu generatora

Page 17: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Praktični rezultati

Diskriminator(kritičar) ima skoro konstantan gradijent

Page 18: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Praktični rezultatiLoss funkcija se može interpretirati na razuman način

Page 19: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Problemi- “Weight clipping” kod diskriminatora.- Spora konvergencija na visokodimenzionim podacima.

Page 20: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch

Teorija i praksa