Generative Adversarial Networks (GANs)

https://www.theverge.com/2019/3/19/18272602/ai-art-generation-gan-nvidia-doodle-landscapes

Colorizing (not Restoring)

https://blog.floydhub.com/colorizing-and-restoring-old-images-with-deep-learning/

https://news.developer.nvidia.com/generating-and-editing-high-resolution-synthetic-images-with-gans/?ncid=--43604

https://github.com/Aggregate-Intellect/awesome-does-not-existPeople• This Person Does Not Exist• Which Face is Real

Animal• This Cat Does Not Exist• These Cats Do Not Exist

Text & Documents• Was this poem written by a

human or a computer?• This Resume Does Not Exist

Objects• This Car Does Not Exist• (Video) These Cars Do Not Exist• (Video) This Ramen Does Not Exist

Rooms• This Rental Does Not Exist

Entities• This Startup Does Not Exist

Cartoon/Anime• This Waifu Does Not Exist• These Waifus Do Not Exist

GANs

• Two deep neural networks working against each other (adversarial)• A generator creates something new• A discriminator tries to identify generated data vs real data

• GANs have been applied in many domains• Computer vision (classification, regression, etc.)• Audio generation• Speech recognition• NLP (generation and recognition)

Generative Adversarial Networks, Goodfellow et al. (2014)

GANs

Real Data

Fake Data(Generated)

Discriminator

GeneratorGenerationParameters(Template)

Prediction

GANs

Real Data

Fake Data(Generated)

Discriminator

GeneratorGenerationParameters(Template)

Prediction

Classification (real or fake)

What kind of network would you use?

GANs

Real Data

Fake Data(Generated)

Discriminator

GeneratorGenerationParameters(Template)

Prediction

Classification (real or fake)

What kind of network would you use?

Regression (generate features that fool the discriminator)

What kind of network would you use?

GANs

Real Data

Fake Data(Generated)

Discriminator(ResNet-34)

Generator(U-Net)

GenerationParameters(Template)

Prediction

Classification (real or fake)

What kind of network would you use?

Regression (generate features that fool the discriminator)

What kind of network would you use?

Example architectures assume we are working with image data.

Fastai Method

Freeze the generator and train the critic for one step:• grab one batch of real images• generating one batch of fake images• compute the critic’s loss on all real and fake images• update the critic’s parameters

Freeze the critic and train the generator for one step:• generating one batch of fake images• use the critic to evaluate the fake images• compute the generator’s loss based on its ability to fool the critic• update the generator’s parameters

Related Technique: Autoencoder

https://blog.keras.io/building-autoencoders-in-keras.html

Autoencoding is a data compression algorithm where the compression and decompression functions are:

1) data-specific2) lossy3) learned automatically from examples rather than

engineered by a human.

Practical applications of autoencoders: 1) data denoising2) dimensionality reduction for data visualization

With appropriate dimensionality and sparsity constraints, autoencoders can learn data projections that are more interesting than PCA or other basic techniques.

Related Technique: Variational Autoencoders

http://kvfrans.com/variational-autoencoders-explained/

In [1]: %reload_ext autoreload%autoreload 2%matplotlib inline

In [2]: from fastai.vision import *from fastai.vision.gan import *

In [4]: data_path = untar_data(URLs.LSUN_BEDROOMS)

data_transforms = get_transforms()

image_size = 64batch_size = 128

def get_data(path, bs, size):return (GANItemList.from_folder(path)#, noise_sz=100)

.no_split()

.label_from_func(noop)

.transform(tfms=[[crop_pad(size=size, row_pct=(0,1), col_pct=(0,1))], []], size=size, tfm_y=True)

.databunch(bs=bs)

.normalize(stats = [torch.tensor([0.5,0.5,0.5]), torch.tensor([0.5,0.5,0.5])], do_x=False, do_y=True))

data = get_data(data_path, batch_size, image_size)

1 of 6

In [5]: data.show_batch()

In [6]: generator = basic_generator(in_size=image_size, n_channels=3, n_extra_layers=1)critic = basic_critic (in_size=image_size, n_channels=3, n_extra_layers=1)

In [7]: learn = GANLearner.wgan(data, generator, critic, switch_eval=False,opt_func = partial(optim.Adam, betas = (0.,0.99)), wd=0.)

2 of 6

In [8]: learn.fit(20, 3e-4)

3 of 6

Total time: 44:42

epoch train_loss gen_loss disc_loss

1 -0.605838 0.424219 -0.800274

2 -0.581525 0.419157 -0.776750

3 -0.505078 0.329789 -0.677572

4 -0.453455 0.324825 -0.608423

5 -0.444416 0.205033 -0.545488

6 -0.353311 0.186529 -0.470391

7 -0.284996 0.151906 -0.379780

8 -0.271385 0.134970 -0.357321

9 -0.241809 0.123559 -0.317124

10 -0.231374 0.123763 -0.298770

11 -0.201024 0.109258 -0.283154

12 -0.188730 0.136541 -0.267419

13 -0.193091 0.096157 -0.247904

14 -0.165972 0.114151 -0.240640

15 -0.179664 0.085967 -0.236525

16 -0.186926 0.062920 -0.226390

17 -0.153478 0.088798 -0.209249

18 -0.176428 0.055510 -0.212063

19 -0.141382 0.089210 -0.192962

20 -0.164651 0.052121 -0.201119

4 of 6

5 of 6

In [9]: learn.gan_trainer.switch(gen_mode=True)learn.show_results(ds_type=DatasetType.Train)

6 of 6