ARShadowGAN: Shadow Generative Adversarial Network for...

ARShadowGAN: Shadow Generative Adversarial Network for Augmented Reality in Single Light Scenes

Daquan Liu1, Chengjiang Long2*, Hongpan Zhang1, Hanning Yu1, Xinzhi Dong1, Chunxia Xiao1,3,4*

1School of Computer Science, Wuhan University

2Kitware Inc., Clifton Park, NY, USA

3National Engineering Research Center For Multimedia Software, Wuhan University

4Institute of Artificial Intelligence, Wuhan University

cjfykx@gmail.com, {daquanliu,zhanghp,fishaning,dongxz97,cxxiao}@whu.edu.cn

∗This work was co-supervised by Chengjiang Long and Chunxia Xiao. 1

With the physically based rendering theory:

• It requires an inverse rendering process which is very expensive and challenging in practice.

• What’s worse, any inaccurate estimation may result in unreasonable virtual shadows.

We propose a data-driven based ARShadowGAN to directly generate virtual shadows without inverse

rendering. With the powerful generation ability of GAN, it is much more efficient and easier to use.

How to Automatically GenerateVirtual Shadows for AR?

Shadow-AR Dataset

• Shadow-AR dataset contains total 3,000 image quintuples

Input Image Input Mask Mask-Real Object Mask-Real Shadow Target Image

ARShadowGAN

Encoder

vShadow

Decoder

ResNetblock

additionDiscrim

RealFake

Encoder

32x32x512

rShadow

Decoder

Occluder

Decoder

16x16x512

Refinement

Pooling

concatenation

Attention Visualization

Input Image Input Mask Attn-Real Shadow Attn-Real Object 5

Experimental Results

Input Image Pix2Pix Pix2Pix-Res ShadowGAN Mask-ShadowGAN ARShadowGAN GT6

Shadow Generation

Input Image Input Mask Attn-Real Shadow Attn-Real Object Output Image

Paper QR Code:

http://www.chengjianglong.com/publications/ARShadowGAN_CVPR.pdf

Thank you!

This work was partly supported by Key Technological Innovation Projects of Hubei Province (2018AAA062), Wuhan

Science and Technology Plan Project (No. 2017010201010109), the National Key Research and Development Program of

China (2017YFB1002600), the NSFC (No. 61672390, 61972298). The corresponding author is Chunxia Xiao.

ARShadowGAN: Shadow Generative Adversarial Network for...

Documents

Generative adversarial networks - University Of Illinoisslazebni.cs.illinois.edu/fall18/lec13_gan.pdfconvolutional generative adversarial networks, ICLR 2016 Four transposed convolution

Semi-Supervised Generative Adversarial Hashing for Image Retrievalopenaccess.thecvf.com › ...Wang_Semi-Supervised_Generative_Adver… · Semi-Supervised Generative Adversarial Hashing

Generative Adversarial Networks (GANs) - Sciencesconf.org · Generative Adversarial Networks (GANs) Anirban Mukhopadhyay TU Darmstadt, Germany. Introduction Generative vs. Discriminative

GENERATIVE ADVERSARIAL NETWORKS - Deep Learning

Generative Adversarial Transformers - Stanford University

Generative Adversarial Transformers

Generative Adversarial Perturbations - arXiv

A (conditional) Generative Adversarial Network

Collaborative Sampling in Generative Adversarial … › pdf › 1902.00813.pdfGenerative Adversarial Networks Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) consist

AnomiGAN: Generative Adversarial Networks forpsb.stanford.edu/psb-online/proceedings/psb20/Bae.pdf2.4. Generative Adversarial Networks GANs25 are designed to complement other generative

Perceptual Generative Adversarial Networks for Small ...openaccess.thecvf.com/content...Generative_Adversarial_CVPR_2017… · Perceptual Generative Adversarial Networks for Small

Wasserstein Generative Adversarial Networksproceedings.mlr.press/v70/arjovsky17a/arjovsky17a.pdf · Wasserstein Generative Adversarial Networks Martin Arjovsky1 Soumith Chintala2

Unsupervised Learning – Generative Models (PixelRNNs, PixelCNNs) – Generative ... · 2019. 3. 28. · Generative Adversarial Networks Ian Goodfellow et al., “Generative Adversarial

Time-series Generative Adversarial Networks

Hypothetical Exploration on Generative Adversarial Networks · Convolutional Neural Networks, Generative Adversarial Net- works, Laplacian Pyramid of Adversarial Networks. I. INTRODUCTION

Lecture 9: Unsupervised, Generative & Adversarial Networks · Lecture 9: Unsupervised, Generative & Adversarial Networks ... GENERATIVE & ADVERSARIAL NETWORKS - 20 o Autoencoders

GENERATIVE ADVERSARIAL NETWORKS - Deep Learningdeeplearning.cs.cmu.edu/F20/document/recitation/GANS-ppt.pdf · 2021. 3. 22. · Generative Adversarial Networks Generative Models We

Lecture 7: Generative Adversarial Networks

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

Shadow Detection With Conditional Generative …openaccess.thecvf.com/content_ICCV_2017/papers/Nguyen_Shadow... · Shadow Detection with Conditional Generative Adversarial Networks