Upload
others
View
63
Download
0
Embed Size (px)
Citation preview
Introduction to Augmented
RealityCreating Augmented reality costume: concepts, tools, decisions, failures
Aurelijus Banelis
Augmented Reality
What is
To the end-user
Augmented Reality
● Real world
● Real world● Camera
● Real world● Camera● Artificial
● Real world● Camera● Artificial● Follows
AugmentedReality
Hello worldexample
Complete product
≠
Augmented Reality
Computer Vision
3D Graphics
Infrastructure
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
RAW data → ObjectsData from
cameraNoise,Useful
Analyse,Compare
Meaning,decision
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
RAW data → ObjectsData from
cameraNoise,Useful
Analyse,Compare
Meaning,decision
125142365814152451542247582451424125384864535315315313547876815453151313515748448343818131
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
RAW data → ObjectsData from
cameraNoise,Useful
Analyse,Compare
Meaning,decision
125142365814152451542247582451424125384864535315315313547876815453151313515748448343818131
001000011100010010100001110010000001010110101001001000101111101010010000101101001000101000
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
RAW data → ObjectsData from
cameraNoise,Useful
Analyse,Compare
Meaning,decision
125142365814152451542247582451424125384864535315315313547876815453151313515748448343818131
001000011100010010100001110010000001010110101001001000101111101010010000101101001000101000
001000011100010010100001110010000001010110101001001000101111101010010000101101001000101000
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
RAW data → ObjectsData from
cameraNoise,Useful
Analyse,Compare
Meaning,decision
125142365814152451542247582451424125384864535315315313547876815453151313515748448343818131
001000011100010010100001110010000001010110101001001000101111101010010000101101001000101000
Found { marker=”m12” x = 10, y = 20, accuracy=0.8}
Found { marker=”m98” x = 15, y = 44, accuracy=0.6}
001000011100010010100001110010000001010110101001001000101111101010010000101101001000101000
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
Noises by nature
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
So filters in use
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
Including redundancy
Folding
if m
arke
r1 a
nd m
arke
r2Low light
Low resolution
Similarity
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
CV → 3DCombinedFrame from
cameraData from ARToolkit
3D models to be drawn
⇒ ++
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
CV → 3DCombinedFrame from
camera
Captured preview image from device’s camera
Data from ARToolkit
3D models to be drawn
⇒ ++
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
CV → 3DCombinedFrame from
camera
Captured preview image from device’s camera
Data from ARToolkit
Found { marker=”m12” rot = (10, 20, 5) scale = 0.3 pos = (5, 4, 22)}
Found { marker=”m98” rot = (5, 33, 44) scale = 0.4 pos = (22, 8, 3)}
3D models to be drawn
⇒ ++
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
CV → 3DCombinedFrame from
camera
Captured preview image from device’s camera
Data from ARToolkit
Found { marker=”m12” rot = (10, 20, 5) scale = 0.3 pos = (5, 4, 22)}
Found { marker=”m98” rot = (5, 33, 44) scale = 0.4 pos = (22, 8, 3)}
3D models to be drawn
OpenGL ES 1.1
⇒ ++
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
CV → 3DCombinedFrame from
camera
Captured preview image from device’s camera
Data from ARToolkit
Found { marker=”m12” rot = (10, 20, 5) scale = 0.3 pos = (5, 4, 22)}
Found { marker=”m98” rot = (5, 33, 44) scale = 0.4 pos = (22, 8, 3)}
● Frame
● Transform● Draw 3D
3D models to be drawn
OpenGL ES 1.1
⇒ ++
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
Idea → 3D models
3D Modeling soft AndroidApp
Idea / 2D pictures
⇒ .objWavefront
⇒
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
⇒ 3D Modeling soft Android
App
Idea → 3D models
Idea / 2D pictures
⇒ .objWavefront
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
Idea → 3D models
3D Modeling soft AndroidApp
Idea / 2D pictures
⇒ .objWavefront
⇒
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
Idea → 3D models
3D Modeling soft AndroidApp
Idea / 2D pictures
⇒ .objWavefront
⇒
Com
pute
r Vis
ion
3D G
raph
ics
Infr
astr
uctu
re
Idea → 3D models
Normals Debug
3D Modeling soft AndroidApp
Idea / 2D pictures
⇒ .objWavefront
Com
pute
r Vis
ion
Infr
astr
uctu
re3D
Gra
phic
s
Invest in 3D rendering
Debug separately
Consider native implementation
Copying ARToolKit native libraries takes time
Implement custom widgets for debugging(faster than reloading whole code)
Garbage collection between Java native and Buffer structures takes time (slow loading)
3D G
raph
ics
Infr
astr
uctu
reC
ompu
ter V
isio
n End-to-end
Android (SDK)
Blender
ARToolKit
QR + Server
Draw on fabric
Wraps computer vision, starting point
Custom logic (e.g. helmet position)
Generating 3D models
Telling, how to find application
Some shops requires vector images
Questions?
References and useful links● http://aurelijus.banelis.lt/prezentations/augmetned-reality-2015/ag-costume.pdf ● https://youtu.be/KIafn8hFi8c?t=4m41s
● http://artoolkit.org/ ● https://github.com/Itseez/opencv ● O’REILLY Learning OpenCV● https://en.wikipedia.org/wiki/Scale-invariant_feature_transform● https://en.wikipedia.org/wiki/Normal_%28geometry%29 ● https://unity3d.com/● https://www.blender.org/ ● https://inkscape.org/en/