Advance Multimedia Tech. Augmented reality. Pertemuan 1

Augmented Reality

Gia MuhammadState Islamic University Syarif Hidayatullah

Apa itu Augmented Reality?

• Virtual Realityo Gameso Simulasio Dll

• ARo Augmented – Penambahano Reality – Realitas (Lingkungan Nyata)o AR – Realitas yang ditambahkan

• Terminologi AR (Azuma, 1997)o Kombinasi Realitas dan Virtualo Interaktif dalam Waktu Nyatao Berjalan dengan model 3D

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Pemakaian AR di dunia industri

• Advertising/Branding/Digital Marketing• Simulasi

o Medicalo Militer

• Edukasi

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Perangkat penunjang AR

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Media untuk melihat AR

• Monitor atau Smart Phone (Handheld Type)• Goggles (Head Mounted Device)• Contact Lens

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Klasifikasi Tracking di AR

• Marker-Based Trackingo Marker Tradisional/Konvensional (Citra Hitam Putih dengan Border, ARToolKit)o 2D Barcode

• ARToolKitPlus• ARTag & WebTag• QR Code

o Hybrid Marker-Based – Konvensional & 2D Barcode• QR Code Marker

• Markerless Tracking/Feature-Based Trackingo Gambar Berwarnao GPSo Biometrik

• Wajah• Tubuh• Hand Gesture• Dll

• Hybrid Tracking – Marker-based dan Feature-Basedo Barcode-Assisted (Park, et al, 2011)o Sensoro Model BasedGiaMuhammad | Praktikum Advance Technology Multimedia

Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Marker-Based Tracking

• Marker Tradisional (Citra Hitam Putih dengan Border, ARToolKit) [1]

• 2D Barcodeo ARToolKitPlus [2]o ARTag & WebTag [3]o QR Code [4]

• Hybrid Marker-Basedo QR Code Marker [5]

1 2 3 4 5

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Markerless/Feature-Based Tracking

• Gambar Berwarna• GPS• Biometrik

o Wajaho Tubuho Hand Gestureo Dll

Bidang Ilmu yang Terkait

• Pengolahan Citra Digital• Computer Vision

o Model Kamera & Kalibrasi Kamerao Perspective Transformationo Segmentasi Citrao Template Matchingo SURF/SIFTo Optical Flowo Stereo Vision

• Grafika Komputer

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

AR Framework/Tools• ARToolKit• ARToolKitPlus• ARTag• NyARToolKit• FlarToolKit• PaperVision 3D• StudierStube• Vuforia• DFusion• metaIO• bazAR• Dan Masih banyak lagi

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Framework untuk Penelitian AR

• Sering dipakai oleh peneliti-peneliti di duniao OpenCV, EmguCV, SimpleCV

• Image Processing• Computer Vision• Machine Learning

o OpenNIo OpenFramework

• Framework 3D Modellingo OpenGLo Ogre3Do OpenSpace3Do Unity3D

• Framework Tambahan jika diperlukano OpenCL

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Sistem AR sederhana

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Komputasi pada AR Marker/Feature-

Based*1. Kalibrasi Kamera2. Deteksi Marker

o Low Level Image Processing3. Proyeksi Perspektif4. Identifikasi

o Template Matchingo PCA (Principal Component Analysis)o Decoding 2D Barcode dengan Algoritma Reed-Solomon

5. Pose Estimationo RPP (Robust Planar Pose)o ARToolKit Pose Estimation

6. Rendering 3D

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Deteksi Marker• Algoritma Deteksi Marker oleh Hirokazu Kato

(ARToolKit)

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Deteksi Marker• Low Level Image Processing

o Akuisisio Grayscalingo Thresholdingo Connected Componentso Contouringo Deteksi Sudut

• Deteksi Marker dengan OpenCVo Grayscalingo Thresholdingo Contouringo Ekstraksi Sudut

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Deteksi Marker• Software Tools

o Microsoft Visual C++ 2008/2010o OpenCV 2.1o ARToolKit 2.7.1

• Hardware Toolso Web Camera 1.3 mp

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

Reference• Siltanen S. “Theory and applications of marker-

based augmented reality”, 2012• Bradski S, Kaehler A. “Learning OpenCV”, 2008• Agusta. G.M. “QR Code Tracking with Merging on

Conventional Marker based Backpropagation Neural Network”

GiaMuhammad | Praktikum Advance Technology MultimediaDepartment of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah