Augmented reality - it.uu.se What is augmented reality Augmented Reality (AR) is the process of detecting

DEPARTMENT OF INFORMATION TECHNOLOGY

Augmented reality Display of possibilities of augmented reality using Java and C++

Marcus Agelii & Gösta Agerberg Supervisor: Maya Neytcheva & Sergio Courtade Project in computer sience: Report January 2013

Jan 2013

P R

O J E

C T

R E

P O

R T

Abstract

The purpose of this project is to explore augmented reality and investigate its possibilities.

Another target was to have software which ran on a mobile device. In this report we

summarize the conducted study on the current progress in the field and describe two

applications, for Android, developed under the project work. The applications

demonstrate what augmented reality can be used for. The applications are written in Java

and C++.

3

Contents

1. Introduction

What is augmented reality ............................................................................................... 4

Markers ............................................................................................................................ 4

Augmented reality - early history and current development ........................................... 5

Augmented reality for mobile devices ............................................................................ 6

Android applications ....................................................................................................... 6

2. Project description and aims

Problem description ......................................................................................................... 7

Limitations ....................................................................................................................... 7

Goals ................................................................................................................................ 7

3. Workflow and implementation ..................................................................................... 8

4. Results and conclusions

Logotype app ................................................................................................................. 12

Virtual buttons app ......................................................................................................... 13

Conclusions .................................................................................................................... 13

5. Discussion

Difficulties ..................................................................................................................... 14

Possible developments of our apps ................................................................................ 14

Possibilities of Augmented Reality ................................................................................ 15

6. Acknowledgements ...................................................................................................... 17

7. References ..................................................................................................................... 18

4

1. Introduction

What is augmented reality

Augmented Reality (AR) is the process of detecting objects in the reality and adding

information to that reality. The best way to explain this further is by an example: Imagine

that you are looking at a screen; the screen shows a camera feed of today’s newspaper.

When the camera comes to a certain picture in the newspaper, instead of just showing that

newspaper with the picture, it instead shows a video taking the picture’s place in the

newspaper.

The effect is achieved by enabling the program that detects the picture to search through

its database looking for what actions to take when it finds that exact image. This specific

program has videos as actions, and uses it by overlaying the image by the video.

Markers

To detect something in reality an AR application needs certain input; this is usually from

a camera but could also be from other sources such as GPS or accelerometers. If the AR

application uses a camera feed as an input and recognizes an object using image

recognition software, this object is called a marker. Often specific images are used as

markers but it could also be real objects such as a car or a building. There is also an AR

technology called markerless based AR, instead of trying to recognize specific markers

from a camera feed, this technology tries to do a mapping of the surrounding environment

and uses this map to augment reality.

Figure 1: Picture in a newspaper being replaced with a video using augmented reality

5

Augmented reality - early history and current development

AR as a concept and idea has been around for many years but it’s only in the recent years

that technology has been made small and fast enough to make AR available for everyday

use. The first AR system was developed in the 60’s but the term AR was not coined until

1990 by a Boeing researcher called Tom Caudell. 1 One of the first uses of simple AR

technology was by pilots in fighter jets, where the pilot had a HUD (Heads Up Display)

showing altitude, velocity and other useful flight data in their main field of view. This can

however be discussed as being real AR but when this technology evolved into also

showing waypoints and cues over targets (Target designators) this becomes more of what

people see AR as today.

In recent years as mobile phones and handheld devices have been made cheaper and more

powerful, the potential of AR applications have increased significantly. Today there are

many apps available to download from both Google play and Apples App store. The

application field of these apps ranges from navigation to entertainment, home furnituring

etc. There is however still a lot of developing needed before these apps surpasses the

usefulness and the reliability of other methods and apps not using AR.

Figure 2: A HUD display in a fighter jet.

1 Kangdon, Lee; http://link.springer.com/article/10.1007%2Fs11528-012-0559-3

6

Augmented reality for mobile devices

AR right now is optimal for mobile devices and especially for smartphones. The

smartphones have the ingredients needed for the AR to work, a camera, a CPU and a

screen, all in the same small package. Another feature that makes smartphones very

suitable for implementing AR apps is their accessibility both to consumers and to

developers. It is easy to use a smartphone and it is easy to develop software for a

smartphone. The smartphone is also very mobile and as such very easy to move around an

augmented object. Since the smartphones also have other detectors such as

accelerometers and GPS, these can also be used when handling AR.

Android applications

Android is the name of an operating system primarily used for mobile and tablet devices

but also in other devices such as cameras and watches. 2 Android is based on Linux but

have been further developed by Google to better suit smartphones and tablets (such as

energy management). Android is open source and primarily developed by Google, but due

to its open source nature many carriers and smartphone manufacturers have adapted

Android to fit their products. For Android there is a wide variety of third-party

applications that can be installed on devices running Android. These applications (apps)

are either distributed by Google’s own store called Google play or by other available

stores such as Amazon store, or by third party websites since it’s possible to

download .apk files to an android device and install them manually.

To develop applications for Android, Google have released an Android Software

Development Kit (SDK) and the applications are developed using Java. This software kit

includes all that is needed to start developing applications for Android including a

debugger, software libraries and tutorials. There is also a native development kit available

for applications or extensions in C or C++. The official Integrated Development

Environment (IDE) supported is Eclipse 3 using the Android Development Tools plugin.

2 http://www.android.com

3 http://www.eclipse.org

7

2. Project description and aims

Problem description

The project description for our project was rather general and did not contain any specific

details and limiting conditions. Our supervisor gave us the “AR for mobile devices” topic

and ideas in this topic to investigate and let us set the goals and limitations by ourselves.

The ideas to focus on were basically three; the image analysis involved in reading the

images, the creation and reading of information from the database, or the projection of

information. Therefore our first task was to actually come up with a more specific project

description. An advice we got from