The Use of Modern Controller Devices at Schools: Game-Based Learning with

the Leap Motion

Norbert Spot

Supervisor: Assoc.Prof. Dipl.-Ing. Dr.techn. Martin Ebner

Contents• Introduction

• Motivation

• The Prototype

• Evaluation

• Conclusion

• Demo

Introduction• Innovative Controller Devices

• MS Kinect

• MakeyMakey

• Bare Conductive’s Touch Board

• Leap Motion

MS Kinect• RGB Camera (VGA @ 30Hz)

• depth sensor (IR laser with a monochrome CMOS, VGA @ 30Hz)

• multi array microphone

• full-body 3D motion capture, facial recognition, voice recognition

• Xbox 360, Xbox One, Windows PC

MakeyMakey• invention kit

• turns everyday objects into touch-pads

• clip two objects to the MakeyMakey (yourself and a banana)

• when you touch the banana the device sends a keyboard message to the computer

• works with any material that conducts at least a tiny bit of electricity

Bare Conductive Touch Board

• similar to MakeyMakey

• works great with Bare Conductive's Electric Paint

• paint a light switch or a custom interactive surface

• connect anything conductive

Leap Motion• 3D IR Camera

• 3 IR light sources, 2 wide angle (150 degrees) monochrome IR cameras @ 200Hz

• tracks all 10 fingers, both hands and arms

• precision up to 0,01mm*

• works on Mac, Linux, Windows (native, web)

*real world avarage accuracy 0,7mm (avarage accuracy of the human hand 0,4mm)

Natural User Interface• according to D. Plemmons and P. Mandel

intuitive interfaces are

• learnable

• understandable

• habitual

• natural

– Steve Ballmer

“I believe we will look back on 2010 as the year we expanded beyond the mouse and keyboard and started incorporating more natural forms of interaction such as touch,

speech, gestures, handwriting, and vision -- what computer scientists call the “NUI” or

natural user interface.”

Game-Based Learning• playing games may support the development of

certain skills and strategies

• problem-solving

• decision-making

• understanding complex systems

• planning, data handling

Motivation

• Game-Based learning with computers can be motivational

• the use of iPads at schools makes teaching/learning more interesting

• possibilities provided by the Leap Motion

• Leap Motion

• assessing a CT scan with the SW called Osirix

• virtual keyboard DexType

• using the Leap Motion with Oculus Rift

• controlling a drone

• painting Heineken bottles

• games

The Goal

• make use of a modern controller device, which provides a natural user interface

• create a prototype of an application with game-based learning in mind

• see how it performs in the real world

The Prototype

• a game for exercising the small multiplication chart

• built in Unity3D

• scripts written in C#

Unity3D• full-featured 3D and 2D game

development environment

• works on Windows andMac OS X

• it was used to create

• Bad Piggies

• Monument Valley

• Threes

The Prototype

Challenges• dealing with Unity3D and C#

• designing an interesting game world and an intuitive interface

• navigating with a "virtual joystick”

• efficient random exercise generation

• Fisher and Yates method

• the game menu

The Game Menu

The Game Menu

Evaluation• 3rd grade elementary school

students

• kids interviewed with the help of cut-off technique

• a small group got 5 statements

• ranked the statements as a group with the help of smileys

Evaluation• statements

• I would like to play the game again

• It was easy to pop the balloons

• It was easy to find the balloons

• I think the exercises were easy

• I think the game was easy to play

Evaluation

• majority gave the best mark for all the statements

• the kids loved the game

• “The best game I have ever played”

Evaluation• four groups at the end

• kids good in playing the game

• kids good in math, having problems with gameplay

• kids good in gameplay, slower in math

• kids bad in math, having problems with gameplay

Conclusion

• Game-Based learning with the Leap Motion is fun and motivational

• might not be for everyone

• fun way to practice a school subject

Demo

Thank you for your attention!

• Colgan A. (2014) How Does the Leap Motion Controller Work? http://blog.leapmotion.com/hardware-to-software-how-does-the-leap-motion-controller-work/

• Griffiths, M. (2014) Playing video games is good for your brain – here’s how http://theconversation.com/playing-video-games-is-good-for-your-brain-heres-how-34034

• Kuntz Ch. (2013) Controlling Osirix with the Leap Motion While Scrubbed into Surgery https://www.facebook.com/photo.php?v=10152121411384392

• Liebeskind D. (2014) The Beginning of a Drone Revolution http://blog.leapmotion.com/the-beginning-of-a-drone-revolution/

• Plemmons D., Mandel P. Introduction to Motion Control https://developer.leapmotion.com/articles/intro-to-motion-control

• Weichert F., Bachmann D., Rudak B., Fisseler D. (2013) Analysis of the Accuracy and Robustness of the Leap Motion Controller, Department of Computer Science VII, Technical University Dortmund, Dortmund, Germany