Walking in Place with Wearable Technology: the development of a system for travel training and...

Walking in Place with Wearable Technology THE DEVELOPMENT OF A SYSTEM FOR TRAVEL TRAINING AND APPLIED TRAVEL FOR PEOPLE WITH A VISUAL IMPAIRMENT.

About this talk

In this talk we present the application of commercial off the shelf mobile Virtual Reality and Wearable Health technologies to formulate a novel system for travel training and applied travel for people with a visual impairment (VI).

At the heart of the system is a walking distance estimation algorithm derived from the design and analysis of a custom cane-use walking classification for applied Orientation Mobility Cane Technique (OMCT).

To demonstrate the effectiveness of the current system we show its application utilising a standard assistive white cane.

Finally we discuss how the system can be improved via the design of a custom tip with odometry sensing capability and haptics.

THE SAMSUNG GEAR VR

THE MICROSOFT BAND

VISUAL IMPAIRMENT AND BLINDNESS

285 million people are estimated to be visually impaired worldwide: 39 million are blind and 246 have low vision.

About 90% of the world's visually impaired live in low-income settings.

82% of people living with blindness are aged 50 and above.

Globally, uncorrected refractive errors are the main cause of moderate and severe visual impairment; cataracts remain the leading cause of blindness in middle- and low-income countries.

The number of people visually impaired from infectious diseases has reduced in the last 20 years according to global estimates work.

80% of all visual impairment can be prevented or cured.

WHO - Fact Sheet N°282

This concept builds upon a collective body of research including that of The Virtual Cane; mapping the technology of the Nintendo WII controller to that of the white cane.

RESEARCH FOUNDATION

During our previous research the following outcomes were identified and demonstrated:

Virtual representations of unknown Physical Environments (PE) allow those

with a VI the opportunity to develop both mental-models and navigational strategies in advance, which can then can then be employed in the PE.

Typically, such systems are found to disrupt the match between both the mental body-model formed from proprioceptive data and the sensory data supplied by the VE.

RESEARCH FOUNDATION

This is because the methods applied for VE navigation often require a mental association between peripheral input and the act of virtual translation.

For those with VI or blindness the level of abstraction of such mappings is further exacerbated due to a lack of or impaired visual sensory cues.

In order to evaluate and ultimately combat the abstraction effect, this research investigates alternative methods of navigation based upon an amalgamation of OMCT and the Walking in Place (WIP) technique.

ORIENTATION MOBILITY CANE TECHNIQUE

When used correctly, the cane searches the ground ahead of each step. It warns the operator of obstacles and drop-offs and informs an operator of what is in front of them.

The correct cane technique, which will provide maximum protection and information, is as follows:

The cane is held with the hand centred in front of the body.

The cane is moved with wrist/finger movement only, with the arm remaining still.

The cane tip is moved in an arc that is about an inch wider than the persons body.

The cane is moved in rhythm with the feet, with the tip always being on the opposite side as the forward foot.

WALKING IN PLACE TECHNIQUE

WHY USE WALKING IN PLACE?

The technique of Walking in Place (WIP) has been shown as an effective means to navigate Virtual Spaces (VS).

It enables individuals to navigate VS without the physical limitations imposed by tracking devices.

It has been shown to improve the level of immersion and spatial orientation of the inhabitant.

Unknown is how WIP might affect these outcomes for those with a VI or Blindness; and ultimately how it might affect the acquisition of spatial knowledge which can then be applied to a PE.

PEDOMETRY WALKING DISTANCE ESTIMATION ALGORITHM

The system uses inertial sensors provided by both the Microsoft Band and the Samsung Gear VR to estimate walking speed.

Typically algorithms for utilising inertial sensors to estimate walking speed can be grouped into three main categories:

Abstraction model

Human gait model

Direct integration

We focus on that latter two however instead of applying assumptions of gate to account for integral error via a standard walking classification; we use a custom classification based on OMCT.

TYPICAL WALKING CLASSIFICATION

OMCT WALKING CLASSIFICATION

THE TECHNOLOGY OF THE MICROSOFT BAND

WHY NOT JUST USE DEDICATED HARDWARE SENSORS?

Typically dedicated mobile step detection and step count solutions have been implemented to maximise the performance life of the hardware.

Updating won’t be in real-time, as the systems generally batch events.

Typically algorithms for utilising inertial sensors to estimate both step count and step detection also work on batch data.

IN THEORY AND “PRACTICE” THIS IS ALL GREAT BUT...

-HOUSTON- ITAG THERE IS A PROBLEM!

BACK TO THE DRAWING BOARD?

Further experiments are being conducted to evaluate, improve, and document the accuracy of the step detection.

These experiments include mixed motion types and additional tracking methods for the purpose of providing truth data

Investigate the potential of using of the system to teach OMCT

Initial/current focus was the design of a custom cane tip with odometry sensing capability.

IMU CANE TIP

9DOF / ODOMETRY / BLUETOOTH

HAPTIC CANE TIP

9DOF / ODOMETRY / BLUETOOTH / FEEDBACK

SIMULATION OF TACTILE PAVING

AND SURFACES

MICROSOFT BAND 2

CHROMECAST DEMOS A FEW LIVE DEMOS IF THE TECHNOLOGY AND TIME ALLOWS – IF NOT COME AND FIND ME AFTER THE TALK!

Dr Steven Battersby Steven.Battersby@ntu.ac.uk

Prof David Brown

David.Brown@ntu.ac.uk

Dr Orly Lahav lahavo@post.tau.ac.il