Body Mass Input

8/3/2019 Body Mass Input

1/13

EXPERI MENT

Par t i c i pant s

To eva lua te the pe r fo rmance o f our sys tem, we rec ru i ted 13par t i c ipan ts (7 female ) f rom the Grea te r Sea t t l e a rea . These

par t i c ipan ts rep resen ted a d ive rse c ross - sec t ion o f po ten t ia l

ages and body types . Ages ranged f rom 20 to 56 (mean38 .3 ) , and computed body mass indexes (BMIs) ranged

f rom 20 .5 (normal ) to 31 .9 (obese ) .

Expe r i me nt a l Condi t i ons

We se lec ted th ree inpu t g roup ings f rom the mul t i tude o f

poss ib le loca t ion combina t ions to t e s t . We be l ieve tha tthese g roup ings , i l lus t ra ted in F igure 7 , a re o f pa r t i cu la r

in te res t wi th respec t to in te r face des ign , and a t the same

t ime , push the l imi t s o f our sens ing capab i l i ty . F rom theseth ree g roup ings , we de r ived f ive d i f fe ren t exper imenta l

cond i t ions , desc r ibed be low.

One se t o f ges tu res we te s ted had pa r t i c ipan ts t app ing on

t he t i p s o f each o f t he i r f i ve f i nge r s ( F i gu r e 6 , F i nge r s ) . The f ingers o f fe r in te res t ing a f fo rdances tha t make them

compel l ing to appropr ia te fo r inpu t . Foremos t , they p rov ide

c lea r ly d i sc re te in te rac t ion po in t s , which a re even a l ready

wel l -named (e .g . , r ing f inger ) . In add i t ion to f ive f inger t ips ,the re a re 14 knuck les ( f ive major , n ine minor ) , which , t aken

toge the r , cou ld o f fe r 19 read i ly iden t i f i ab le inpu t loca t ions

on the f ingers a lone . Second , we have excep t iona l f inger - to f ingerdex te r i ty , a s demons t ra ted when we coun t by tapp ing

on our f ingers . F ina l ly , the f ingers a re l inea r ly o rde red ,

which i s po ten t ia l ly use fu l fo r in te r faces l ike number en t ry ,magni tude con t ro l ( e .g . , vo lume) , and menu se lec t ion .

At the same t ime , f ingers a re among the mos t un i fo rm appendages

on the body , wi th a l l bu t the thumb shar ing a s imi la r

ske le ta l and muscu la r s t ruc tu re . Th is d ras t i ca l ly reducesacous t ic va r ia t ion and makes d i f fe ren t ia t ing among them

di f f icu l t . Addi t iona l ly , acous t ic in fo rmat ion mus t c ross a s

many as f ive ( f inger and wr i s t ) jo in t s to reach the fo rea rm,which fu r the r dampens s igna l s . For th i s exper imenta l cond i t ion ,

we thus dec ided to p lace the sensor a r rays on the fo rea rm,

j us t be lo w the e lb ow.


2/13

Desp i te these d i f f i cu l t i e s , p i lo t exper iments showed measureab le

acous t ic d i f fe rences among f ingers , which we theor izei s p r imar i ly re la ted to f inger l eng th and th ickness , in te rac t ions

wi th the complex s t ruc tu re o f the wr i s t bones , and

var ia t ions in the acous t ic t r ansmiss ion p roper t i e s o f the

musc les ex tend ing f rom the f ingers to the fo rea rm.

Whol e Ar m ( F i v e Loc a t i ons )

Another ges tu re se t inves t iga ted the use o f f ive inpu t loca t ions

on the fo rea rm and hand : a rm, wr i s t , pa lm, thumb and

mi dd l e f i nge r ( F i gu r e 7 , W hol e Ar m) . W e s e l ec t ed t hes e loca t ions fo r two impor tan t r easons . F i r s t , they a re d i s t inc t

and named pa r t s o f t he body ( e .g . , wr i s t ) . Th i s a l l owed

pa r t i c ipan ts to accura te ly t ap these loca t ions wi thou t t r a in ing

or mark ings . Addi t iona l ly , these loca t ions p roved to beacous t ica l ly d i s t inc t dur ing p i lo t ing , wi th the l a rge spa t ia l

sp read o f inpu t po in t s o f fe r ing fu r the r va r ia t ion .

We used these loca t ions in th ree d i f fe ren t cond i t ions . One


3/13

cond i t ion p laced the sensor above the e lbow, whi le ano the r

p laced i t be low. Th is was incorpora ted in to the exper imentto measure the accuracy loss ac ross th i s s ign i f i can t a r t i cu la t ion

po in t ( the e lbow) . Addi t iona l ly , pa r t i c ipan ts repea ted

the lower p lacement cond i t ion in an eyes - f ree con tex t : pa r t i c ipan ts were

to ld to c lose the i r eyes and face fo rward ,bo th fo r t r a in ing and te s t ing . Th is cond i t ion was inc luded to

gauge how wel l use r s cou ld t a rge t on-body inpu t loca t ions

in an eyes - f ree con tex t ( e .g . , d r iv ing) .

For e ar m ( Te n Loc a t i ons )

I n an e f f o r t t o a s s es s t he uppe r bound o f ou r appr oach s s ens ing reso lu t ion , our f i f th and f ina l exper imenta l cond i t ion

us ed t en l oca t i ons on j u s t t he f o r ea r m ( F i gur e 6 , For ea r m) .

No t on ly was th i s a ve ry h igh dens i ty o f inpu t loca t ions

(un l ike the whole -a rm cond i t ion) , bu t i t a l so re l i ed on an inpu t su r face

( the fo rea rm) wi th a h igh degree o f phys ica l un i fo rmi ty (un l ike , e .g . , thehand) . We expec ted tha t these fac to rs would make acous t ic sens ing

d i f f i c u l t . Mo r e o v e r, t h i s l o c a t i o n wa s c o m p e l l i n g d u e t o i t s l a r g e a n d f l a tsu r face a rea , a s we l l a s i t s immedia te access ib i l i ty , bo th v i sua l ly and fo r

f inger inpu t . S imul taneous ly , th i s makes fo r an idea l p ro jec t ion su r face

fo r dynamic in te r faces . To maximize the su r face a rea fo r inpu t , we p lacedthe sensor above the e lbow, l eav ing the en t i re fo rea rm f ree . Ra the r than

naming the inpu t loca t ions , a s was done in the p rev ious ly desc r ibed

cond i t ions , we employed smal l , co lo red

s t i ckers to mark inpu t t a rge t s . Th is was bo th to reduce confus ion(s ince loca t ions on the fo rea rm do no t have common

names) and to inc rease inpu t cons i s tency . As ment ioned

prev ious ly , we be l ieve the fo rea rm i s idea l fo r p ro jec ted

in te r face e lements ; the s t i cke rs se rved as low- tech p laceho ldersfo r p ro jec ted bu t tons .

De s i gn and Se t up

We employed a wi th in -sub jec t s des ign , wi th each pa r t i c ipan t

pe r fo rming ta sks in each o f the f ive cond i t ions in randomizedorder : f ive f ingers wi th sensors be low e lbow; f ive

po in t s on the whole a rm wi th the sensors above the e lbow;

the same po in t s wi th sensors be low the e lbow, bo th s igh ted

and b l ind ; and ten marked po in t s on the fo rea rm wi th thesensors above the e lbow.

Par t i c ipan ts were sea ted in a conven t iona l o f f i ce cha i r , in

f ron t o f a desk top compute r tha t p resen ted s t imul i . For cond i t ionswi th sensors be low the e lbow, we p laced the a rmband

~3cm away f rom the e lbow, wi th one sensor package

n e a r t h e r ad i us and the o the r nea r the u l na . For cond i t ions


4/13

with the sensors above the e lbow, we p laced the a rmband

~7cm above the e lbow, such tha t one sensor package res tedo n t h e b i c e ps . R igh t -handed pa r t i c ipan ts had the a rmband

placed on the l e f t a rm, which a l lowed them to use the i r dominan t

hand fo r f inger inpu t . For the one l e f t -handed pa r t i c ipan t ,

we f l ipped the se tup , which had no apparen t e f fec t onthe opera t ion o f the sys tem. T igh tness o f the a rmband was

ad jus ted to be f i rm, bu t comfor tab le . Whi le pe r fo rming

tasks , pa r t i c ipan ts cou ld p lace the i r e lbow on the desk ,

t ucked aga i ns t t he i r body , o r on t he cha i r s ad j us t ab le a rmres t ;

mos t chose the l a t t e r .

PROCEDURE

For each cond i t ion , the exper imente r wa lked th rough the

inpu t loca t ions to be t e s ted and demons t ra ted f inger t aps oneach . Pa r t i c ipan ts p rac t iced dup l ica t ing these mot ions fo r

approx imate ly one minu te wi th each ges tu re se t . Th is a l lowedpar t i c ipan ts to fami l ia r ize themse lves wi th our naming

conven t i ons ( e .g . p i nky , wr i s t ) , and t o p r ac t i ce t app i ng

the i r a rm and hands wi th a f inger on the oppos i te hand .I t a l so a l lowed us to convey the appropr ia te t ap fo rce to

par t i c ipan ts , who o f ten in i t i a l ly t apped unnecessa r i ly ha rd .

To t ra in the sys tem, pa r t i c ipan ts were ins t ruc ted to comfor tab ly

tap each loca t ion ten t imes , wi th a f inger o f the i rchoos ing . Th is cons t i tu ted one t ra in ing round . In to ta l , th ree

rounds o f t r a in ing da ta were co l lec ted pe r inpu t loca t ion se t

(30 examples pe r loca t ion , 150 da ta po in t s to ta l ) . An excep t ion

to th i s p rocedure was in the case o f the t en fo rea rmloca t ions , where on ly two rounds were co l lec ted to save

t ime (20 examples pe r loca t ion , 200 da ta po in t s to ta l ) . To ta l

t r a in ing t ime fo r each exper imenta l cond i t ion was approx imate lyth ree minu tes .

We used the t r a in ing da ta to bu i ld an SVM c lass i f i e r . Dur ing

the subsequen t t e s t ing phase , we p resen ted pa r t i c ipan tswi t h s i mpl e t ex t s t i mu l i ( e . g . t ap your wr i s t ) , wh i ch i n s t r uc t ed

them where to t ap . The o rder o f s t imul i was randomized ,

wi th each loca t ion appear ing ten t imes in to ta l .

The sys tem per fo rmed rea l - t ime segmenta t ion and c lass i f i ca t ion ,and p rov ided immedia te feedback to the pa r t i c ipan t

( e .g . you t apped your wr i s t ) . W e p r ov i ded f eedback s o

tha t pa r t i c ipan ts cou ld see where the sys tem was makinger ro rs (a s they would i f us ing a rea l app l ica t ion) . I f an inpu t

was no t segmented ( i .e . the t ap was too qu ie t ) , pa r t i c ipan ts

cou ld see th i s and would s imply t ap aga in . Overa l l ,


5/13

menta t ion e r ro r ra tes were neg l ig ib le in a l l cond i t ions , and

no t inc luded in fu r the r ana lys i s .

Fi gur e

Fi gur e


6/13

RESULTS

In th i s sec t ion , we repor t on the c lass i f i ca t ion accurac ies fo r

the t e s t phases in the f ive d i f fe ren t cond i t ions . Overa l l ,

c la ss i f i ca t ion ra tes were h igh , wi th an average accuracyacross cond i t ions o f 87 .6%. Addi t iona l ly , we p resen t p re l iminary

resu l t s exp lo r ing the cor re la t ion be tween c lass i f i ca t ion

accuracy and fac to rs such as BMI , age , and sex .

Fi ve F i nge r s

Desp i te mul t ip le jo in t c ross ings and ~40cm of separa t ion

be tween the inpu t t a rge t s and sensors , c la ss i f i ca t ion accuracyremained h igh fo r the f ive - f inger cond i t ion , ave rag ing

87 .7% (SD=10 .0%, chance=20%) ac ross pa r t i c ipan ts . Segmenta t ion , a s ino the r cond i t ions , was essen t ia l ly pe r fec t .

Inspec t ion o f the confus ion mat r ices showed no sys temat ic

e r ro rs in the c lass i f i ca t ion , wi th e r ro rs t end ing to be even lyd is t r ibu ted over the o the r d ig i t s . When c lass i f i ca t ion was

incor rec t , the sys tem be l ieved the inpu t to be an ad jacen t

f inger 60 .5% of the t ime ; on ly marg ina l ly above p r io r p robab i l i ty

(40%) . Th is sugges t s the re a re on ly l imi ted acous t iccon t inu i t i e s be tween the f ingers . The on ly po ten t ia l excep t ion

to th i s was in the case o f the p inky , where the r ing f inger

cons t i tu ted 63 .3% percen t o f the misc lass i f i ca t ions .

Whol e Ar m

Par t i c ipan ts pe r fo rmed th ree cond i t ions wi th the whole -a rmloca t ion conf igura t ion . The be low-e lbow p lacement pe r fo rmed

the bes t , pos t ing a 95 .5% (SD=5.1%, chance=20%)

average accuracy . Th is i s no t su rp r i s ing , a s th i s cond i t ionp laced the sensors c lose r to the inpu t t a rge t s than the o the r

cond i t ions . Moving the sensor above the e lbow reduced

accuracy to 88 .3% (SD=7.8%, chance=20%) , a d rop o f

7 .2%. Th is i s a lmos t ce r ta in ly re la ted to the acous t ic loss a tthe e lbow jo in t and the add i t iona l 10cm of d i s tance be tween

the sensor and inpu t t a rge t s . F igure 8 shows these resu l t s .

The eyes - f ree inpu t cond i t ion y ie lded lower accurac iesthan o the r cond i t ions , ave rag ing 85 .0% (SD=9.4%,

chance=20%) . Th is rep resen t s a 10 .5% drop f rom i t s v i s ionass i s ted , bu t

o the rwise iden t ica l coun te rpa r t cond i t ion . I t


7/13

was apparen t f rom watch ing pa r t i c ipan ts comple te th i s cond i t ion

tha t t a rge t ing p rec i s ion was reduced . In s igh ted cond i t ions ,pa r t i c ipan ts appeared to be ab le to t ap loca t ions wi th pe rhaps a 2cm

rad ius o f e r ro r . Al though no t fo rmal ly cap tu red , th i s marg in o f e r ro r

appeared to doub le o r t r ip le when the eyes were c losed . We be l ieve tha t

add i t iona l t r a in ing da ta , which be t te r covers the inc reased inpu tvar iab i l i ty , would remove much o f th i s de f ic i t . We would a l so cau t ion

des igners deve lop ing eyes - f ree , on-body in te r faces to ca re fu l ly cons ide r

the loca t ions pa r t i c ipan ts can tap accura te ly .

For e ar m

Class i f i ca t ion accuracy fo r the t en - loca t ion fo rea rm cond i t ions tood a t 81 .5% (SD=10 .5%, chance=10%) , a su rp r i s ing ly

s t rong resu l t fo r an inpu t se t we dev ised to push our

s ys t em s s ens i ng l i m i t ( K=0 .72 , cons i de r ed ve r y s t r ong) .

Fo l lowing the exper iment , we cons ide red d i f fe ren t ways to

improve accuracy by co l laps ing the t en loca t ions in to l a rge rinpu t g roup ings . The goa l o f th i s exe rc i se was to exp lo re

the t r adeof f be tween c lass i f i ca t ion accuracy and number o f inpu t loca t ions on the fo rea rm, which represen t s a pa r t i cu la r ly

va luab le inpu t su r face fo r app l ica t ion des igners . We grouped ta rge t s in to

se t s based on wha t we be l ieved to be log ica l spa t ia l g roup ings (F igure 9 ,A-E and G) . In add i t ion to exp lo r ing c lass i f i ca t ion accurac ies fo r l ayou ts

tha t we cons ide red to be in tu i t ive , we a l so pe r fo rmed an exhaus t ive

sea rch (p rogrammat ica l ly ) over a l l poss ib le g roup ings . For mos t loca t ion

coun ts , th i s sea rch conf i rmed tha t our in tu i t ive g roup ings were op t imal ;however , th i s sea rch revea led one p laus ib le , a l though i r regu la r , l ayou t

wi th h igh accuracy a t s ix inpu t loca t ions (F igure 9 , F ) .

Unl ike in the f ive - f ingers cond i t ion , the re appeared to be

shared acous t ic t r a i t s tha t l ed to a h igher l ike l ihood o f confus ionwi th ad jacen t t a rge t s than d i s tan t ones . Th is e f fec t

was more p rominen t l a te ra l ly than long i tud ina l ly . F igure 9

i l lus t ra tes th i s wi th l a te ra l g roup ings cons i s ten t ly ou tpe r fo rmings imi la r ly a r ranged , long i tud ina l g roup ings (B and C vs . D and E) . Th is i s

unsurpr i s ing g iven the morpho logy o f the a rm, wi th a h igh degree o f

b i l a te ra l symmet ry a long the long ax i s .

BMI Ef f e c t s

Ear ly on , we suspec ted tha t our acous t ic approach was suscep t ib le tovar ia t ions in body compos i t ion . Th is inc luded ,

mos t no tab ly , the p reva lence o f fa t ty t i s sues and the dens i ty /

mass o f bones . These , r e spec t ive ly , t end to dampen o r fac i l i t a te thet ransmiss ion o f acous t ic energy in the body . To assess how these

var ia t ions a f fec ted our sens ing accuracy , we ca l cu l a t ed each pa r t i c i pan t s

body mass index (BMI) f rom se l f - repor ted we igh t and he igh t . Da ta and


8/13

observa t ions f rom the exper iment sugges t tha t h igh BMI i s co r re la ted wi th

decreased accurac ies . The pa r t i c ipan ts wi th the th ree h ighes t BMIs (29 .2 ,29 .6 , and 31 .9 represen t ing border l ine obese to obese ) p roduced the

th ree lowes t ave rage accurac ies . F igure 10 i l lus t ra tes th i s s ign i f i can t

d i spa r i ty - he re pa r t i c ipan ts a re separa ted in to two g roups , those wi th

BMI grea te r and le s s than the US na t iona l med ian , age andsex ad jus ted [5 ] (F1 ,12=8 .65 , p=.013) .

Other fac to rs such as age and sex , which may be cor re la ted

to BMI in spec i f i c popu la t ions , migh t a l so exh ib i t a co r re la t ionwi th c lass i f i ca t ion accuracy . For example , in our pa r t i c ipan t

poo l , ma les y ie lded h igher c lass i f i ca t ion accurac ies

than females , bu t we expec t tha t th i s i s an a r t i f ac t o f BMIcor re la t ion in our sample , and p robab ly no t an e f fec t o f sex

d i rec t ly .

SUPPLEMENTAL EXPERI MENTS

We conduc ted a se r ie s o f smal le r , t a rge ted exper iments to

exp lo re the feas ib i l i ty o f our approach fo r o the r app l ica t ions .In the f i r s t add i t iona l exper iment , which te s ted pe r fo rmance

of the sys tem whi le use r s wa lked and jogged , we rec ru i ted one male (age

23) and one female (age 26) fo r a s ing le -purpose exper iment . For the res to f the exper iments , we rec ru i ted seven new par t i c ipan ts (3 female , mean

age 26 .9 ) f rom wi th in our ins t i tu t ion . In a l l cases , the sensor a rmband

wa s p l a c e d j u s t b e l o w t h e e l b o w. S i m i l a r t o t h e p r e v i o u s e x p e ri m e n t , e a c h

add i t iona l exper iment cons i s ted o f a t r a in ing phase , where pa r t i c ipan tsp rov ided be tween 10 and 20 examples fo r each inpu t type , and a t e s t ing

phase , in which pa r t i c ipan ts were p rompted to p rov ide a pa r t i cu la r inpu t

( ten t imes pe r inpu t type) . As be fo re , inpu t o rde r was randomized ;

segmenta t ion and c lass i f i ca t ion were pe r fo rmed in rea l - t ime .

Wal ki ng and Jogg i ng

As d i scussed p rev ious ly , acous t ica l ly -d r iven inpu t t echn iques

a re o f ten sens i t ive to env i ronmenta l no i se . In regardto b io -acous t ic sens ing , wi th sensors coup led to the body ,

no ise c rea ted dur ing o the r mot ions i s pa r t i cu la r ly t roub lesome,

and wa lk ing and jogg ing represen t pe rhaps the mos t

common types o f whole -body mot ion . Th is exper imentexp lo red the accuracy o f our sys tem in these scenar ios .

Each pa r t i c ipan t t r a ined and te s ted the sys tem whi le wa lk ing

and jogg ing on a t r eadmi l l . Three inpu t loca t ions wereused to eva lua te accuracy : a rm, wr i s t , and pa lm. Addi t iona l ly ,

the ra te o f f a l se pos i t ives ( i . e . , the sys tem be l ieved

the re was inpu t when in fac t the re was no t ) and t rue pos i t ives


9/13

( i . e . , the sys tem was ab le to co r rec t ly segment an in tended

inpu t ) was cap tu red . The te s t ing phase took rough lyth ree minu tes to comple te ( four t r i a l s to ta l : two pa r t i c ipan ts ,

two cond i t ions ) . The male wa lked a t 2 .3 mph and jogged a t

4 .3 mph; the female a t 1 .9 and 3 .1 mph , re spec t i ve ly .

In bo th wa lk ing t r i a l s , the sys tem never p roduced a fa l sepos i t iveinpu t . Meanwhi le , t rue pos i t ive accuracy was

100%. Class i f i ca t ion accuracy fo r the inpu ts (e .g . , a wr i s t

t ap was recogn ized as a wr i s t t ap ) was 100% for the maleand 86 .7% for the female (chance=33%) . In the jogg ing t r i a l s , the sys tem

had four fa l se -pos i t ive inpu t even ts ( two per pa r t i c ipan t ) over s ix minu tes

of con t inuous jogg ing . True-pos i t ive accuracy , a s wi th wa lk ing , was100%. Cons ide r ing tha t jogg ing i s pe rhaps the ha rdes t inpu t f i l t e r ing and

segmenta t ion t e s t , we v iew th i s r e su l t a s ex t remely pos i t ive .

Class i f i ca t ion accuracy , however , dec reased to 83 .3% and 60 .0% for the

male and female pa r t i c ipan ts re spec t ive ly (chance=33%) .

Al though the no i se genera ted f rom the jogg ing a lmos t ce r ta in lydegraded the s igna l ( and in tu rn , lowered c lass i f i ca t ion

accuracy) , we be l ieve the ch ie f cause fo r th i s dec reasewas the qua l i ty o f the t r a in ing da ta . Pa r t i c ipan ts on ly p rov ided

ten examples fo r each o f th ree t e s ted inpu t loca t ions .

Fur the rmore , the t r a in ing examples were co l lec ted whi lepar t i c ipan ts were jogg ing . Thus , the resu l t ing t ra in ing da ta

was no t on ly h igh ly va r iab le , bu t a l so spa rse ne i the r o f

which i s conduc ive to accura te mach ine l ea rn ing c lass i f i ca t ion .

We be l ieve tha t more r igorous co l lec t ion o f t r a in ingda ta cou ld y ie ld even s t ronger re su l t s .

Si ng l e - Hande d Ge s t ur e s

In the exper iments d i scussed thus fa r , we cons ide red on lyb imanua l ges tu res , where the sensor - f ree a rm, and in pa r t i cu la r

the f ingers , a re used to p rov ide inpu t . However , the re a re a range o f

ges tu res tha t can be pe r fo rmed wi th jus t the f ingers o f one hand . Th is wasthe focus o f [2 ] , a l though th i s work d id no t eva lua te c lass i f i ca t ion

accuracy . We conduc ted th ree independen t t e s t s to exp lo re onehanded

ges tu res . The f i r s t had pa r t i c ipan ts t ap the i r index , midd le , r ing and p inky

f ingers aga ins t the i r thumb (ak in to a p inch ing ges tu re ) t en t imes each .Our sys tem was ab le to iden t i fy the four inpu t types wi th an overa l l

accuracy o f

89 .6% (SD=5.1%, chance=25%) . We ran an iden t ica l exper imentus ing f l i cks ins tead o f t aps ( i . e . , u s ing the thumb as a

ca tch , then rap id ly f l i ck ing the f ingers fo rward) . Th is

y ie lded an impress ive 96 .8% (SD=3.1%, chance=25%) accuracy


10/13

in the t e s t ing phase .

Th is mot iva ted us to run a th i rd and independen t exper imenttha t combined taps and f l i cks in to a s ing le ges tu re se t .

Pa r t i c ipan ts re - t ra ined the sys tem, and comple ted an independen t

tes t ing round . Even wi th e igh t inpu t c la sses in ve ry

c lose spa t ia l p rox imi ty , the sys tem was ab le to ach ieve aremarkab le 87 .3% (SD=4.8%, chance=12 .5%) accuracy .

Th is re su l t i s comparab le to the a fo rement ioned ten loca t ion

forea rm exper iment (which ach ieved 81 .5% accuracy) ,l end ing c redence to the poss ib i l i ty o f hav ing ten o r

more func t ions on the hand a lone . Fur the rmore , p ropr iocep t ion

of our f ingers on a s ing le hand i s qu i te accura te , sugges t inga mechan ism fo r h igh-accuracy , eyes - f ree inpu t .

Sur f ac e and Obj e c t Re c ogni t i on

Dur ing p i lo t ing , i t became apparen t tha t our sys tem had

some ab i l i ty to iden t i fy the type o f ma te r ia l on which theuse r was opera t ing . Us ing a s imi la r se tup to the main exper iment ,

we asked pa r t i c ipan ts to t ap the i r index f inger

aga ins t 1 ) a f inger on the i r o the r hand , 2 ) a paper pad approx imate ly 80pages th ick , and 3 ) an LCD sc reen . Resu l t s

show tha t we can iden t i fy the con tac ted ob jec t wi th abou t

87 .1% (SD=8.3%, chance=33%) accuracy . Th is capab i l i ty

was never cons ide red when des ign ing the sys tem, so super io racous t ic fea tu res may ex i s t . Even as accuracy s tands

now, the re a re severa l in te res t ing app l ica t ions tha t cou ld

take advan tage o f th i s func t iona l i ty , inc lud ing works ta t ions

or dev ices composed o f d i f fe ren t in te rac t ive su r faces , o rrecogn i t ion o f d i f fe ren t ob jec t s g rasped in the env i ronment .

I de nt i f i c a t i on o f F i nge r Tap Type

Us er s can t ap s u r f aces w i t h t he i r f i nge r s i n s eve r a l d i s t i nc t ways . For example , one can use the t ip o f the i r f inger

(po ten t ia l ly even the i r f inger na i l ) o r the pad ( f l a t , bo t tom)

of the i r f inger . The fo rmer t ends to be qu i te boney , whi le

the l a t t e r more f l e shy . I t i s a l s o poss ib le to use the knuck les(bo th major and minor me t ac ar popha l ange a l jo in t s ) .

To eva l ua t e ou r appr oach s ab i l i t y t o d i s t i ngu i s h these inpu t

types , we had pa r t i c ipan ts t ap on a t ab le s i tua ted in f ron t o f them in th ree ways ( t en t imes each) : f inger t ip , f inger pad ,

and major knuck le . A c lass i f i e r t r a ined on th i s da ta y ie lded

an average accuracy o f 89 .5% (SD=4.7%, chance=33%)


11/13

dur ing the t e s t ing pe r iod . Th is ab i l i ty has severa l po ten t ia l uses . Pe rhaps

the mos t no tab le i s the ab i l i ty fo r in te rac t ive touch su r faces to d i s t ingu ishd i f fe ren t types o f f inger con tac t s (which a re ind i s t ingu ishab le in e .g . ,

capac i t ive and v i s ion-based sys tems) . One example in te rac t ion cou ld be

t ha t doub l e - knock i ng on an i t em opens i t , wh i l e a pad - t ap ac t i va t e s

an op t ions menu .

Se gme nt i ng F i nge r I nput

A pragmat ic concern regard ing the appropr ia t ion o f f inger t ips

fo r inpu t was tha t o the r rou t ine t a sks would genera tefa l se pos i t ives . For example , typ ing on a keyboard s t r ikes

the f inger t ips in a ve ry s imi la r manner to the f inger - t ip inpu t

we p roposed p rev ious ly . Thus , we se t ou t to exp lo re

whe ther f inger - to - f inger inpu t sounded su f f ic ien t ly d i s t inc t

such tha t o the r ac t ions cou ld be d i s regarded .As an in i t i a l a s sessment , we asked pa r t i c ipan ts to t ap the i r

index f inger 20 t imes wi th a f inger on the i r o the r hand , and20 t imes on the su r face o f a t ab le in f ron t o f them. Th is da ta

was used to t r a in our c lass i f i e r . Th is t r a in ing phase was

fo l lowed by a t e s t ing phase , which y ie lded a pa r t i c ipan twideaverage accuracy o f 94 .3% (SD=4.5%, chance=50%) .

EXAMPLE I NTERFACES AND I NTERACTI ONS

We conce ived and bu i l t severa l p ro to type in te r faces tha t

demons t ra te our ab i l i ty to appropr ia te the human body , in

th i s case the a rm, and use i t a s an in te rac t ive su r face . These

in te r faces can be seen in F igure 11 , a s we l l a s in the accompanying v ideo .Whi le the b io -acous t ic inpu t moda l i ty i s no t s t r i c t ly t e the red

to a pa r t i cu la r ou tpu t moda l i ty , we be l ieve the sensor fo rm

fac to rs we exp lo red cou ld be read i ly coup led wi th v i sua lou tpu t p rov ided by an in tegra ted p ico-pro jec to r . There a re

two n ice p roper t i e s o f wear ing such a p ro jec t ion dev ice on

the a rm tha t pe rmi t us to s ides tep many ca l ib ra t ion i s sues .F i r s t , the a rm i s a re la t ive ly r ig id s t ruc tu re - the p ro jec to r ,

when a t t ached appropr ia te ly , wi l l na tu ra l ly t r ack wi th the

a rm ( see v ideo) . Second , s ince we have f ine -g ra ined con t ro l

o f the a rm, making minu te ad jus tments to a l ign the p ro jec tedimage wi th the a rm i s t r iv ia l ( e .g . , p ro jec ted hor izon ta l

s t r ipes fo r a l ignment wi th the wr i s t and e lbow) .

To i l lus t ra te the u t i l i ty o f coup l ing p ro jec t ion and f ingerinpu t on the body (as re sea rchers have p roposed to do wi th

pro jec t ion and compute r v i s ion-based techn iques [19] ) , we

deve loped th ree p roof -o f -concep t p ro jec ted in te r faces bu i l t


12/13

on t op o f ou r s ys t em s l i ve i npu t c l a s s i f i ca t i on . I n t he f i r s t

in te r face , we p ro jec t a se r ie s o f bu t tons on to the fo rea rm, onwhich a use r can f inger t ap to nav iga te a h ie r a rch ica l menu

(F igure 11 , l e f t ) . In the second in te r face , we p ro jec t a sc ro l l ing

menu (cen te r ) , which a use r can nav iga te by tapp ing a t

the top o r bo t tom to sc ro l l up and down one i t em respec t ive ly .Tapp ing on the se lec ted i t em ac t iva tes i t . In a th i rd

i n t e r f ace , we p r o j ec t a numer i c ke ypad on a us e r s pa l m and

a l low them to t ap on the pa lm to , e .g . , d ia l a phone number( r igh t ) . To emphas ize the ou tpu t f l ex ib i l i ty o f approach , we

a l so coup led our b io -acous t ic inpu t to aud io ou tpu t . In th i s

case , the use r t aps on p rese t loca t ions on the i r fo rea rm andhand to nav iga te and in te rac t wi th an aud io in te r face .

F UT UR E

f I GUR E

The sk inpu t t echno logy has been thus fa r success fu l ly implemented fo rthe game of Te t r i s ( see f ig ) and the comple te con t ro l o f the i -pod .The

average success ra te found i s a round 96%.Ef fo r t s a re on to ra i se th i s

f igure to a s c lose to 100 as poss ib le .accord ing to the o f f ic ia l word , the

Sk inpu t t echno logy shou ld be l aunched in the marke t wi th in the nex t 2 -7years . The combina t ion o f these two leaves p len ty o f room fo rdeve lopment o f d i f fe ren t app l ica t ions , o r , i f we ' re inc red ib ly lucky ,

games .I pe r sona l ly am look ing fo rward to a fu tu re where a l l those coo l spy

games wi th a rm bands tha t do amaz ing th ings a re l i t e ra l ly on my own a rm.

I t wi l l f ee l ve ry James Bond l ike ! ! .


13/13

CONCLUSI ON

In th i s paper , we have p resen ted our approach to appropr ia t ing the

human body as an inpu t su r face . We have desc r ibed a nove l , wearab le b io -acous t ic sens ing a r ray tha t we bu i l t in to an a rmband in o rde r to de tec t and

loca l ize f inger t aps

on the fo rea rm and hand . Resu l t s f rom our exper imentshave shown tha t our sys tem per fo rms ve ry we l l fo r a se r ie s

of ges tu res , even when the body i s in mot ion . Addi t iona l ly ,

we have p resen ted in i t i a l r e su l t s demons t ra t ing o the r po ten t ia luses o f our approach , which we hope to fu r the r exp lo re

in fu tu re work . These inc lude s ing le -handed ges tu res , t aps

wi th d i f fe ren t pa r t s o f the f inger , and d i f fe ren t ia t ing be tween

mate r ia l s and ob jec t s . We conc lude wi th desc r ip t ions

of severa l p ro to type app l ica t ions tha t demons t ra te the r i chdes ign space we be l ieve Sk i npu t enab les .

Documents

Body Mass Input