6
Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance Eric Kukula & Stephen Elliott, Ph.D. [email protected] & [email protected] Biometric Standards, Performance, & Assurance Laboratory College of Technology, Purdue University, 401 N. Grant Street, West Lafayette, IN 47906 USA ABSTRACT This paper discusses the implementation issues of installing a commercially available hand geometry system in the current infrastructure of Purdue University's Recreational Sports Center. In addition to a performance analysis of the system, a pre- and post- data collection survey was distributed to the 129 test subjects gathering information on perceptions of biometrics, in particular hand geometry, as well as participants' thoughts and feelings during their interaction with the hand geometry device. The results of the survey suggest that participants were accepting of hand geometry. Additionally, analyses of the participants' survey responses revealed that 93% liked using hand geometry, 98% thought it was easy to use, and 87% preferred it to the existing card-based system, while nobody thought the device invaded their personal privacy. System performance achieved a 3-try match rate of 99.02% (FRR 0.98%) when "gaming"/potential impostor attempts were removed from analysis. The failure to enroll rate was zero. Statistical analyses exposed a significant difference in the scores of attempts made by users with prior hand geometry usage, and subjects that could not straighten out their hand on the device. However, there were no statistical difference in the effects of rings/no rings, improper hand placements/proper hand placements, or gender on hand geometry score. 1. INTRODUCTION This paper explores the feasibility and performance related to implementing a hand geometry system for access and audit control at Purdue University's Recreational Sports Center (RSC). The current access control system requires a user to swipe a student identification card through a magnetic strip reader to gain access to the recreational center. Since the decision is based solely on a token, which can be stolen, lost, handed to others, or copied; accurate auditing is not possible, causing a two fold problem: (1) an insurance risk - as accurate records of members in the facility are not available, and (2) a loss of revenue for the recreational center - as multiple people could use the same identification card. The use of biometrics, particularly hand geometry, removes the token (something members "have") and replaces or adds to it a physical or behavioral trait (something members "are"), which removes the possibility for lost or stolen tokens to be used for fraudulent access. This paper describes a modified operational evaluation of Recognition Systems Handkey II hand geometry system at Purdue University's Recreational Sports Center (RSC) during the Spring semester of 2005. The project began in Fall 2003 as a class project for IT 545, a graduate level biometrics course in the College of Technology, with the purpose of assessing the facility's environment and infrastructure, as well as the management team and potential users [1]. Initially, a survey of 453 participants was distributed and analyzed during the Fall of 2003 to determine RSC usage patterns, perceptions of biometrics, and thoughts about the current access control system. The results revealed that 55% of the participants felt fingerprint recognition was least intrusive, followed by hand geometry (30%), eye - iris and retinal imaging (7%), and face (6%), which is shown in Figure 1. [1] also conducted an environmental assessment of the Purdue RSC, which investigated four biometric applications based on assumed user familiarity, ease of use, and climate conditions at the two entrance locations - the front gate located indoors, and the back gate located outdoors. No Multiple Response - 1% Moda ities - 1% I Hand -30% 7TA. Figure 1: 2003 Biometric gym access survey - Responses for the least intrusive biometric technology [1] The recommendations by the IT545 class to RSC Management were to install hand geometry due to its track record for access control applications (AC) and use in outdoor environments. It was determined by RSC Management that Purdue University Facilities Services would install the hand geometry device separate from the current access control system, so testing and evaluation would not interfere with students and faculty attempting to gain access to the facility. Further discussion on the environment and system setup occurs later in this paper. 2. MOTIVATION The motivation behind the research was four-fold: 1) To collect prior biometric use information, as well as participants perceptions of biometrics before using the hand geometry system, to establish implementation feasibility and user acceptance of the technology; 2) To monitor participants initial perceptions and interactions during training and enrollment; 3) To collect user perception and interaction data after the evaluation concluded, and 4) To measure the performance of the 0-7803-9245-O/05/$20.00 C2005 IEEE Authorized licensed use limited to: Purdue University. Downloaded on February 28,2010 at 16:52:06 EST from IEEE Xplore. Restrictions apply.

(2005) Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance

Embed Size (px)

DESCRIPTION

This paper discusses the implementation issues of installing a commercially available hand geometry system in the current infrastructure of Purdue University's Recreational Sports Center. In addition to a performance analysis of the system, a pre- and post- data collection survey was distributed to the 129 test subjects gathering information on perceptions of biometrics, in particular hand geometry, as well as participants' thoughts and feelings during their interaction with the hand geometry device. The results of the survey suggest that participants were accepting of hand geometry. Additionally, analyses of the participants' survey responses revealed that 93% liked using hand geometry, 98% thought it was easy to use, and 87% preferred it to the existing card-based system, while nobody thought the device invaded their personal privacy. System performance achieved a 3-try match rate of 99.02% (FRR 0.98%) when "gaming"/potential impostor attempts were removed from analysis. The failure to enroll rate was zero. Statistical analyses exposed a significant difference in the scores of attempts made by users with prior hand geometry usage, and subjects that could not straighten out their hand on the device. However, there were no statistical difference in the effects of rings/no rings, improper hand placements/proper hand placements, or gender on hand geometry score.

Citation preview

Page 1: (2005) Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance

Implementation of Hand Geometry at Purdue University's Recreational Center:An Analysis of User Perspectives and System Performance

Eric Kukula & Stephen Elliott, [email protected] & [email protected]

Biometric Standards, Performance, & Assurance LaboratoryCollege ofTechnology, Purdue University, 401 N. Grant Street, West Lafayette, IN 47906 USA

ABSTRACTThis paper discusses the implementation issues of installing acommercially available hand geometry system in the currentinfrastructure of Purdue University's Recreational Sports Center.In addition to a performance analysis of the system, a pre- andpost- data collection survey was distributed to the 129 testsubjects gathering information on perceptions of biometrics, inparticular hand geometry, as well as participants' thoughts andfeelings during their interaction with the hand geometry device.The results of the survey suggest that participants wereaccepting of hand geometry. Additionally, analyses of theparticipants' survey responses revealed that 93% liked usinghand geometry, 98% thought it was easy to use, and 87%preferred it to the existing card-based system, while nobodythought the device invaded their personal privacy. Systemperformance achieved a 3-try match rate of 99.02% (FRR0.98%) when "gaming"/potential impostor attempts wereremoved from analysis. The failure to enroll rate was zero.Statistical analyses exposed a significant difference in the scoresof attempts made by users with prior hand geometry usage, andsubjects that could not straighten out their hand on the device.However, there were no statistical difference in the effects ofrings/no rings, improper hand placements/proper handplacements, or gender on hand geometry score.

1. INTRODUCTIONThis paper explores the feasibility and performance related toimplementing a hand geometry system for access and auditcontrol at Purdue University's Recreational Sports Center(RSC). The current access control system requires a user toswipe a student identification card through a magnetic stripreader to gain access to the recreational center. Since thedecision is based solely on a token, which can be stolen, lost,handed to others, or copied; accurate auditing is not possible,causing a two fold problem: (1) an insurance risk - as accuraterecords of members in the facility are not available, and (2) aloss of revenue for the recreational center - as multiple peoplecould use the same identification card. The use of biometrics,particularly hand geometry, removes the token (somethingmembers "have") and replaces or adds to it a physical orbehavioral trait (something members "are"), which removes thepossibility for lost or stolen tokens to be used for fraudulentaccess.

This paper describes a modified operational evaluation ofRecognition Systems Handkey II hand geometry system atPurdue University's Recreational Sports Center (RSC) during

the Spring semester of 2005. The project began in Fall 2003 as aclass project for IT 545, a graduate level biometrics course in theCollege of Technology, with the purpose of assessing thefacility's environment and infrastructure, as well as themanagement team and potential users [1]. Initially, a survey of453 participants was distributed and analyzed during the Fall of2003 to determine RSC usage patterns, perceptions ofbiometrics, and thoughts about the current access control system.The results revealed that 55% of the participants felt fingerprintrecognition was least intrusive, followed by hand geometry(30%), eye - iris and retinal imaging (7%), and face (6%), whichis shown in Figure 1. [1] also conducted an environmentalassessment of the Purdue RSC, which investigated fourbiometric applications based on assumed user familiarity, easeof use, and climate conditions at the two entrance locations - thefront gate located indoors, and the back gate located outdoors.

NoMultiple Response - 1%

Moda ities - 1% IHand -30%

7TA.

Figure 1: 2003 Biometric gym access survey - Responses forthe least intrusive biometric technology [1]

The recommendations by the IT545 class to RSC Managementwere to install hand geometry due to its track record for accesscontrol applications (AC) and use in outdoor environments. Itwas determined by RSC Management that Purdue UniversityFacilities Services would install the hand geometry deviceseparate from the current access control system, so testing andevaluation would not interfere with students and facultyattempting to gain access to the facility. Further discussion onthe environment and system setup occurs later in this paper.

2. MOTIVATIONThe motivation behind the research was four-fold: 1) To collectprior biometric use information, as well as participantsperceptions of biometrics before using the hand geometrysystem, to establish implementation feasibility and useracceptance of the technology; 2) To monitor participants initialperceptions and interactions during training and enrollment; 3)To collect user perception and interaction data after theevaluation concluded, and 4) To measure the performance of the

0-7803-9245-O/05/$20.00 C2005 IEEE

Authorized licensed use limited to: Purdue University. Downloaded on February 28,2010 at 16:52:06 EST from IEEE Xplore. Restrictions apply.

Page 2: (2005) Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance

hand geometry system, including: the failure to enroll rate(FTE), failure to acquire rate (FTA), and the false reject rate(FRR).

3. ENVIRONMENT OVERVIEWThe hand geometry system was installed in the front entrance ofPurdue University's RSC. The implementation was isolatedfrom the current access control system (magnetic ID cards), so itwould not interfere with members trying to enter the RSC. Inaddition, the current design of the access control area was notsuitable to accommodate the commercially off the shelf (COTS)hand geometry device without significantly affectingperformance, which will be discussed in section 3.2. Theenvironment, which the hand geometry system was installed, isshown in Figure 2.

Figure 2: Purdue RSC access control area

3.1 Characteristics of Current SystemThe current access control system at the Purdue University RSCuses magnetic strip readers mounted on top of a turnstile. Togain access to the facility, a member swipes their identificationcard through the magnetic strip reader. If the card is active, theturnstile will unlock and the member can gain access. Thecurrent system involving the magnetic card readers has twolanes or stations. One of the stations is depicted on the left sideof Figure 3.

Figure 3: Experimental setup

3.2. Characteristics of Operational System EvaluatedFigure 3 depicts the hand geometry unit placement in the frontentranceway. It was noted by the authors that by separating thehand geometry device from the turnstile, participant interactionwith the hand geometry device would not be identical to that ofa real-world deployment. There are two significant explanationsfor this. First, the COTS hand geometry unit was a right-handedmodel and the base of the turnstile was located on the left.

Therefore if a user were to use the hand geometry device, theywould have to reach across their body to use the device,influencing users' interaction with the device. The results of thisincorrect setup could possibly cause inconsistent handplacements on the device ultimately affecting the performance ofthe algorithm, which in turn could influence users' perceptionsof hand geometry. Installation was performed by Purdue'sFacility Services, which closely followed the protocol developedin the Fall 2003 feasibility study.

3.2.1 Scenario Testing in an Operational EnvironmentThe primary goal of the hand geometry implementation was toexamine the performance of the hand geometry system at thePurdue RSC on a population of students and faculty. This goal isharmonious with the ISO/IEC Biometric Performance Testingand Reporting definition that states operational testing is an"evaluation in which the performance of a complete biometricsystem is determined in a specific application environment witha specific target population [2]. However the end result of a userinteracting with the hand geometry system did not affect thementering the facility, thus was not a true operational test; and wasclassified as a modified scenario test.

3.2.2 Operational Environment during TestingThe operational environment shown in Figure 2 had a meantemperature in the evaluation area of roughly 74°F. Since Purdueis located in Indiana and the test was conducted betweenFebruary and May, outdoor temperatures were variablethroughout the evaluation with a mean high of 65°F and low of43°F [3], however quantitative measures on the effects oftemperature changes on hand geometry scores were not recordedas the evaluation type was unattended. Illumination levels werenot monitored during the evaluation, but were consistent with abuilding entrance with glass doors. Quantitative measures on theeffects of illumination on the performance of the hand geometrysystem were also outside the scope of this evaluation.

3.3 Future StateThe RSC has plans to remodel the front entrance within the nexttwo years. Regardless of whether or not hand geometry isadopted and implemented system wide, the re-design shouldinclude a reversal of the turnstiles to accommodate COTS handgeometry readers allowing members to walk up to the turnstile,stop in front of the barrier, type their ID number or swipe theirID card, place their hand on the platen, and proceed. Figure 4depicts an entranceway modeled in this fashion at anotheruniversity recreational sport facility in the United States.

Figure 4: Future hand geometry access control system [5]

Authorized licensed use limited to: Purdue University. Downloaded on February 28,2010 at 16:52:06 EST from IEEE Xplore. Restrictions apply.

Page 3: (2005) Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance

4. TECHNOLOGY OVERVIEWHand geometry is one of the oldest commercialized biometricmodalities, with foundation patent applications dating from thelate 1960s [4]. The technology came of age in the 1980s when itwas selected for deployment in several US nuclear power plants.Since then, over 100,000 devices have been installed forphysical access control (AC), identity management (IM), andtime & attendance (TA) applications. Typical AC and IM sitesinclude airports, prisons, hospitals, military bases, commercialbuildings, dining applications, and health clubs. In TAapplications, hand geometry units replace traditional time clocksto eliminate "buddy punching" (in which one employee signsin/out for another employee so they are paid for more hours thanthey actually worked) and to control/authorize overtimeexpenses.

Hand geometry devices measure the size and shape of humanhands. They do not measure palm prints, fingerprints, veinpatterns, or other traits. The type of hand reader used at the RSCis shown in Figure 5. This device uses a CMOS sensor and nearinfrared illumination to image the hand from above and from theside (the side-view is captured via a mirror adjoining the handsurface). These views are converted to binary silhouettes thatare traversed to calculate over 90 length, width, thickness, andarea measurements. These measurements are "compressed" intoa proprietary 9-byte template in such a way that only the uniquecharacteristics of each hand are stored; a hand image cannot beuniquely reconstructed from its 9-byte template.

Figure 5: Recognition Systems Handkey II

The Handkey II is generally considered a verification device(where the user proves a claim of identity), not an identificationdevice (where the user is selected from a list of possibleidentities). Due to the small 9-byte template, the Handkey II can

store up to 32,516 users in its internal database (without an

external PC). Hand geometry can also be used in smartcardapplications, where memory and processing power are at a

premium. For a more detailed discussion on hand geometry, see

[5, 6].

4.1 Hand Geometry Modifications for Data CollectionIn this evaluation, a COTS hand geometry device was modifiedfor testing purposes by adding a data logging card containing a

flash memory card similar to those used in digital cameras.

After each hand placement, the hand reader's main processor

spent approximately two seconds writing to flash. During thistime, the processor did not respond to network requests,including enrollment or verification requests. For this reason

some users were enrolled at the hand reader, but the PC auditing

software occasionally timed out waiting for the enrollmentnotification. Subsequently, these users were considered invalidby the auditing software and were summarily rejected during theverification attempts following enrollment. Re-enrolling theseusers solved the problem.

Because re-enrollment was triggered by a software timeoutcaused by using a non-standard hand reader, not by imageacquisition or algorithm problems, these cases were notconsidered failures to enroll (FTE). Moreover, enrollment trialswithout the auditing software were conducted offline in alaboratory environment and no enrollment problems wereapparent. See section 7.1 for further enrollment analysis.

5. EXPERIMENTAL SETUPBefore the experimental setup is discussed, the application underwhich this evaluation occurred must be defined. The taxonomyfor an application falls under six categories: cooperative/non-cooperative, covert/overt, habituated/non-habituated, attended/unattended, public/private, and open/closed. For furtherinformation on application taxonomies, refer to [7, 8]. For thisaccess control evaluation, the taxonomy was cooperative, overt,habituated and non-habituated, unattended - except for trainingand enrollment, private, and closed.

As discussed previously, all aspects of this evaluation took placein the front entrance of the Purdue University RSC (Figure 2),including training, enrollment, and unattended verification. Inaddition, two surveys were distributed before and after theevaluation to collect demographic information, RSC usagepatterns, prior knowledge, and/or use of biometrics, perceptionstoward hand geometry, and feelings towards the current accesscontrol system.

5.1 TrainingBefore training, each subject was presented an IRB humansubjects consent form to read and sign. Once this was signedtraining began. Training consisted of a 5-10 minute briefingincluding: the purpose of the project, a technology overview, ahand placement tutorial, instruction on specific enrollment andverification procedures to follow, and a question and answerperiod. For repeatability purposes a manual was created,ensuring that each individual received the same orientation andtraining.

5.2 EnrollmentEnrollment was attended by one of the authors to ensure eachtest subject followed the test protocol for enrollment. Eachsubject was asked to provide a four digit number that they couldremember for subsequent verification attempts. Enrollmentconsisted of three hand placements, to create a unique templatefor that test subject; however the device may have requiredadditional hand placements if the first three did not satisfy theenrollment criteria. It was observed by the authors thatenrollment generally took thirty seconds or less, although thisdata was not collected or documented.

5.3 Unattended VerificationThe COTS hand geometry unit functions as a one-to-one(verification) system. During each verification attempt, the testsubject entered the four digit number provided during

Authorized licensed use limited to: Purdue University. Downloaded on February 28,2010 at 16:52:06 EST from IEEE Xplore. Restrictions apply.

Page 4: (2005) Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance

enrollment, and when prompted by the system - placed theirhand on the platen around the guide pins. Since the handgeometry system was separated from the functioning accesscontrol system, as discussed earlier in the paper, a failedverification attempt did not result in a user not entering thefacility. To enter, a test subject still had to provide her/hisPurdue ID card and follow the access control procedure.Furthermore, as one of the purposes of the evaluation was toexamine participants' interactions and perceptions of the device,they were allowed to use the device repeatedly at theirdiscretion. This resulted in some participants never using thedevice after the first visit, while others used the device once aweek, once per day, and even multiple times per day.

5.4 Survey DistributionTwo electronic surveys were distributed during the evaluation togather demographic information, RSC usage patterns, priorknowledge and/or use of biometrics, feelings toward handgeometry, and feelings towards the current access control systemof the test subjects. The first survey was given after theparticipant enrolled using the hand geometry device. At theconclusion of the test period, each test subject was asked tocomplete a follow-up survey to assess if their interaction withthe device over a two-month period changed their thoughts orperceptions of hand geometry. As the surveys were voluntaryand electronic, response rates varied, which is discussed in thenext section.

6. TEST POPULATION & SURVEY RESULTSThe evaluation involved 129 test subjects from PurdueUniversity, including students, faculty, and full-time RSCemployees. The pre- evaluation survey resulted in a 92%response rate, while the survey distributed after the evaluationended was 42%. The latter survey was available for a monthafter data collection ended, but reduced response rates wereexpected by the authors due to the end of the academic semester,although multiple email messages were sent prior to the surveydistribution, as well as three reminder messages afterdistribution.

6.1 DemographicsFrom the 119 out of 129 respondents, 88% were 18-25 years ofage, with 71% male and 29% female. 88% were students, 8%were Purdue University faculty, and the remaining 3% wereRSC employees. 104 test subjects, or 87%, classified themselvesas right-handed. 87% of the respondents were Caucasian, 9%Asian, 2% African American, and 1% Native American.Moreover, none of the participants had missing digits. Refer to[9] for a complete listing of the demographic survey results.

6.2. Results for Subjects' Prior Use & Perceptions ofBiometricsTest subjects were asked seven questions regarding prior usageand perceptions of biometrics after training and enrollment.From the 119 survey respondents 37% stated they have neverused biometrics, while 42% stated they had used fingerprintrecognition, followed by dynamic signature verification (28%),voice recognition (21%), and hand geometry (18%). From thetest subjects that have previous biometric experience, 34% usedmultiple biometric technologies. A complete analysis of theresponses can be found in [9]. It should be noted that there are

fingerprint ATM's on campus, and near the vicinity of students,which might account for prior usage.

With regards to hand geometry, 94 of the 119 responses fromtest subjects never used hand geometry prior to this study (79%),98% stated that the hand geometry device did not look hard touse, and 92% stated the device did not look invasive to them.However, there were conversations with some subjects duringthe training and enrollment sessions that revealed many commonmisconceptions such as: hand geometry collected fingerprints,the platen scanned their fingerprints, "people or governmentcould steal their fingerprints from the hand reader", and "thatgovernment agencies were collecting their data from the handreader".

6.3 User ExperiencesAfter the data collection concluded, a post data collection surveywas released to all participants. Of the 129 participants, 55responded (42% response rate). The results from the surveydealing with participant experiences can be found in Table 1.

Analysis of the user experiences survey results revealed that85% remembered how to use the hand geometry device eachtime they used it, while 15% forgot the correct procedure or howto place their hand on the device correctly. From therespondents, 71% used the hand geometry device up to 20 timesin a two-month period. When participants did use the device, 19of the 55 respondents performed multiple verification attemptsin the same day; either in succession or over a period of time.More generally, 93% of the survey respondents liked using thedevice, and 98% stated the hand geometry device was easy touse. Furthermore, all 55 survey responses stated they felt thehand geometry device did not invade their privacy or personalspace.

Another interesting point is that 64% of the respondingparticipants showed either their friends or co-workers how thehand geometry device worked. It is hypothesized that duringthese explanations to friends and co-workers that imposterattempts and experimental or "game" hand placements tookplace. While this cannot be proven due to the unattended natureof this test, analysis of the raw images stored with each handplacement strongly supports this theory. Lastly, when askedwhether participants preferred to use hand geometry or magneticstrip cards to access the RSC, 87% responded with handgeometry, 5% responded with the magnetic strip cards, 7% wereundecided, and 1% did not respond.

Table 1: User experiences survey results post data collection

Hw anytimes didyou gse tie handgeometry d_ev?Less than 10 22 40%11-20 times 17 31%21-30 times 7 13%31-40 times 5 9%41-50 times 0 0%More than 50 times 4 7%How mnytimesdidnyouforget how to use the handdgonetrydevice?Never 47 85%1-3 6 11%4-6More than 6 times.

0 0%2 4%

Didyou1 perfrm multple attemts on the hand reade duig thesne visit, either in suesion or once enteriWn andonce leavinNo 36 65%Yes 19 35%

Authorized licensed use limited to: Purdue University. Downloaded on February 28,2010 at 16:52:06 EST from IEEE Xplore. Restrictions apply.

Page 5: (2005) Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance

Didyou ike usig hand gemetry devie.?No 3 5%Yes 51 93%No Response 1 2%Was the hand_geoer device hard to use?No 54 98%Yes 0 0%No Response 1 2%Did the haidgeotrd_evice feel ike i Ilinvdedyour brivacy orpernoal space?No 55 100%Yes 0 0%Didylo rhow ay oWyurfriends or co-wokers how thge devieworked?No 20 31

Yes36%

35 64%

7. SYSTEM PERFORMANCE

7.1 Enrollment AnalysisAs stated in section 4, a conflict arose between the auditingsoftware and the flash card inside the hand geometry device,which collected the raw images. The result of this conflictcaused 16 test subjects' unique user identification numbers notto be recognized, causing additional enrollment sessions forthese participants. Moreover, an analysis of the enrollmentimages of these 16 subjects revealed that there were no problemsin the acquired images, as the samples were of sufficient qualitythus these 16 subjects who enrolled more than once weredeemed not be classified as failure to enroll (FTE) attempts. Forfurther explanation, please refer back to sections 4 and 5.2.

The analysis of the enrollment data confirmed 156 enrollmentsequences were completed during the evaluation. The flash cardand auditing software error required one subject to perform 3additional enrollment sequences, 8 subjects requiring 2additional enrollment sequences, and 7 subjects requiring 1additional enrollment sequence. Four test subjects were requiredto provide 4 hand placements, as the device deemed the standard3 enrollment images as not sufficient. In summary, the handgeometry device had a 0% FTE, when the additional enrollmentsdue to the external software data communication error wereremoved.

7.2 Failure to Acquire AnalysisThe failure to acquire (FTA) rate is the percentage of attemptswhere the system fails to capture a biometric sample ofsufficient quality; including attempts where extracted featuresare substandard [5]. As such, the ground truth of the FTA cannotbe ascertained, as this was an unattended evaluation.

7.3 Verification System AnalysisSince this was a simulated scenario evaluation, it is essential toreport the false reject rate (FRR) and the false accept rate (FAR).However, since the evaluation was unattended, the ground truthFAR could not be ascertained since it was untenable todifferentiate impostor attempts from subjects "gaming" thedevice. Thus, the analysis below represents the authors' bestefforts to remove imposter and gaming attempts from the data topredict biometric performance. For all analyses the systemthreshold was set at 100.

7.3.1 Performance analysis ofthe hand geometry systemAfter analyzing the data for specific hand placementsconstituting imposter attempts or "gaming" of the system, theauthors removed 37 placements from the dataset. Seventeenother placements were removed for reasons including: multiplehands present at the same time, jacket sleeves occluding thehand, and grossly incorrect hand placements.

The remaining 1788 hand placements were analyzed for 1-try, 2-try, and 3-try rejection rates. For users placing the hand morethan 3 times in close succession (within 15 minutes and no otherusers utilizing the system in-between), only the first 3placements from that session were analyzed. Figure 6 shows theresulting rejection rates versus hand reader threshold. TheHandkey II deployed at Purdue's RSC was set at a globalthreshold of 100, therefore the 1-try false reject rate was 2.26%,the 2-try rate was 1.18%, and the 3-try rate was 0.98%.Extremely low 3-try false rejection rates of 0.1% were possible,but only at a false acceptance rate approaching 2%.

10

C-,a-(a

a.

owL

987654321050 100 150

Threshold

- FAR1 try2try

-3try

200

Figure 6: Error Rates vs. Threshold

The false acceptance rate was estimated by comparing the same1788 verification attempts against the enrollment database. Thisis only an estimate, as imposter attempts may be present in thedata since the installation was unattended. At the defaultthreshold of 100, the predicted false accept rate was 0.87%.

The large number of gaming attempts, as well as the surveyresult that 65% of users were excited about hand geometrytechnology to show their friends how it worked, questions thevalidity of the unattended dataset. This may account for thediscrepancy in performance reported here and claims made bythe device manufacturer. The authors made a best-effort attemptto identify all gaming of the system, but if 17 placements wentundetected, the FRR would be overstated by more than 1%.

7.3.2 Unique characteristic performance analysisDuring image analysis, it was noticed that there wereproblematic attempts. These attempts were classified into 3categories, including: shirt/coat sleeves covering the hand,ring(s) present, improper thumb placement, and little fingers thatwould not straighten. One-way analysis of variance (ANOVA)tests were conducted on each of the categories to determine ifthe group had an effect on the score. Results were comparedbased at alpha=.05.

Authorized licensed use limited to: Purdue University. Downloaded on February 28,2010 at 16:52:06 EST from IEEE Xplore. Restrictions apply.

Page 6: (2005) Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance

The first ANOVA tested attempts with rings versus attemptswith no rings. The results stated that there was not a significantdifference between scores for attempts with rings and withoutrings, F(1, 1686)=.11 (p=.743), however larger rings wereproblematic as they fused multiple fingers together changing thesilhouette of the hand. Moreover, there was a significantdifference between correct and incorrect thumb placement on thescore, however only 6 verification attempts were recorded. Themean scores for correct thumb placement attempts was 39.9(standard deviation of 173.7) and 1125.2 (1145.6) for incorrectthumb placement attempts.

Shirt and coat sleeves were easily determined by examiningmultiple images of a user. There was a significant differencebetween the 40 attempts with shirt/coat sleeves covering thehand and those attempts without a sleeve present, F(1,1665)=173.27 (p=.0000). Mean scores for shirt/coat sleeveswere 476.9 (760.8) and 39.8 (173.7) without a shirt/coat sleevecovering the hand. The last analysis looked at one particular userwho had difficulty straightening out their little finger. The mean(and standard deviation) for these attempts was 133.1(101.8),causing false rejections at a threshold of 100. In a real worlddeployment, this user would obtain a special threshold setting toaccommodate their variable hand placement.

7.3.3 Demographics andperformanceA statistical analysis was also conducted on gender versus thescore and prior hand geometry use versus the score. The resultsrevealed that there was no significant difference between genderand score, F(1, 1840)=.73 (p=.48). Moreover, the mean andmedian for gender were approximately the same - 50.6 and 49.6for females and males respectively. However, there was asignificant difference between the scores of subjects who usedhand geometry prior to this study and those who had not - F(1,1840)=5.90 (p=.003). In particular, the mean score for subjectswho had used hand geometry before was 25, compared to 62.1for those who had never used hand geometry prior to this study.This is expected as users who had used hand geometry prior tothis evaluation have been habituated with the device for a periodof time, where as the rest of the test subjects had never used thedevice and had a longer acclimation period.

8. CONCLUSION & FUTURE WORKThe data presented here suggest that participants were acceptingof hand geometry at Purdue University's Recreational Center.Analyses of the participants' survey responses revealed that 93%liked using hand geometry, 98% thought the technology waseasy to use, and 87% preferred hand geometry to the existingcard-based system, while nobody thought the device invadedtheir personal privacy.

The complexity of analyzing an unattended system was notedthroughout this paper, specifically the difficulty ofdifferentiating a genuine attempt, a genuine "game" attempt, andan impostor attempt. These difficulties call into question thevalidity of the performance data in Section 7.1. In addition, theauthors will provide these questions to groups such as ISO/IECJTC1 SC37 to seek guidance for future operational tests,including treatment of imposter and "gaming" attempts that mayhave corrupted the data. Such guidance is critical to generatingmeaningful performance curves for operational tests in thefuture. Recommendations for future research would be toinclude a monitoring device in the evaluation area to assess whois actually using the device, differentiating genuine users fromthose "gaming" the device and impostors. This would assist inquantifying the number of attempts attacking the system, so theFAR and FRR could be calculated with higher confidence.

9. REFERENCES1. Elliott, S.J. & Scheller, A.R. (2003). Accessing

Recreational Facilities: Choosing the AppropriateBiometric. Unpublished manuscript, Purdue University,West Lafayette, IN

2. Mansfield, A.J. (2005). ISO/IEC FCD 19795-1, BiometricPerformance Testing and Reporting, Part 1 - Principlesand Framework (SC37N908): ISO/IEC JTC1 SC37.

3. Indiana State Climate Office, Purdue University. (2005).West Central Indiana Weather Observation Data.Available fromh

4. Miller, R.P., (1971). Finger Dimension ComparisonIdentification System. United States Patent 3,576,538.

5. Recognition Systems (2004). Hand Geometry. From

6. Zunkel, D. (1999). Hand Geometry Based Verification. InBiometrics: Personal Identification in a NetworkedSociety. S. Pankanti, Editors. 1999, Klewer AcademicPublishers Group: Norwell, MA. Pp. 87-101.

7. Wayman, J.L., (2000). A Definition of Biometrics. InNational Biometric Test Center Collected Works. 21-25.San Jose, CA: National Biometric Test Center

8. Elliott, S., Sickler, N, Kukula, E., & Modi, S. (2004).Introduction to Biometric Technology. Copymat.

9. Kukula, E. & Elliott, S. (2005) Results of the Spring 2005Hand Geometry Study at the RSC. From

System performance in this study of 129 subjects suggest thathand geometry would be an appropriate fit for this ACapplication. The FTE was 0% as all users were able to enroll.However, it will be interesting to continue data collection to seethe reactions of users and system performance in a longitudinalevaluation. Furthermore, the individual analyses reveal thateither more training or stricter policies for subjects interactingwith the device need to be implemented for some users, asroughly 100 attempts deviated from the training that was given.

Authorized licensed use limited to: Purdue University. Downloaded on February 28,2010 at 16:52:06 EST from IEEE Xplore. Restrictions apply.