D6.2 Report on application testing and evaluation · D6.2 - Report on application testing and evaluation MATTHEW D6.2 Page 9 of 37 Chapter 2 Test and Evaluation for payment application

D6.2

Report on application testing and evaluation

Project number: 610436

Project acronym: MATTHEW

Project title:

MATTHEW: Multi-entity-security using active

Transmission Technology for improved Handling

of Exportable security credentials Without

privacy restrictions

Start date of the project: 1st November, 2013

Duration: 36 months

Programme: FP7-ICT-2013-10

Deliverable type: Report

Deliverable reference number: ICT-610436 / D6.2/ RELEASE 1.0

Activity and Work package contributing to

the deliverable: WP6

Due date: October 2016 – M36

Actual submission date: October 31st, 2016

Responsible organisation: GTO

Editor: Gregory CAPOMAGGIO

Dissemination level: Public

Revision: RELEASE 1.0

Abstract:

This document reports the test and evaluation

works of the hardware demonstrators developed

according to the three project use cases defined

in WP1.

Keywords:

EMVCo certification, Physical Unclonable

Function, Attribute-Based Credentials.

This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement n° 610436.

D6.2 - Report on application testing and evaluation

MATTHEW D6.2 Page II

Editor

Gregory CAPOMAGGIO (GTO)

Contributors (ordered according to beneficiary numbers)

Pavel KRISTOF, (IMA)

Cécile DELERABLEE (CRX)

Disclaimer

The information in this document is provided "as is", and no guarantee or warranty is given that the information is fit for any particular purpose. The user uses the information at its sole risk and liability.


MATTHEW D6.2 Page III

Executive Summary

This WP6 report of the MATTHEW project describes all tests and evaluations performed on the different demonstrators realized in the context of the WP4 (application software development) and WP5 (hardware integration and prototyping). The consortium defined at the beginning of the MATTHEW project 3 different use cases, each one focusing on different aspects of the MATTHEW system requirements. For this reason, this report has been split into 3 different chapters addressing the “Advanced mobile payment”, the “enhanced access control” and the “privacy-preserving access to remote services” application tests.

Concerning the payment application, which has been reoriented since the submission of the MATTHEW project to address instead the wearable devices due to the market evolution, the evaluation is mainly focusing on the interoperability of the MATTHEW SiP platform with EMVCo infrastructures to demonstrate the outstanding contactless performances achieved thanks to the ALM technology.

For the enhanced access control application, a real deployment of access control equipment has been realized in the IMA building in Prague in order to evaluate the ergonomics and timing performances of the three different scenarios: single card, PIN card and chained card.

Finally, for privacy-preserving access to remote services, the last chapter evaluates the timings performances of software applications implementing ABC-based cryptography and protocol on an Android mobile phone, a simulated transportation access gate, and a simulated transportation ticketing server, in order to check if such implementation could fit with standard transportation timing constraints.


MATTHEW D6.2 Page IV

Contents

Chapter 1 Introduction ......................................................................................... 8

Chapter 2 Test and Evaluation for payment application ................................... 9

2.1 Goals of measurements ................................................................................... 9

2.2 Introduction of the EMVCo test equipments ..................................................... 9

2.2.1 EMVCo analogue test ............................................................................................ 9

2.2.1.1 EMVCo analogue test bench .........................................................................10

2.2.1.1.1 EMVCo Test PCD ................................................................................................... 10

2.2.1.1.2 EMV Test CMR ....................................................................................................... 10

2.2.1.1.3 EMV Test PICC ...................................................................................................... 10

2.2.1.1.4 EMV Test Suite ....................................................................................................... 11

2.2.1.2 EMVCo analogue test description .................................................................11

2.2.2 EMVCo combination tests .....................................................................................12

2.2.3 EMVCo operating volume .....................................................................................13

2.3 Device under test (DUT) setups ..................................................................... 14

2.3.1 Active RF antenna sizes .......................................................................................14

2.3.2 Contactless protocols ............................................................................................15

2.3.3 Hardware equipment .............................................................................................15

2.4 Tests results .................................................................................................. 16

2.4.1 SiP Class 6 – ISO14443-A ....................................................................................17

2.4.1.1 EMVCo analogue tests results ......................................................................17

2.4.1.2 EMVCo combination tests results ..................................................................17

2.4.2 SiP Class 6 – ISO14443-B ....................................................................................18



2.4.3 SiP Class 5 – ISO14443-A ....................................................................................19



2.4.4 SiP Class 5 – ISO14443-B ....................................................................................21



2.5 Tests interpretation ........................................................................................ 22

Chapter 3 Test and evaluation for enhanced access control application ..... 23

3.1 Access control test components .................................................................... 23

3.2 Reader tests .................................................................................................. 23

3.2.1 SAM interface tests ...............................................................................................23


MATTHEW D6.2 Page V

3.2.2 NFC interface tests ...............................................................................................25

3.2.3 Android APP tests .................................................................................................25

3.2.3.1 Unit tests .......................................................................................................26

3.2.3.2 Integration tests .............................................................................................26

3.3 ACCESS control operation tests description .................................................. 26

3.3.1 Single card scenario testing ..................................................................................26

3.3.2 PIN card scenario testing ......................................................................................27

3.3.3 Chained card scenario testing ...............................................................................28

3.4 Test results .................................................................................................... 29

3.4.1 Hardware and components tests results ...............................................................29

3.4.2 Operation test results ............................................................................................30

3.4.2.1 Single card scenario test result......................................................................30

3.4.2.2 PIN card scenario tests result ........................................................................30

3.4.2.3 Chained card scenario test result ..................................................................30

Chapter 4 Test and evaluation for privacy-preserving access to remote services 31

4.1 Demonstrator overview .................................................................................. 31

4.2 Goal of the demonstrator evaluation .............................................................. 32

4.3 Measured timings ........................................................................................... 32

4.3.1 Ticket purchase timings ........................................................................................33

4.3.2 Ticket presentation timings....................................................................................33

4.4 PUF Framework Testing ................................................................................ 34

Chapter 5 Conclusion ......................................................................................... 36

Chapter 6 List of Abbreviations ........................................................................ 37


MATTHEW D6.2 Page VI

List of Figures

Figure 1: EMV analogue test bench ......................................................................................10

Figure 2: List of POS terminals used for combination tests ...................................................12

Figure 3: EMVCo PCD operating volume geometry ..............................................................13

Figure 4: PICC position within the EMVCo operating volume ...............................................14

Figure 5: Class 6 wearable NFC antenna .............................................................................14

Figure 6: Class 5 wearable NFC antenna .............................................................................15

Figure 7: Class 6 wearable DUT ...........................................................................................15

Figure 8: Class 5 wearable DUT ...........................................................................................16

Figure 9: Combination fail positions with SiP Class 5 antenna (ISO14443-A) and Ingenico iPP350 terminal ............................................................................................................20

Figure 10: Integration pass/fail positions with SiP Class 5 antenna (ISO14443-A) and Ingenico iCT250 terminal ..............................................................................................20

Figure 11: Integration pass/fail positions with SiP Class 5 antenna (ISO14443-A) and OTI Saturn 6000 terminal ....................................................................................................20

Figure 12: Integration pass/fail positions with SiP Class 5 antenna (Type B) and Ingenico iCT250 terminal ............................................................................................................22

Figure 13: Integration pass/fail positions with SiP Class 5 antenna (Type B) and OTI Saturn 6000 terminal ................................................................................................................22

Figure 14: MATTHEW access control components ..............................................................23

Figure 15: IDSIMA K4 components ......................................................................................23

Figure 16: First SAM reader toolset ......................................................................................24

Figure 17: Final SAM reader toolset .....................................................................................24

Figure 18: PN532 development board ..................................................................................25

Figure 19: BT / NFC access control reader ...........................................................................25

Figure 20: Single card access control scenario environment ................................................27

Figure 21: PIN card access control scenario environment ....................................................27

Figure 22: Chained card access control scenario environment .............................................28

Figure 23: MATTHEW ticketing Android application screenshot ...........................................31

Figure 24: MATTHEW ticketing Android application screenshot ...........................................32


MATTHEW D6.2 Page VII

List of Tables Table 1: EMVCo analogue tests list for ISO14443-A and ISO14443-B PICC ........................12

Table 2: EMVCo analogue tests results with SiP Class 6 antenna (ISO14443-A) .................17

Table 3: EMVCo combination tests results with SiP Class 6 antenna (ISO14443-A) ............17

Table 4: EMVCo analogue tests results with SiP Class 6 antenna (ISO14443-B) .................18

Table 5: EMVCo combination tests results with SiP Class 6 antenna (ISO14443-B) ............18

Table 6: EMVCo analogue tests results with SiP Class 5 antenna (ISO14443-A) .................19

Table 7: EMVCo combination tests results with SiP Class 5 antenna (ISO14443-A) ............19

Table 8: EMVCo analogue tests results with SiP Class 5 antenna (ISO14443-B) .................21

Table 9: EMVCo combination tests results with SiP Class 5 antenna (ISO14443-B) ............21


MATTHEW D6.2 Page 8 of 37

Chapter 1 Introduction

The MATTHEW project defined a platform architecture for upcoming and future generations of mobile devices with multiple secure entities involved, creating ways to transfer some of those entities or security relevant credentials stored on such entities taking privacy protection into consideration.

In addition, the MATTHEW project takes care on prerequisites for the highly reliable implementation of the NFC (Near Field Communication)-based communication dedicated for small form factor communicating objects by driving active transmission technology to a high level of radio transmission quality.

The project illustrates the flexibility and adaptability of the MATTHEW architecture and concepts through three different use cases. Two types of use cases are represented within MATTHEW. One type being a set of use cases close to industrial exploitation (advanced mobile payment and access control), the other type being more research oriented and still further away from market exploitation (privacy enhanced transferable public transport credentials).

During the project timeline, the MATTHEW partners defined and realized hardware demonstrators, one for each area of use, in order to point out the integratability and the performances of the different cryptographic and hardware bricks part of the MATTHEW architecture. The aims of the document are to summarize the different tests and evaluations performed on those hardware demonstrators, depict the state of the art of the developed bricks, and give an overview of the possible future enhancement of the MATTHEW technology.



Chapter 2 Test and Evaluation for payment application

2.1 Goals of measurements

Since secure contactless applications for wearable devices are mainly lead by the payment market, the biggest concern of wearable manufactures nowadays is to evaluate the interoperability of the MATTHEW SiP platform, once integrated into their equipment, with all deployed payment infrastructures.

In this context, executing the EMVCo certification process on the MATTHEW SiP platform appears to be a suitable solution to demonstrate the maturity and performances of the ALM technology to future customers.

The complete EMVCo process usually requires to pass a large panel of tests (electrical, analogue, integration, etc…) in order to certify a contactless product. In the context of the MATTHEW SiP platform evaluation, we decided just to focus on the analogue and combination tests since digital and application aspects are purely linked to the secure element software implementation which was previously tested during the contactless card certification process.

2.2 Introduction of the EMVCo test equipments

2.2.1 EMVCo analogue test

To check the RF performances of contactless devices, EMVCo has developed a specific test bench that operates as a reference POS terminal. This test bench could be calibrated to simulate the different panels of payment terminals deployed on the field, modifying transmission parameters like the RF field strength or communication timings in the range specified by EMVCo.

Furthermore EMVCo specified a series of analogue tests to be performed once the contactless device (DUT) is placed on the test bench inside the EMVCo operating volume (Refer to chapter 2.2.3).



2.2.1.1 EMVCo analogue test bench

Figure 1: EMV analogue test bench

2.2.1.1.1 EMVCo Test PCD

The EMV Test PCD has a circular antenna of about 7 cm, which is in the small range of antenna sizes encountered in EMV terminals. The circular antenna creates a symmetric field distribution from the z-axis, which simplifies measurements. When fed with 600 mW into its 50Ω input impedance at resonance, the EMV Test PCD provides a magnetic field which is representative for most POS terminals. The EMV Test PCD allows commands to be sent to PICCs when connected to a signal generator, in this case a contactless tester along with a power amplifier required to achieve maximum RF field value defined in the EMVCo specification.

2.2.1.1.2 EMV Test CMR

The aim of the EMV Test CMR is to form a signal switching and conditioning unit for the test bench before proceeding to the signal analysis. This electronic board implements a synchronous demodulation technology which does not distort or filter the PICC signal in order to realise accurate measurements. The PICC demodulated signal generated by the EMV Test CMR is analysed thereafter by a digital oscilloscope. The EMV Test CMR is directly controlled by the EMV Test Suite through a SPI Aardvark probe.

2.2.1.1.3 EMV Test PICC

The EMV Test PICC has an antenna similar to those found in ID-1 cards. It allows the analysis of the signal as sent out by a PCD. For analysing the frequency content of these signals, it is equipped with a pickup coil, which is an integral part of the EMV Test PICC. The EMV Test PICC can be configured with a linear and a non-linear load. The non-linear load is self-adapting to the magnetic field strength. The variable load parameters are set based on the maximum power consumption in current contactless cards. The maximum power consumption represents a worst case scenario for a PCD. It is expected that a PICC will



require less power than the EMV Test PICC. For the EMV PICC Test Suite, the EMV Test PICC is mainly used to calibrate the field strength and modulated signals of the EMV Test PCD before performing all tests.

2.2.1.1.4 EMV Test Suite

The EMV Test Suite is a complete software application which facilitates testing with automation features. The EMV Test Suite controls all test bench equipment like the contactless reader, the digital oscilloscope or the EMV Test CMR board, and provides full visibility of all test results including logs and all traces of communications between the test bench and the tested device.

2.2.1.2 EMVCo analogue test description

Table 1 enumerates the different analogue tests a contactless device (PICC) has to successfully pass in order to be eligible for an EMVCo certification. These tests have been grouped according to the dedicated communication layer:

Test ID Test description A B

Radio frequency and power tests

CAB111 PICC influence on the field. X X

Cx112 PICC power off state field. X X

Cx113 PICC power on state field. X X

Signal interface PCD to PICC tests

Cx121 PICC responsiveness at minimum field intensity. X X

Cx122 PICC responsiveness at maximum field intensity. X X

Cx123 PICC responsiveness at nominal field intensity. X X

Cx124 PICC responsiveness at minimum carrier frequency. X X

Cx125 PICC responsiveness at maximum carrier frequency. X X

CA126 PICC responsiveness at minimum timing t1. X

CA127 PICC responsiveness at maximum timing t1. X

CB126 PICC responsiveness at minimum PCD modulation index. X

CB127 PICC responsiveness at maximum PCD modulation index. X

CB128 PICC responsiveness after a WUPA with maximum timing t1 at minimum field intensity.

X

Signal interface PICC to PCD tests



Cx131 PICC load modulation. X X

Cx132.1 PICC subcarrier frequency and bitrate at nominal carrier frequency. X X

Cx132.2 PICC subcarrier frequency and bitrate at minimum carrier frequency. X X

Cx132.3 PICC subcarrier frequency and bitrate at maximum carrier frequency. X X

Cx133 PICC subcarrier modulation. X X

Cx134 PICC detectable disturbance. X X

Sequence and frame bit coding and synchronization tests

Cx143 Bit coding and de-synchronisation PICC to PCD. X X

CA144 Verifying Type A PICC timings. X

CB144 Verifying Type B bit boundaries. X

CB145 Verifying Type B PICC timings X

CB147 Verifying Type B maximum bit boundaries. X

Table 1: EMVCo analogue tests list for ISO14443-A and ISO14443-B PICC

2.2.2 EMVCo combination tests

To verify the interoperability and the responsiveness of tested contactless devices, complete payment transactions are performed on real POS terminals during combination tests. Usually different terminal models, representative for the equipment deployed in the field, are selected for this test. Since the official list of POS terminals is kept secret by the certification laboratory, Gemalto selected implicitly the four following devices:

Ingenico iPP350 Ingenico iCT250 VIVOPay 5000 OTI Saturn 6000

Figure 2: List of POS terminals used for combination tests

During combination tests, the tested device is successively placed in front of the payment terminal at each position of the operating volume defined by EMVCo specification (refer to chapter 2.2.3), and a tentative of transaction is executed. Each successful transaction increase a score value; the number of points depending on the position location.



Mastercard and VISA establish different acceptance criteria according to the form factor of the tested contactless device. For example, an ID1 contactless card will have to obtain an average score of 80% on positions up to 4cm to be certified, while a NFC mobile phone or a contactless sticker will have to obtain an average score of 80% on positions up to 2cm. Since the acceptance criteria for wearable devices is not yet fully defined by Mastercard and VISA, combination tests on MATTHEW SiP platforms have been performed on the whole EMVCo operating volume.

2.2.3 EMVCo operating volume

The operating volume of a PCD is the 3-dimensional space for which the EMV specification imposes requirements on the PICC performances. The geometry of the operating volume is shown in Figure 3.

The operating volume is measured from the centre of the landing plane of PCD, usually identified by the contactless symbol, along an axis perpendicular to the landing plane.

Figure 3: EMVCo PCD operating volume geometry

The position of a PICC within the operating volume is represented by the triplet (z, r, ϕ) as described by Figure 4. According to the test performed, all, few or just one position must be tested. The same operating volume is specified for EMV analogue test, performed on the EMV Test PCD, and for EMV combination tests, performed on real POS terminals.

200 110 120

213

223

219

229

216 226

Z = 2 cm

300 310 320

313

323

319

329

316 326

Z = 3 cm

000 010

013

019

016

Z = 0 cm

100 110 120

113

123

119

129

116 126

Z = 1 cm

400 410

413

419

416

Z = 4 cm



Figure 4: PICC position within the EMVCo operating volume

2.3 Device under test (DUT) setups

According to the PICC device form factor, Mastercard and Visa usually define specific analogue and digital test suites in order to match the targeted payment ergonomics and infrastructure constraints. For example, a mobile EMV solution is required to pass successfully all analogue and integration tests in the 0-2cm operating range, while a classic contactless payment card has to operate in the full 0-4cm operating volume.

Since the wearable market is pretty new, no process has been clearly defined to certify such device like smartwatches or smart bands so far. In this context we decided to follow the most constraining process for the MATTHEW SiP platform to have a clear picture of the ALM technology position compared to the expected EMV performances.

This chapter describes the different hardware setups especially assembled to perform the EMV analogue and interoperability tests.

2.3.1 Active RF antenna sizes

Since the MATTHEW SiP platform doesn’t embed the contactless RF antenna, unlike MATTHEW nanoSIM card, in order to provide a maximum of flexibility to wearable device manufacturers to customize antenna size and placement, measurements will be realized using 2 different RF antenna sizes. Each antenna has been optimized to fit the two different integration approaches:

A tiny 5x15mm class 6 PICC antenna that could be embedded in wearable devices offering a metal-free area location. Indeed, in such non-restrictive environment a small antenna of few tens of millimetres square coupled with the ALM technology is enough to provide the same RF performance as a legacy contactless ID1 card.

Figure 5: Class 6 wearable NFC antenna

A 30mm diameter class 5 PICC antenna used for wearable devices (especially round smart watches) where electronics fill the whole space, and antenna must be necessarily stacked above metallic elements. In such configuration, a RF antenna covering much of surface of the device could be used to compensate the shielding effect.



Figure 6: Class 5 wearable NFC antenna

2.3.2 Contactless protocols

Nowadays, contactless secure applications embedded in wearable devices are mainly lead by the payment and transportation markets. Many of these infrastructures support only one of the contactless protocols defined by the ISO14443 international standard (ISO14443-A or ISO14443-B protocol).

In order to guarantee the full interoperability of the MATTHEW SiP platform with such infrastructures, the RF performance tests have been performed on both contactless protocols.

2.3.3 Hardware equipment

The MATTHEW SiP expansion board, initially designed in work package 5 to realize the advanced payment demonstrator, has been used here to perform all EMVCo tests. A custom “Setup” daughter board has been realized and permit, once plugged on the SiP expansion board, to power this one and quickly update the MATTHEW SiP RF settings like the contactless protocol mode (ISO14443-A or B), the transmission mode (AND or XOR mode) or the transmission phase, without updating the SiP flash memory content.

Figure 7: Class 6 wearable DUT

SiP Class 6 Expansion Board

Setup Board

+ =



The MATTHEW SiP expansion board was derived in a 3 wires class 5 planar antenna (Ø30mm round form factor) that has been shielded with a piece of metal to simulate the integration constraints of the SiP solution into round smart watches as described in Chapter 2.3.1.

Figure 8: Class 5 wearable DUT

2.4 Tests results

This chapter summarizes the EMVCo analogue and combination tests results of the 4 different MATTHEW SiP expansion board setups:

MATTHEW SiP Class 6 expansion board configured in ISO14443-A.

MATTHEW SiP Class 6 expansion board configured in ISO14443-B.

MATTHEW SiP Class 5 expansion board configured in ISO14443-A.

MATTHEW SiP Class 5 expansion board configured in ISO14443-B.

SiP Class 5 Expansion Board

Setup Board

+ =



2.4.1 SiP Class 6 – ISO14443-A

2.4.1.1 EMVCo analogue tests results

Test Position

0 cm 1 cm 2 cm 3 cm 4 cm

0 0 0

0 1 0

0 1 3

0 1 6

0 1 9

1 0 0

1 1 0

1 1 3

1 1 6

1 1 9

1 2 0

1 2 3

1 2 6

1 2 9

2 0 0

2 1 0

2 1 3

2 1 6

2 1 9

2 2 0

2 2 3

2 2 6

2 2 9

3 0 0

3 1 0

3 1 3

3 1 6

3 1 9

3 2 0

3 2 3

3 2 6

3 2 9

4 0 0

4 1 0

4 1 3

4 1 6

4 1 9

CAB111 CA112 CA113

CA121 CA122 CA123 CA124 CA125 CA126 CA127

CA131 CA132.1 (1) CA132.2 (1) CA132.3 (1) CA133 CA134

CA143 CA144 (1) Pass Fail Skip Not Relevant

Table 2: EMVCo analogue tests results with SiP Class 6 antenna (ISO14443-A)

(1)Tests not relevant since measurement of PICC sub carrier frequency and PICC bit boundaries on digital oscilloscope is not correctly performed by the EMV Test Suite.

2.4.1.2 EMVCo combination tests results

Test Score

Up to 2cm Up to 4cm

Ingénico iPPP350 100% 100%

Ingénico iCT250 100% 100%

VIVOPAY 5000 100% 100%

OTI Saturn 6000 100% 100%

Table 3: EMVCo combination tests results with SiP Class 6 antenna (ISO14443-A)



2.4.2 SiP Class 6 – ISO14443-B


Test Position


0 0 0

0 1 0

0 1 3

0 1 6

0 1 9

1 0 0

1 1 0

1 1 3

1 1 6

1 1 9

1 2 0

1 2 3

1 2 6

1 2 9

2 0 0

2 1 0

2 1 3

2 1 6

2 1 9

2 2 0

2 2 3

2 2 6

2 2 9

3 0 0

3 1 0

3 1 3

3 1 6

3 1 9

3 2 0

3 2 3

3 2 6

3 2 9

4 0 0

4 1 0

4 1 3

4 1 6

4 1 9

CAB111 CB112 CB113

CB121 CB122 CB123 CB124 CB125 CB126 CB127 CB128

CB131 CB132.1 (1) CB132.2 CB132.3 CB133 CB134

CB143 CB144 (1) CB145 (1) CB147 (2) Pass Fail Skip Not Relevant

Table 4: EMVCo analogue tests results with SiP Class 6 antenna (ISO14443-B)


(2)Test not relevant since EMV Test Suite failed to correctly apply test conditions.


Test Score

Up to 2cm Up to 4cm



VIVOPAY 5000 100% 100%

OTI Saturn 6000 100% 100%

Table 5: EMVCo combination tests results with SiP Class 6 antenna (ISO14443-B)



2.4.3 SiP Class 5 – ISO14443-A


Test Position


0 0 0

0 1 0

0 1 3

0 1 6

0 1 9

1 0 0

1 1 0

1 1 3

1 1 6

1 1 9

1 2 0

1 2 3

1 2 6

1 2 9

2 0 0

2 1 0

2 1 3

2 1 6

2 1 9

2 2 0

2 2 3

2 2 6

2 2 9

3 0 0

3 1 0

3 1 3

3 1 6

3 1 9

3 2 0

3 2 3

3 2 6

3 2 9

4 0 0

4 1 0

4 1 3

4 1 6

4 1 9

CAB111 CA112 CA113

CA121 CA122 CA123 CA124 CA125 CA126 CA127

CA131 CA132.1 CA132.2 CA132.3 CA133 CA134

CA143 CA144 (1) Pass Fail Skip Not Relevant

Table 6: EMVCo analogue tests results with SiP Class 5 antenna (ISO14443-A)



Test Score

Up to 2cm Up to 4cm



VIVOPAY 5000 100% 100%

OTI Saturn 6000 73% 80%

Table 7: EMVCo combination tests results with SiP Class 5 antenna (ISO14443-A)



Ingenico iPP350

Figure 9: Combination fail positions with SiP Class 5 antenna (ISO14443-A) and Ingenico iPP350 terminal

Ingenico iCT250

Figure 10: Integration pass/fail positions with SiP Class 5 antenna (ISO14443-A) and Ingenico iCT250 terminal

OTI Saturn 6000

Figure 11: Integration pass/fail positions with SiP Class 5 antenna (ISO14443-A) and OTI Saturn 6000 terminal

200 110 120

213

223

219

229

216 226

Z = 2 cm

300 310 320

313

323

319

329

316 326

Z = 3 cm

000 010

013

019

016

Z = 0 cm

100 110 120

113

123

119

129

116 126

Z = 1 cm

400 410

413

419

416

200 110 120

213

223

219

229

216 226

Z = 2 cm

300 310 320

313

323

319

329

316 326

Z = 3 cm

000 010

013

019

016

Z = 0 cm

100 110 120

113

123

119

129

116 126

Z = 1 cm

400 410

413

419

416

200 110 120

213

223

219

229

216 226

Z = 2 cm

300 310 320

313

323

319

329

316 326

Z = 3 cm

000 010

013

019

016

Z = 0 cm

100 110 120

113

123

119

129

116 126

Z = 1 cm

400 410

413

419

416

Z = 4 cm

Z = 4 cm

Z = 4 cm



2.4.4 SiP Class 5 – ISO14443-B


Test Position


0 0 0

0 1 0

0 1 3

0 1 6

0 1 9

1 0 0

1 1 0

1 1 3

1 1 6

1 1 9

1 2 0

1 2 3

1 2 6

1 2 9

2 0 0

2 1 0

2 1 3

2 1 6

2 1 9

2 2 0

2 2 3

2 2 6

2 2 9

3 0 0

3 1 0

3 1 3

3 1 6

3 1 9

3 2 0

3 2 3

3 2 6

3 2 9

4 0 0

4 1 0

4 1 3

4 1 6

4 1 9

CAB111 CB112 CB113

CB121 CB122 CB123 CB124 CB125 CB126 CB127 CB128

CB131 CB132.1 CB132.2 (1) CB132.3 (1) CB133 CB134

CB143 CB144 (1) CB145 (1) CB147 (2) Pass Fail Skip Not Relevant

Table 8: EMVCo analogue tests results with SiP Class 5 antenna (ISO14443-B)


(2)Test not relevant since EMV Test Suite failed to correctly apply test conditions.


Test Score

Up to 2cm Up to 4cm



VIVOPAY 5000 100% 100%

OTI Saturn 6000 84% 88%

Table 9: EMVCo combination tests results with SiP Class 5 antenna (ISO14443-B)



Ingenico iCT250

Figure 12: Integration pass/fail positions with SiP Class 5 antenna (Type B) and Ingenico iCT250 terminal

OTI Saturn 6000

Figure 13: Integration pass/fail positions with SiP Class 5 antenna (Type B) and OTI Saturn 6000 terminal

2.5 Tests interpretation

These EMVCo test results demonstrate the outstanding contactless performances of the MATTHEW SiP platform integrated in restrictive environment. Thanks to the ALM technology, small objects like wearable devices are able to propose now efficient contactless solutions competing with classic smartcard form factors and offering a very good user experience.

At the opposite of the EMVCo mobile phone or stickers acceptance criteria (usually limited up to 2cm), the technology implemented in the MATTHEW SiP solution already guarantee to pass the EMVCo certification process up to 4cm, improving ergonomics and interoperability with POS terminals deployed on the field.

EMVCo combination tests also reveal weaknesses of any current POS terminal (refer to Table 7) that do not support contactless devices generating high load modulations staying nevertheless in the range defined by the specification. Indeed, according to the demodulation technology, reception circuits of any terminals are “blinded”, usually at short distance, by contactless devices embedding a powerful ALM frontend. Concretely, strong standardisation efforts are actually ongoing by the MATTHEW partners in order to design and specify new test tools to improve the next terminal generation.

200 110 120

213

223

219

229

216 226

Z = 2 cm

300 310 320

313

323

319

329

316 326

Z = 3 cm

000 010

013

019

016

Z = 0 cm

100 110 120

113

123

119

129

116 126

Z = 1 cm

400 410

413

419

416

200 110 120

213

223

219

229

216 226

Z = 2 cm

300 310 320

313

323

319

329

316 326

Z = 3 cm

000 010

013

019

016

Z = 0 cm

100 110 120

113

123

119

129

116 126

Z = 1 cm

400 410

413

419

416

Z = 4 cm

Z = 4 cm



Chapter 3 Test and evaluation for enhanced access control application

3.1 Access control test components

For full testing of the access control system, we used the following MATTHEW specific components:

Reader RSW04.

SAM card with CIPURSE applet.

Mobile phone with ANDROID OS.

SIM card with CIPURSE applet.

Figure 14: MATTHEW access control components

Standard IDSIMA K4 components used for testing were:

Access control terminal CKP-2N.

Concentrator K4 MASTER.

K4 server.

Figure 15: IDSIMA K4 components

3.2 Reader tests

3.2.1 SAM interface tests

For the first development and testing of the SAM interface, simple tools were used for connecting a standard ID size card (refer to Figure 16). For this, a standard JAVA card with JCOP OS J3A041 was used at first. Because during the tests we found that the speed of



calculation of the cryptogram is too slow, we selected another type of SAM chip where the security application was written in native C language.

As the final solution, we selected the standard CIPURSE SAM from INFINEON.

Figure 16: First SAM reader toolset

After successful SAM testing, we used SAM in standard size.

Figure 17: Final SAM reader toolset

We equipped 3 readers with a standard SAM size interface that were used for operating tests in IMA. While standard readers are flooded by special resin and thus only the PCB with the SAM connector protrudes from the flood resin, Figure 17 depicts one of our readers which is not flooded.



3.2.2 NFC interface tests

For the start of the development and the NFC testing, the development board PN532 comprising an NFC antenna and an NFC chip was used.

Figure 18: PN532 development board

After the first tests, the development of the RS.04 reader was started, which was described in the document “D4.2 High Security Access control system with multiple security modules M30 PU”.

Figure 19: BT / NFC access control reader

Later on, the BT smart module to NFC antenna module of the reader was integrated. Then, the reader is prepared for CIPURSE communication over the BT. A photo of the reader is depicted in Figure 19. The complete reader was certified by the Czech certification organization EZU according to the standard CCA EMC. In addition, the reader was certified by the NBU (National Security Office) in the department Physical Security. All IDSIMA K4 chain is certified by NBU as one budget.

3.2.3 Android APP tests

There were two phases of the Android application testing. First, the code was continuously tested during implementation by unit tests. Second, the completed application was then tested as a whole with the reader in integration tests.



3.2.3.1 Unit tests

Unit tests are software tests which provide an automated testing of particular software modules. An environment used for the development was the official IDE for Android, Android Studio. It fully supports all required types of unit tests.

One type are local unit tests. Those were used to validate the implementation of the communication protocol between the handset and the reader along with other modules which are abstracted from Android specific APIs. Because local unit tests enable testing of a code which is independent from the Android platform, these can be run in a standard Java environment on a PC which makes testing quicker and more convenient.

The second type, instrumented tests, run on an emulator or a real Android device so it is suitable for an Android specific code. So they were used mainly to test UI related code.

3.2.3.2 Integration tests

After the code was validated by the unit tests the application was handed over to testers for the integration tests. They tested the complete PIN scenario with the MATTHEW reader and the handsets with the application and the CIPURSE SIM. An emphasis was not only placed on correct behavior, but also on convenient user experience.

3.3 ACCESS control operation tests description

All partially developed and tested components have been installed in IMA building in Prague. Because the testing of different scenarios required different environments, we had to choose between different places according to various parameters.

We had to choose a place where the scenario will be tested enough, but it had not to make opening the door more difficult than it was before.

For operation testing, persons must be chosen who want to help with testing.

People who do not perform testing should be able to open the door in a standard way (standard card).

These conditions could be simply fulfilled for the “Single Card Scenario” and the “Pin Card Scenario”. For the “Chained Card Scenario”, we had to use a place where only persons who are testing the MATTHEW system have access.

3.3.1 Single card scenario testing

The easiest scenario to test is the single card scenario. For the testing of this scenario we chose a standard door for the corridor on the first floor.

To the reader, a firmware was downloaded which works with standard IMA access control cards. This firmware was combined with the CIPURSE reader firmware.

To the SAM card, IMA Keys were downloaded.

In the IDSIMA K4MANAGER module, a new ID was generated.

The new ID was added to the database of enabled cards.

To the SIM card, the CIPURSE APPLET with IMA keys was downloaded

The new ID was personalized to the SIM card.

After these steps, the single card scenario was prepared for testing. During testing, these parameters were checked:

Reliability.

Reading speed.

Reading reliability.

Time between successful reading and door opening.



Figure 20: Single card access control scenario environment

3.3.2 PIN card scenario testing

To test the PIN card scenario, we chose the main gate to IMA private parking. Only a few people tested this scenario, because it is not so simple to open the door by PIN. Other people use a standard card or a remote control transceiver.

To the reader, a firmware was downloaded which works with standard IMA access control cards. This firmware was combined with the CIPURSE reader firmware.

To the SAM card, the IMA key was inserted.

In IDSIMA K4 MANAGER, a new ID with the PIN flag being set was generated. The PIN itself was generated in IDSIMA K4 manager too.

To the SIM card, the APPLET with personalized ID, PIN flag and PIN data was downloaded.

During test we checked:

Reliability.

PIN entering.

Time for opening.

Figure 21: PIN card access control scenario environment



3.3.3 Chained card scenario testing

For testing of the “Chained card” scenario we chose a key safe from TRAKA of the ASSA ABLOY group.

Hereby, a special firmware was downloaded to the reader RSW04. Two cards (mobile devices with a CIPURSE SIM) with special APPLETS had to be consequently inserted near the reader, to open the door of the safe. After opening the door, only the appropriate key inside the safe is unlocked and can be removed from the safe. As a transaction owner, the ID from second card (mobile device) being presented is recorded.

To the reader CIPURSE firmware was downloaded.

To the SAM card, the IMA key was inserted.

In IDSIMA K4MANAGER, a new ID with the CHAINED flag being set was generated.

To the SIM card, the APPLET with personalized ID and CHAINED flag being set was downloaded.

During test we checked:

Reliability.

Sequential reading of card.

Time for opening.

Figure 22: Chained card access control scenario environment



3.4 Test results

3.4.1 Hardware and components tests results

Components which were delivered by a third party (SAM and SIM) were tested according to the documents “CIPURSE V2 – Operation and Interface specification” and “CIPURSE V2 – SAM Specification”. Communication inside the reader between SIM and SAM was tested according to the documents “CIPURSE V2 – Cryptographic Protocol” and “CIPURSE V2 – SAM Specification”. All components work well.

A first test was made on the testing hardware shown before in Figure 18, where the SAM was programed in JAVA card. The final test was made with original hardware. The hardware of reader was certified in testing laboratory of the Department of defence:

Based on this, a successful certification CE declaration has been issued:



3.4.2 Operation test results

3.4.2.1 Single card scenario test result

The single card scenario was tested in real time on the doors to the IMA first floor corridor. The only difference that could be observed between the standard card and the CIPURSE SIM card inside the mobile phone was the processing time. In particular, here are results:

Standard IMA ID card CIPURSE SIM inside phone

Operating time 200ms 400ms

Opening time 300ms 500ms

Reliability OK OK

Usability Yes Yes

The single card scenario works well for all tests The opening time of 500ms between tapping the phone and opening of the door is on the border of usability. Higher security of CIPURSE is redeemed at a slower rate.

3.4.2.2 PIN card scenario tests result

The PIN card scenario was tested on the IMA main gate to private parking. For testing, only one mobile phone was used, because using the phone with a PIN is not very handy. One has to get out of the car, put the phone to the reader, remove the phone from the reader, insert the PIN and put the phone to the reader again.

It is a big difference if someone only pushes a button on the remote control. However, the scenario works well, but thanks to the complicated control of doors it is not usable in this place. The Android application for inserting PIN works well too. This scenario can thus be used to for access control to highly secured areas.

3.4.2.3 Chained card scenario test result

The chained card scenario was tested on the key safe. For this place, the security was sufficient and for this highly secure place it is not a problem that for opening the key safe two people have to be present.

The times for reading the card were the same as the operating time for the single card scenario. The same applies to the opening time when inserting the second mobile phone. For this case, the access control ID of second phone was recorded. The first phone with chained SIM card is not recorded in the access control, but on the SIM is present the list of cards which can be used as second. In the access control is recorded ID from SIM from second phone.

This scenario works well and is fully usable for use cases similar to the one tested.



Chapter 4 Test and evaluation for privacy-preserving access to remote services

This section is dedicated to the tests applied to the demonstrator of use case 3, implementing a public transport ticketing application on an Android smartphone. The demonstrator is composed of 3 entities:

An issuing server: in charge of issuing tickets,

An access gate: in charge of verifying the validity of a ticket,

A smartphone: used to purchase tickets from the issuing server and to present them at the access gate.

The version of the demonstrator tested here relies on the pairings in C library developed by IAIK to implement the ABC-based protocol described in Deliverable 2.2. In this protocol, users can purchase long term tickets and prove to the access gate that they have sufficient credentials without disclosing their exact credentials. In particular, multiple presentations of these credentials are unlinkable, making it impossible for the transport operator to track its users.

4.1 Demonstrator overview

The setup of the demonstrator is identical to the demonstrator described in Deliverable 4.3 which implemented a single use ticket protocol:

The issuing server is a computer running an Apache web server (backed by a MySQL database) with a Python WSGI module to answer the different requests,

The access gate is an USB NFC reader connected to a computer, running a Python application for the computations,

The Android application is a mix of Java for the interface, and a JNI/C module for the cryptographic operations.

The interface of the application displays the current long term ticket that will be presented to the access gate and allows buying tickets from the issuing server. When buying a ticket, the timing of the different operations is displayed, as on the screenshot below.

Figure 23: MATTHEW ticketing Android application screenshot



Note that the long term tickets in the demonstrator are valid for a single minute. In a real application, this would be replaced by a more appropriate duration, typically a week or a month, but this would not change the protocol in any way. There are always 2 credentials to present to an access gate: a zone number (between 1 and 3) and a validity period (here, a specific minute). However, a long term ticket can contain several zones, meaning that only a subset of the credentials is presented. This illustrates the minimum exposure principle that ABCs are intended for.

The interface to the access gate is very simple. It allows choosing the zone in which it is “installed”, that is, the zone the ticket has to contain in order to grant access, and displays a transcript of the APDUs exchanged over NFC between the reader and the smartphone.

Figure 24: MATTHEW ticketing Android application screenshot

4.2 Goal of the demonstrator evaluation

The security and soundness of the protocol implemented have already been studied in Deliverable 2.2, so the only remaining point to evaluate is the practicality of the protocol in a real implementation. Here, practicality means that the ticket presentation at the access gate should be fast enough so as not to cause waiting lines, but also that ticket purchase should take a reasonable time.

Usual constraints for this kind of application are:

300ms for a contactless operation (here, the ticket presentation)

1s for an online operation (here, the ticket purchase)

Note, however, that the demonstrator does not try to optimize these timings in any way. The operations are performed in a “didactic” way, rather than being optimized. It is not a problem if these constraints are not met, as it only means that some specific aspects would require some optimization effort before a product could be put on the market.

4.3 Measured timings

The timings can be measured at three different points: on the server, on the smartphone, or at the access gate. For these measures, the computer running the server and access gate is equipped with an Intel core i7-2600 CPU at 3.40 GHz and the smartphone is a Sony Xperia



Z5 compact equipped with a Qualcomm Snapdragon 810 CPU at 2.0 GHz. All computations are run using a single core, with a single computation thread.

4.3.1 Ticket purchase timings

The measures we performed show that there are no significant timing differences at the server when purchasing a long term ticket for a single zone or for multiple zones.

Operation Timing Description

Database operations 11ms All the database operation performed on the server: storing a user public key and the ticket issued.

Commitment verification 14.5ms Verification of the commitment sent by the user.

Ticket signature 8.5ms Computation of the long term ticket signature.

Total 34ms

On the phone side, however, the timings for the computation of the commitment differ slightly depending on the number of zones the ticket contains. Other timings are independent of this.


Commitment computation 105ms 147ms 175ms

Computation of the commitment presented to the server when requesting a ticket. Timings for 1, 2, and 3 zones.

Web server connection 55ms Connection to the remote ticketing server.

Ticket verification 700ms Verification of the term ticket signature.

Database operation 30ms Storage of the long term ticket in the Android SQLite database.

Total 890ms

Note that the server connection timings are not necessarily representative of a real life scenario: the measures were performed on a local network, meaning that for a distant server accessed through a 3G/4G connection, the timings would rather be between 200ms and 500ms. The ticket verification is the longest part in all the protocols but is not really necessary as the users can trust that the server will send them a valid ticket. It can therefore be run in the background, just as an additional security measure.

4.3.2 Ticket presentation timings

When presenting a ticket to the access gate, all the timings were measured on the reader side, but, as expected, identical timings were measured on the phone side.




Ticket request 44ms Initial APDU sent by the reader to request a ticket.

Credential computation 435ms 475ms 496ms

Computation of the credentials to present to the reader for the specific zone. The timings are given for tickets valid for 1, 2, and 3 zones.

Retrieval of the credentials 138ms Transmission of the credentials (cred and W) from the smartphone to the reader.

Zero-knowledge proof protocol 173ms Running time for the interactive zero-knowledge protocol assessing the validity of the presented credentials.

Proof verification 47ms Verification of the zero-knowledge proof sent by the smartphone.

Total 837ms

An important thing to note here is that the transmission time of the different elements of the protocol is between 200ms and 300ms, which is a lot when targeting a total execution time of 300ms. This could be improved by merging several APDUs together. The rest of the time is mostly spent on the credential computations. This is a little more complex to improve as it is really the cryptographic operations that need to be made faster. In this demonstrator, the implementation is made in pure C, and using a few optimized assembly routines could improve the timings by an important factor.

4.4 PUF Framework Testing

The protocol implemented in the demonstrator makes use of a private/public key pair on the user side. In the implementation of the demonstrator, this key pair is generated in software, and stored in a standard Android database. This makes it possible (with a little bit of Android hacking) to clone a ticket and the associated key pair to another smartphone. This is definitely not something transportation operators would want!

In order to avoid such cloning, a very efficient method is to derive the private key from a PUF, which are especially designed so as not to be clonable. However, such PUFs also need to be tested. Below are the results of the tests that were run.

Testing a PUF based key generation and storage framework can be divided in two phases. The first phase takes place before deployment, the second during deployment. When looking at a PUF framework globally, the bit error-rate is the one parameter of high importance. It might cause the entire system to fail, as the helper data algorithm might not be able to reconstruct the masked key due to an increased error-rate. Therefore, the PUF source is tested during the development process on different FPGA platforms in order to ensure that the error-rate is low, independent of the implemented technology.

In our case, the PUF was evaluated on a Xilinx KC705 and a Xilinx Zynq board. In both cases the evaluations provided similar results. As the error-rate also depends on the



environmental conditions, it is necessary to evaluate the PUF in different temperature ranges. By doing so it is possible to extract the “worst case” error-rate and to design the corresponding error-correcting code accordingly. The failure rate is defined as the probability that the key remonstration fails and typical values are 10-6. This means that only in one of one million reconstructions, the key is wrong and thus the process stops. We have designed our error correction mechanism to meet this standard requirement. Further, to counteract the phenomenon of aging, an additional safety margin was added. Aging causes the error-rate to slightly increase over time, which could cause troubles, when ignoring it.

As soon as the PUF framework is deployed in its environment, one has two possibilities to perform live-tests. The first option is to store an additional hash value of the key and always compare the reconstructed hash-value of the key with the stored one. The second option is based on the architecture of the helper-data algorithm, namely the code-offset construction. This architecture allows performing a live-test of the PUF’s error-rate as elaborated in D2.3. By doing so, the live-error rate can be internally compared to a threshold, which indicates the maximum noise-level for which the error-correcting code was designed. If the noise-level is above this threshold, an alarm can be triggered that indicates that it might be not possible to reconstruct the key anymore. Note that this testing is also able to detect fault attacks on the system as the error-rate also increases if an attacker is manipulating the system by, e.g., changing the supply voltage.



Chapter 5 Conclusion

The results of the different tests and analysis performed on the three project demonstrators point out the quality of the different MATTHEW concepts developed by the consortium during the last 3 years in accordance with the initial MATTHEW platform requirements.

The efficiency of ABC-based transaction, preserving the end user privacy and offering the possibility to anonymously transfer credentials between an Android smartphone and a ticketing server, has been clearly demonstrated here in the context of a transport application. To note that this security scheme could be totally transposed to any others application domain involving two or many multiple secure entities, opening the door to a large panel of new use cases. If a full software implementation of such privacy preserving algorithms basically shows pretty long transaction times, a real hardware implementation (like SIM card processing) performed in coming months, outside the MATTHEW project timeline, should definitively match the transport timings recommendations.

In parallel, this evaluation also demonstrated the capability for MATTHEW architectures to fit in small wearable devices, like smart watches or wristbands, thanks to the outstanding contactless performances provided by the ALM-Based secure nanoSIM and SiP platforms developed within the project. We demonstrated here that the MATTHEW contactless technology already match the requirements and ensure interoperability with worldwide ISO14443 standard -based contactless payment infrastructure, offering the opportunity for a future large deployment on this growing market.



Chapter 6 List of Abbreviations

ABC Attribute-Based Credential

ALM Active Load Modulation.

PCD Proximity Coupling Device.

PICC Proximity Integrated Circuit Card.

PUF Physical Unclonable Function

SAM Secure Access Module

SIM Subscriber Identity Module

Documents

D6.2 Report on application testing and evaluation · D6.2 - Report on application testing and evaluation MATTHEW D6.2 Page 9 of 37 Chapter 2 Test and Evaluation for payment application