View
5
Download
0
Category
Preview:
Citation preview
RESULT SUMMARY OF 4TH ITS COOPERATIVE MOBILITY SERVICES PLUGTEST
© ETSI 2011. All rights reserved
Sebastian Müller 27.3.2015 ETSI Centre for Testing and Interoperability ITS Workshop, Helmond (NL)
Plugtest Organization
© ETSI 2011. All rights reserved 2
Organizer Test Tools
ITS modem for ETSI Test System Supporter
ETSI TC ITS Test Activities
© ETSI 2014. All rights reserved
Conformance Validation
Framework
ITS Test Framework ETSI EG 202 798
Test Architecture ETSI TR 103 099
CAM Tests ETSI TS 102 868
DENM Tests ETSI TS 102 869
GeoNetworking Tests ETSI TS 102 871
Open Source Platform
Standardized test system interfaces for
test execution, logging, and adaptation
Free tests, codec and adaptation
Tests implemented in the international
standardized testing language TTCN-3
ITS Plugtest 2015
Interop Tests
Conformance Tests
C2C CC Validation
Real Time Channel Simulation/Replay Tests
ETSI Plugtest Live Demo
65 person month of effort invested since 2010
ETSI Conformance Test System
CAM/DENM/GN and Security Tests with TESTINGTECH runtime environment ITS modem support from COHDA and COMMSIGNIA
© ETSI 2011. All rights reserved 4
Interop Test System
© ETSI 2011. All rights reserved 5
Attenuator matrix is controlled by the QOSMOTEC QPER software Message flow between the DUTs is supervised by a monitor device and displayed in the VECTOR CANoe software
Summary of test infrastructure
More than 20 vendors ! • 3 Open Source Implementations (see Annex)
ETSI provided Wireshark dissector for all ITS protocols including Security to assist the debugging Security PKI framework provided for Conformance and Interop tests via ETSI Event WIKI • Release of ETSI Security Conformance Tests via ETSI Forge available
since Feb 2015
4 ETSI Conformance Test Systems 3 Interop areas for face-2-face 2 Forwarding setups for 2 x 3 devices Each vendor scheduled for all tests 5 test session chairs
© ETSI 2011. All rights reserved 6
Interop Issues GN and DENM
GeoNetworking • Sender Position Vector vs Source Position Vector
(SePV deleted from packets but still used in algorithms)
• GBC Advanced Fwd Algorithm not often supported
DENM • Keep Alive not often supported
© ETSI 2011. All rights reserved 7
Interop Issues Security
No Profile 3 tests were executed due to the open base spec issue on ITS-AID usage for messages other than CAM and DENM Pseudonym Change needs to be defined better Some ITS stations did not send long packets (694 byte signed CAM including certificate chain) • TS 102 687 (DCC Access Layer) restricts maxPacketDuration
to 0,6 msec at 6MBit/sec Sender CAM Certificate inclusion • Reaction to request and unknown digest • Timing
Exceptional behavior tests very useful for vendors
© ETSI 2011. All rights reserved 8
ETSI Plugtest Live Demo
© ETSI 2011. All rights reserved 9
Demo system • 2 RSUs and 4
OBUs • HMI on Tablet • Set up under 2
days V2I use-cases: • Road Hazard
Warning • Traffic Jam
Ahead Warning V2V use-case: • Approaching
Emergency Vehicle
Vehicle 1: Vialis Vehicle 2: ITRI Vehicle 3-4: Commsignia
RSU 1: Commsignia RSU 2: Vialis
Results
All vendors had ITS modem with radio interface (no more SW prototypes) Most of the vendors have implemented Security Most of the vendors have implemented ETSI Upper Tester Interface 20% of all vendors passed all conformance tests (security included) More than 2600 tests executed Only 22 potential base spec issues identified (see Annex) Devices ready for deployment! © ETSI 2011. All rights reserved 10
PHY Performance Evaluation by HHI
© ETSI 2011. All rights reserved 11
Statistical Channel Emulation • Based on ETSI ITS channel models • Covers main characteristics of the
doubly-selective vehicular radio channel, e.g. multipath and Doppler
Replay Channel Emulation • Based on channel measurements
conducted by HHI • Covers exact channel properties of
selected measurement runs Interface to receive packet count
Channel Emulator
Statistical Model
Device B
Replay Data
Device A
Next step: Feedback to ETSI ITS WG4 on channel models (ETSI TS 103 257)
Example Result from HHI
© ETSI 2011. All rights reserved 12
Motion Emulation by octoScope/TASS/SAVARI
© ETSI 2011. All rights reserved 13
Emulate motion of vehicles and attenuation Monitoring BER performance Next Step: Demonstrate potential hardware in the loop test capabilities including application performance and behavior
Prescan Scenario
generator
Octoscope testbed
attenuator ITS G5 RF
signals position updates
octoBox
octoBox
DUT
DUT
Distances between DUTs
Mobility Emulation Scenarios by Qosmotec and TASS
© ETSI 2011. All rights reserved 14
Scope of next Plugtest
Test only features relevant for Day 1, e.g. no GeoUnicast, no TSB , no KeepAlive , i.e. like C2C CC Profile Develop security tests for exceptional behavior of DENM and conformance of certificates Traces / PathHistory . Test that data is interpreted correctly Include road operator in test infrastructure Robustness tests Extend demonstrated channel simulation scenarios of 3 DUTs to • dynamic channel simulations with up to 6 DUTs • emulation of real traffic scenarios
DCC tests (Sensitivity threshold correction, energy threshold test, header decoding test, DCC stability) Out of band emission + CAM Coexistence feature Drive Tests
© ETSI 2011. All rights reserved 15
Concerns
No PKI deployment plan/protocols available (for infrastructure) Pseudonym Changes / Anti finger printing No Profile 3 tests were executed due to the open base spec issue on ITS-AID SPAT/MAP needs to be made available soon DCC needs to be made available soon Make C2C CC profile available to all vendors of next plugtest (use only 1 profile in Europe for C2C Communication) Create an European wide compatible C2X profile © ETSI 2011. All rights reserved 16
On behalf of the Plugtest Team
© ETSI 2011. All rights reserved 17
Thank you!
ANNEX
18 © ETSI 2011. All rights reserved
ITS Vendors
© ETSI 2011. All rights reserved 19
Autotalks Cohda/Nxp Commsignia Ctag Denso *I-GAME (TNO + Viktoria)
Hitachi/Renesas Imtech Itri Kapsch
Marben Neavia Nec Nordsys Novero Qmic Siemens *Trialog Unex *Yogoko Vialis
* Partly Open Source
Issues GN EN 302 636-4-1 v1.2.1
Typo in 9.3.12.2 Source operations – Issue6982 ambigiuous description in 9.3.8.3 Forwarder operations – Issue6983 error in 9.3.12.2 Source operations – Issue6984 DPD in 9.3.11.3 forwarder and receiver operations – Issue6985 GN protocol constant itsGnGeoAreaLineForwarding is set to TRUE – Issue6986 clock drifting – Issue6987 Restructure of FWD algorithm – Issue6988 Duplicate Packet Detection – Issue6989 MAC and GN ID – Issue6990
© ETSI 2011. All rights reserved 20
Issues Security TS 103 097 V1.1.20
Uncomplete description of creating hash for signature of secured messages – Issue6950 Wrong description of considering the public key algorithm in the signature of CAMs – Issue6949 Wrong description of considering the signer info field in the signature of CAMs - Issue6948 ITS-S should accept request of unrecognized AA certificate from untrusted sender – Issue6972
© ETSI 2011. All rights reserved 21
New feature for next revision of Security TS 103 097
Request of unrecognized AA certificate create high channel load when all receivers reply with a certificate chain – Issue6969 Security link between CertID and relevant IDs (Pseudonym change) – Issue6979 ITS-S should stop requesting an unrecognized AA certificate if the issuer of the AA certificate is untrusted – Issue6973 Request of unrecognized AA certificate create high channel load when all receivers reply with a certificate chain – Issue6969 Do we need certificates in CAM each second – Issue6976 add example how to define time – Issue6978 Security link between CertID and relevant IDs – Issue6979
© ETSI 2011. All rights reserved 22
Issues DENM EN 302 637-3 v1.2.2
pseudonym change for active DENMs – Issue6980 StationID collisions - Issue6981
© ETSI 2011. All rights reserved 23
Useful Links
ETSI Certificate Toolkit (Generator, SignatureVerifier) http://forge.etsi.org/websvn/ITS/branches/STF484_VALIDATION/tools/itscertgen/ FOKUS SecureMessage & Certificates Validator https://werkzeug.dcaiti.tu-berlin.de/etsi/ts103097 Link to Wireshark dissectors http://forge.etsi.org/websvn/WIRESHARK_ITS_PLUGINS/releases/wireshark-1.12.x/ Link to ETSI conformance framework http://forge.etsi.org/websvn/ITS/branches/STF484_VALIDATION/ ETSI GPSD Server http://forge.etsi.org/websvn/ITS_GPSD_SERVER/ Blog http://www.etsi.org/news-events/past-events/846-plugtests-2015-itscms4?tab=3
© ETSI 2011. All rights reserved 24
Contact
© ETSI 2011. All rights reserved 25
Sebastian Müller
Project Manager ETSI - CTI
T: +33 4 92 94 43 13 | M: +33 6 74 40 83 66 |
E: sebastian.mueller@etsi.org
Recommended