Embed Size (px)
Citation preview
Guideline to Set Up a Joint Test Laboratory for Testers and their EquipmentGitte Oberbossel
Rambøll, but works in a Banedanmark project
#EuroSTARConf
Agenda
• Short history of the Signalling Programme and introduction to Joint Test Lab (JTL) testing
• JTL setup• JTL setup and tips on how to make it work.
• Year wheel – to keep the Laboratory up to date
• Last tips
The Signalling Programme and JTL- Short Introduction to the Biggest Software and Infrastructure Project in Denmark, and How JTL Is Setup Today
The History of the Signalling Programme (SP)
➢Replacement of all signalling on the entire Banedanmark railway network, with the standard “ERTMS” by the end of 2030
➢ It was initiated by the Danish Parliament in 2009.
➢The railway system East and West + Onboard (Trains) are tested in JTL; S-Banen (Intercity) is not tested in JTL.
➢A total renewal of all signalling on a national rail network, in a single programme, has never been attempted on this scale before in Europe
➢Denmark will be the first country to carry out a country-wide implementation of an ERTMS level 2, baseline 3; the newest European common signalling standard.
➢Roll-out is split into lines, and we are typically running 3 lines in parallel ➢This means that we are also testing 2-3 lines in parallel, in JTL.
The History of the Signalling Programme (SP)
This includes, in short:
➢3 Railway suppliers (Alstom, Thales and Alstom - Onboard (Train) and Netcompany (ESB), Eltel (Network)
➢53 IT interfaces to be created against current the systems
➢An ESB (Enterprise Service Bus) platform ➢ In short, an intelligent gateway between IT interface and the Railway Core system.
➢A new GSM-R incl. voice and GPRS network (Radio communication)
➢A DK-STM (Integrations with current train communication system)
➢A new Fixed Transmission Network (FTN) to be implemented
➢173 new operational rules to be used by all future users
Simple Solution Overview of SP and Scope of the Joint Test Lab (JTL) Test
TMS
Point Machine
AxleCounter
LX/PWS/SC
IXL RBC
Balise
SIL2
TMS
Point Machine
AxleCounter
LX/PWS/SC
RBC IXL
Balise
SIL2
On-lineScheduler
KeyManagement
Server
Long TimePlanner
DecisionSupportSystem
Hand Held Terminal
Hand Held Terminal
TRIS
EnterpriseService
Bus
External
GSM-R
Integration between different
suppliers
Who Is Using the JTL and What Is Being Tested?Who is using the JTL laboratory?
➢All Railway suppliers
➢Banedanmark test
➢External interface integration testers
➢ESB integration testers
➢Banedanmark Pre-Site testers
➢Network (FTN) testers
Other “stuff”
➢Demos, inspections, witnessing
➢ISO27002 review
➢Audit by the Danish Transport Authority, “Trafikstyrelsen”
➢Visits from other countries
What is being tested?
➢All integration between the different deliveries from Supplier and Banedanmark
➢End-to-End tests
➢Customer acceptance tests
➢Customer Pre-Site tests
➢External solutions system integration tests
➢Performance tests
➢IT Security tests
➢Failover/recovery tests
➢Banedanmark operational rules tests
➢Banedanmark usability tests
➢Maintenance processes tests
Photos from the JTL House (492 m2)
Server Room
The Unit Sitting in the Trains
Train Antennas on the Roof
TrIS – Passenger Info System
Test Room Test Office + Canteen
Level Crossing, Cooling, Diesel Generator
Level crossing
Server Room Server Room
Server Room
JTL House
Balcony
Stairs
Room 210
Room 209
Room 227
Room 208
Room 207
Room 211
Room 203Room
204
Room 202
Room 206
Room 218
Room 219
Room 220
Room 221
Room 213
Room 222
Room 212
Room 214
Room 225
Room 224
Room 211a
= Meeting Room= Rack in Server Room
= Tester Room - Desk
Kitchen
Sho
we
r
LiftStairs
Room 201
Printer
Sto
rage
Storage
= Circuit Brake Panel
UPS
NitrogenAircon Aircon Aircon Aircon
= Fire Extinguisher
Thales Level Crossing
Diesel Generator
JTL Stakeholder - Organisation
Interface with Projects + the TSAs
JTL
East (Alstom) West (Thales)Onboard
(Alstom)FIT
IT External
interfaces DSB
Trafik-
vërketSTM
Netcom
-panyPVVRingsted TRIS
Programme Management Team
JTL Infrastructure
(Network+Servers)
Nokia
Eltel
TEKNIK
Coromatic
FIT
JTL Maintenance/Pre-Prod.
• Alignment
• Environments in JTL
Nokia
Eltel
Banedanmark
CoromaticFIT
JTL House Facilities
BanedanmarkWork
Environment
DSB
(House Owner)
Banedanmark
Solution owners of the
system (TSA)
Safety &
Engineering
Operations
Management
Group
Site Test
Team
Signalling Programme JTL Laboratory & Tool Test Team Stakeholders & Interfaces
JTL Test Factory- Creating a Test Factory Model
To Be Used by 5 Suppliers and Banedanmark, including 16 Different Nationalities
Quick guidelines
JTL Is Growing and Continuously Changing from year 2013-2030➢Due to iterative development, JTL is continuously changing and is growing
➢This means that nothing is stable:
➢Test scope and content changes:➢Tests get more and more complex, until the final functionality delivery. Afterwards, the focus is on the
railway line configuration, incidents found in production, and new project changes.
➢Resources change for both testers and test managers, and they are ‘railway people’:➢They are experts on the railway system, not test models and standard test methodology.
➢Equipment is being replaced, or more is coming.
➢Network is changing: ➢Productions, or the JTL Supplier, change the network IP address and routing strategy, etc., for new
releases.
JTL Test Phase
JTL test phase – split into x number of test cycles
➢ JTL runs in accordance with a joint test schedule, which constitutes all tests being grouped into test cycles.
Test Laboratory
Elements
Laboratory House Facilitation• Everything within house facilitation• Work environment
JTL Infrastructure Document Network + Server• Design of the infrastructure and rules• Management of infrastructure • Process for updates
JTL Test Tools • Management• Support/guidelines • Define approach and strategy
Contract and Licence Management• Contract Management• Licence Management
Monitoring, Quality and Status • Communications plan• Quality check• Statistical reporting of status and quality
Laboratory Test StrategyLab test strategy and processes used for testing
Test Organisations • Roles and responsibility • People
Test Schedule & Scope • Test schedule • Test scope • Conflict monitoring and management
IT Security strategy• Document ISO27002 area• Rules and process
Defect Management • Defect process• Process for analysing defects• Defect boards
Test Meetings and Stakeholder List• Meeting plan• Meeting facilitation • Stakeholder list and management
Test Laboratory Configurations Management • CM within each Test Cycle• CM baselines
Elements of the “Test Factory”➢The “test factory” is running for all test cycles, for each test cycle’s content,
configurations and setup/installations that are changed
JTL Test Strategy
➢All tests in JTL are grouped into test cycles, which are typically
new baselines, including all Supplier and Customer tests
➢Make a joint test process used by everyone and create a test strategy
➢Tip!➢Create a test process that is used by everyone, and then have a chapter in the strategy that defines
how each step is done (rules, process, etc.). Have a standard template for meetings, test reports, etc.
➢Make quick guidelines and hang them up in the house and in the program location.
➢Make alignment with contracts and put the requirement into the test strategy
➢Tip! ➢For each chapter in the JTL strategy, list the contract requirements that support this.
➢This saves time by not discussing if the Supplier must do it based on contract, or not.
➢Make standard templates to be used where possible, including the contract requirements, and include them in the templates as a checkpoint.
➢E.g.: Contract ID 2.35 Test procedure must be reviewed and approved – template question: Is the test procedure reviewed and approved?
Templates
JTL Test Process
JTL Schedule ➢The JTL Schedule is the “heartbeat” of JTL; it documents ALL test activities.
➢What test is done and who is responsible for each test?
➢Who supports each test? The status of support is confirmed or rejected.
➢What equipment is used? Status for possible conflicts.
➢What baseline version is tested?
➢What type of “train” is the Onboard (Train unit) setup to run?
➢JTL Schedule has a process and a guideline
➢JTL Schedule “next 8 weeks” walkthrough on the Tuesday
status meeting
➢For requesting tests to be inserted into JTL schedule, there
is a template
➢All new Test Managers get an introduction in the Schedule. Quick guideline also exists. Tip! Important activity
Template
IT Security – Setup and Maintenance➢Create an IT Security policy and validate it each year, to ensure it is still valid
➢ TIP! In this process, create sub-documents within relevant areas, and templates to be used. E.g.:
➢ Guideline for handling and maintaining test servers in JTL, and a template to be used for handling a server.
➢ Backup/Restore policy, planned quality tests with Backup/Restore – using standard test report templates for this.
➢Use the ISO27002 standard as your checklist
➢ The Information Security standard ISO/IEC 27002:2013 is the “Code of Practice for Information Security Controls”.
➢ The standard contains 14 security control clauses, collectively containing a total of 35 main security categories.
➢ Information – Training in JTL IT Security TIP! It is important to ensure you can continuously train new users.➢ Adhoc users of JTL - give them the Quick Guideline.
➢ If they spend more than 60% of their working time in JTL: 1-hour introductory presentation of the JTL IT security policy.
➢Yearly ISO27002 review➢ To keep the IT Security rules up to date, and to make sure they are complied with, put in yearly checks.
➢ Cabling, labelling, and equipment review – do a house walkthrough.
➢ Review of Configurations Management in JTL.
➢ Review of server setup.
➢ Review of Licence policy to ensure it is kept and all licences are valid.
➢ Review of access permission to house, equipment, and AD (Active Directory). Done every 3 months.
IT Security – Setup and Maintenance➢Tip! Create an asset list for both physical requirements and virtual servers.
Tip! Include it in the request form for installing equipment in JTL, to get the information.
IT Security –Tip for Setting It Up
ISO27002 Checklist, clause 11
“Removal of assets”
JTL IT Security policyChapter 7, Clause 11.
“Removal of assets”
JTL Infrastructure and Server➢Define the network design for the laboratory
➢Make alignment with production setup
➢Network design documentation – keep it updated!
➢Network drawing. ➢TIP! Hang them up – then they are easy to use for everyone.
➢Define a cable-, naming-, and labelling-policy
➢Other areas to be defined➢Process for installation or replacement of equipment.
➢Tip! This includes how to handle all the cardboard – IT IS A LOT!!
➢How to get rid of old equipment.
➢Access policy, both physical and local, for equipment.
➢Maintenance, e.g., Backup/Restore and IT Security scan.
➢Monitoring of IT equipment and how to react regarding errors:➢Tip! Set up monitoring screens, then everyone can see if there are any failures.
House Facilitation & Work Environment
➢Different countries, different working hours
➢When is the morning? Depending on the country, the answer will be e.g., 8:00, 9:00, or 9:30.
➢Tuesday meetings – an important part of working together
➢What is going to happen this week, who is coming, and when are we working together?
➢Work environment walkthrough – take a walk and ask if there are any issues
➢Lessons learned every 6 months – everyone who has used JTL in the last 6 months, for more than 6 days, are invited
➢Each person mentions 5 good things and 5 things that need improvement, and then we talk about it. On the next meeting, there is a follow-up.
➢E.g.: Last time, based on the meeting, we changed how things are setup in the test room.
➢Tip: Important to focus on:
➢Noise – it is possible to use noise-cancelling head phones.
➢Continually following up on keeping the house clean.
➢Handling of closed and open windows.
House Tips
Balcony
Stairs
Room 210
Room 209
Room 227
Room 208
Room 207
Room 211
Room 203Room
204
Room 202
Room 206
Room 218
Room 219
Room 220
Room 221
Room 213
Room 222
Room 212
Room 214
Room 225
Room 224
Room 211a
= Meeting Room= Rack in Server Room
= Tester Room - Desk
Kitchen
Sho
we
r
LiftStairs
Room 201
Printer
Sto
rage
Storage
= Circuit Brake Panel
UPS
NitrogenAircon Aircon Aircon Aircon
= Fire Extinguisher
Thales Level Crossing
Diesel Generator
Year Wheel -Keep Your Lab Up to Date with the Needs of Today and the Future
Year Wheel ➢All processes, rules, and models must be kept up to date
➢Have a yearly validation; update where necessary, and remove what is now obsolete.
➢Clean up SharePoint – make old documents obsolete.
➢Create a year wheel➢Tip! Align the year wheel with next year’s scope and budget definitions
➢ June – July ➢ Licence validations.
➢August – September➢ Define the scope of next year.
➢ Equipment / network strategy for next year – are any updates needed?
➢ Team organisations – roles, activities, and workshops within the JTL team.
➢October – November➢ ISO review – action list.
➢ Define budget and final scope.
➢ Update test organisations and role definitions.
➢December – January ➢ Document validations and clean-up, including review and approval if needed.
Last Tips
Last Tips!
➢Most importantly, keep it happy and “hyggeligt” in the Lab - if people are happy, it is possible to solve everything and do great work
➢Continuously keep a firm grip on the budget➢Teach all suppliers that we “don’t pay overtime”. We agree on the used hours and the
price for all of the work to be done, before doing the work.
➢Have a process for review and approval of new offers, and get more than one review of the offers.
➢If you see suppliers starting to disagree – go in and be the facilitator until it is solved
➢Talk to the leaders if failures happen, and the Supplier doesn’t
know where the failure is – initiate a Task Force
Thank You!
Gitte Oberbossel
Rambøll, but works in a Banedanmark project
#EuroSTARConf
