Foreword - CNRgosweb.artov.isac.cnr.it/cmem_du_docs/CMP_Configur… · Web viewConfiguration Management is concerned with the unequivocal definition and unique identification of all

COPERNICUS MARINE ENVIRONMENT MONITORING SERVICE

CMEMS

MY DU

Configuration Management Plan

Reference: CMEMS-MYDU-CMP

Validated by: Vega Forneris (CNR) Document release number: 1.0 Date: 06-10-2017

Contributors : Vega Forneris (CNR), Antonio Novellino (ETT), Giuseppe Manzella (ETT)

MY DU Configuration Management PlanRef : CMEMS-MYDU-CMP

Date : 06/10/2017

Issue : 1.0

CHANGE RECORD

Issue Date § Description of Change Author

1.0 2017-10-06

all First version of document Vega FornerisAntonio NovellinoGiuseppe Manzella

COPERNICUS MARINE ENVIRONMENT MONITORING SERVICE PAGE 2/ 27


Date : 06/10/2017

Issue : 1.0

TABLE OF CONTENTS

I Introduction........................................................................................................................................................7

II Configuration Management Plan.....................................................................................................................8

II.1 Technical proposal overview and identification of configuration items....................................................8

II.2 Configuration control activity.....................................................................................................................12

II.3 Version control..............................................................................................................................................12

II.4 Configuration versioning.............................................................................................................................13

II.5 Configuration status auditing......................................................................................................................14

III Release and Transition Management.........................................................................................................16

IV Change Management.......................................................................................................................................23

V Problem Management......................................................................................................................................25



Date : 06/10/2017

Issue : 1.0

LIST OF TABLES

LIST OF FIGURES

FIGURE 1 – DU MY ARCHITECTURE................................................................................................10

FIGURE 2 – DU NRT&MY ARCHITECTURE.......................................................................................11

FIGURE 3: DU RELEASE MANAGEMENT SCHEME.........................................................................16

FIGURE 4: DU CHANGE MANAGEMENT SCHEME...........................................................................23

FIGURE 5: DU PROBLEM MANAGEMENT SCHEME........................................................................25



Date : 06/10/2017

Issue : 1.0

GLOSSARY AND ABBREVIATIONS

Additional terms:



Date : 06/10/2017

Issue : 1.0

Applicable and Reference Documents

Ref Title Date / Version

DA 1

DA 2

RA 1



Date : 06/10/2017

Issue : 1.0

I INTRODUCTION

Configuration Management is concerned with the unequivocal definition and unique identification of all configuration items in the WITS system, their constituent components and their respective interrelations. It has been built starting from Configuration Management process implemented by the Consortium partners in the framework of MyOcean projects.

The configuration items managed at WITS system level are:

Documentation (resident or reference) Software managed as resident: All software maintained by the consortium, Software managed as reference: Basic software (OSS, compilers…), Reused software (CFIs) and COTS (Note: COTS are identified by their name and version as

defined by the provider) Data (resident or reference), Hardware (reference)

The configuration item versions (N.n) are managed by the DU managers and recorded in a Configuration Control Sheet (spreadsheet) maintained by the DU Configuration Manager; other SW are under evaluation. Any and all changes to the configuration items, including their interrelations, are managed through revision control of the document. The Configuration Control Sheet will be available to Mercator Océan upon request.

System and code changes will tend to be managed via the Release Management process (cf. Chapter II). However, minor changes may be necessary outside of a major release. Such changes will be dealt with through the Change Management process, and the local DU managers will ensure that configuration update details are communicated to the DU Configuration Manager as part of the change.

Each DU partner has its own processes and tools for software configuration management (SVN, Subversion, CVS, or similar). The version information recorded in the Configuration Control Sheet will be traceable within the local DU versioning system.

DU Deputy will play the role of DU Configuration Manager.

The Configuration Management process ensures that the selected components (the Configuration Items) are identified, baselined, and maintained and that changes to them are duly controlled.

This control aims at verifying in particular that:

• each item is defined by applying the procedure described in the present document • each item is uniquely identified; • the design standard of the items is defined, traceable and retrievable anytime; • effective change control is established and maintained;

The present plan may evolve during the project lifetime and shall be equally applicable to the whole consortium



Date : 06/10/2017

Issue : 1.0

II CONFIGURATION MANAGEMENT PLAN

Key content is:

Identify Configuration Items (Hardware, Software, Documentation, Data…) and versioning rules

Describe Change Control Management (Request For Changes)

Describe how the status of a version is described (Version Description Document

Configuration Management is part of the quality control of a project and will provide control of the activities and activity outputs.

The major tasks will include:

Identify configuration items

o Hardware

o Software

Establish a coding system which will uniquely identify each item and final product

Identify the creator of any item, the reviser and final product version

Recording, monitoring and reporting on status of each item as it progresses through its own specific cycle

The identification of the configuration items will consist in giving a unique identification of each of them (including documentation) to allow the distinction of the different versions.

The identification of the configuration items will be followed by the control of the contents (configuration control and version control).

To allow also any follow-up of the configuration status, it will be assured the maintenance of all documents (including internal documents) that will constitute the history of the configuration.

II.1 Technical proposal overview and identification of configuration items

Covers: <<REQ-MY-DU-001>>, <<REQ-MY-DU-002>>, <<REQ-MY-DU-003>>, <<REQ-MY-DU-004>>, <<REQ-MY-DU-005>>, <<REQ-MY-DU-011>>, <<REQ-MY-DU-012>>, <<REQ-MY-DU-015>>, <<REQ-MY-DU-016>>, <<REQ-MY-DU-017>>, <<REQ-MY-DU-018>>, <<REQ-MY-DU-019>>

IMPORTANT: due to the lack of information about the Cloud Environment Mercator Océan will provide, the Architecture and Technical proposal are generic solutions applicable to all systems. The DU operator reserves the right and opportunity to re-analyse the solutions proposed based on MO inputs in order to improve the general Architecture Design (e.g. type and size of the overall resources at disposition, advanced storage features such as auto-tiering, data duplication, etc.)

The first main update since the actual DU system is to provide a single point of access (URL) for all data (NRT and/or MY). This will be performed by a load-balancer/traffic-dispatcher system used as unique frontend URL.



Date : 06/10/2017

Issue : 1.0

Behind this frontend, data is logically divided by typology into three main sub-DUs: Observation data (OBS), Marine Forecast data (MFC) and in-situ data (INS), each operated by a specific Partner who’s particularly expert in that field.

Each sub-DU will be composed by two (2) or more Virtual Machines (VM) connected to a common storage system/space, which assures the consistency of the interfaces without impacting on data volume (no duplication).

If allowed by MO’s Cloud System, VMs will be distributed on different physical hosts (at least three (3)).

Splitting the DU into sub-DUs (and more VMs per sub-DU) will improve:

performances - improved further in case enough physic hosts thanks to more independent CPUs and disks used

robustness - more components can crash without impacting the access to data

flexibility – changes on a sub-DU won’t impact others interfaces; VMs can be deployed/un-deployed depending on needs and resources available; new interfaces can be set up on same data without impacting operative tasks

tuning of interfaces - due to the different nature of the data, DU operator will apply specific best practise and even different SW/approaches (e.g. INS data won’t use THREDDS data server catalogue).

The scalability of the system is granted (and limited) by Cloud Environment

Concerning the interface DU-PU for gathering and delivering products/files, the DU operator will set up a Delivery Buffer Server (DBS), where PUs will upload files. As requested by the “INTERFACE REQUIREMENTS FOR PRODUCTION UNITS/ DISSEMINATION UNITS” (IR) document, a continuously running process will:

analyse the Delivery Note text sent by PUs

identify files by filenames & Delivery Note inputs

perform quality checks, such as data integrity, formats, etc., as foreseen by IR

manage files, delivering in proper storages, deleting superseded files, etc.

raise alerts in case of files rejection/problems.

measure the delivery time (against the expected one)

The DBS (“Error: Reference source not found”) will be configured for collecting logs from various servers/services, providing a single-access log-server, and graphical dashboards will be set up for quick looks (at least) on

data delivery statistics

data download statistics

service’s interfaces availability statistics

During the first year of the contract (see “Error: Reference source not found”), the DU operator will present a detailed solution for the “Disaster Recovery” system, a component which due to its particular requirements may present high costs in terms of budget and efforts. A sustainable alternative will be presented in section “Error: Reference source not found - Error: Reference source not foundError:Reference source not found”, keeping into account the need of a different and geographical separated environment.



Date : 06/10/2017

Issue : 1.0

Figure 1 – DU MY architecture

“Figure 1 – DU MY architecture” presents a simplified schema of the DU Architecture for one LOT.



Date : 06/10/2017

Issue : 1.0

Figure 2 – DU NRT&MY architecture

“Figure 2 – DU NRT&MY architecture” presents a simplified schema of the DU Architecture for both LOTs.

Each sub-DU will present all interfaces required by the call: Direct Get File (DGF), Subsetter (aka MOTU), WMS/ncWMS, FTP. All interfaces will present same variables, temporal extension and spatial resolution. Variables which will be declared as “not intented for visualisation” will not be exposed/accessible by users on advanced interfaces (WMS, MOTU).

Future evolutions will be implemented when needed (e.g. SOS, ERDDAP).

IMPORTANT: DU Operator cannot take the responsibility of the infrastructure operated by a separate entity.

In particular, the DU Operator will not be responsible for

HW (availability, performances, maintenance and upgrades); e.g. physical networks, storages and nodes. It includes any PSO/Maintenance the provider may perform.

Core Cloud management; e.g. set up & management of the virtualization layer

DU Infrastructure technicians/managers will operate and maintain the base layer of the DU: Virtual Machines set up (e.g. first phase, transition phases, etc), virtual network set up, virtual storage set up. They will set up and operate the main DU frontend, the DBS and dashboards connected.

XXX-YYY-DU-SD technicians/managers will operate and maintain upper DU layer: operating systems, CMEMS interfaces, specific procedures.

HW and physical devices (nodes, networks and storages) will be under IT Infrastructure technicians/managers responsibility and won’t be discussed/detailed further in this document. DU Operator will interact with IT managers/operators (once identified/communicated by MO) in order to define and set up cross-procedures.



Date : 06/10/2017

Issue : 1.0

State-of-the-art solutions will be provided: DU operator will use open source/customized SoftWare whenever possible, however, in order to match the high level of service required

II.2 Configuration control activity

This activity involves:

procedures for manipulating the items that constitute the configuration

procedures that allow the software configuration to evolve.

The configuration control activity will be done regularly in order to be reported during the six-monthly meetings. Configuration records will be produced by each partner involved in the implementation/development and maintained centrally by DUL and DUP in a working intranet. DUL and DUP will develop a library and will appoint a responsible librarian during the kickoff meeting, in agreement with MO.

Each implementation, development, configuration, document will have a naming convention including the version. The librarian will check the formal versioning.

The content of each version will be analysed by DUL and DUP in order to assure the adherence of each version with the global system.

In case of not-adherence, an ad-hoc meeting among DUL, DUL and responsible parties will be called, in order to agree on changes.

Implementation/Developments: working areas

Work areas per developers will be created to allow each developer to create and test the modules she/he is responsible for.

An integration area under control of DUL will be created. In it modules will be uploaded, tested and validated (by DUL and DUP). No new modules can be put in the integration area without re-testing being performed

A delivery area will be created, in which all modules constituting the deliverables will be kept.

II.3 Version control

The naming convention will be a code. The librarian will use a tool for the version control of each component. In case, he will be able to generate a new code (under request) using command files.

In the case of documentation, the librarian will assure that an obsolete version of the document is withdrawn and only up-to-date versions are used.

A name and a version/revision number identify any object managed in configuration. This identification is unique. The identification of the documents delivered in the frame of the WITS TAC follows a common approach in terms of Document filename.

The documentation naming convention is based on the following decomposition:



Date : 06/10/2017

Issue : 1.0

<ProjectAcronym>-<COMP>-<Type>-<X>.<Y>-<Extra information>.ext

Where:

<Project Acronym> will be CMEMS <COMP> is the entity who is responsible for the document, it will be DU <Type> is the document type, for example SRD, ADD, Monthly-Report-201507 <X> is the issue number <Y> is the release number <Extra information> extra textual information if required

Document issues Software configuration Versioning

The generic Configuration Item naming convention is based on the following decomposition:

<ProjectAcronym>-CI-<COMP>-<Partner>-<Unit>-<Item>[_Extra_information]

Where (refer to the above list for common acronyms):

CI = "Configuration Item" <Unit> = DU (Dissemination Unit) <Item> can be one of the following:

HW = Hard Ware SW = Soft Ware EI = External Interface II = Internal Interface

[_Extra_information] extra textual information for better identifying an Item, if required; e.g. WS = Web Service TDS = THREDDS Data Server MGA = MISGW Gateway Agent FTP = File Transfer Protocol IMS = Interface Monitoring System DBS = Delivery Buffer System (service) …

II.4 Configuration versioning

Configuration status versioning consists in recording, storing, archiving, distributing and presenting configuration identifications and variances with respect to the configuration baselines.

The status accounting function provides traceability of configuration baselines and of changes to these baselines.

Configuration Status recording activities consist in:

Recording all configuration management information and maintaining such information. Such information concerns:

o Technical identifications (numbers of items, documents, engineering change requests, etc.) and their relations,

o Actions concerning the process of technical events (approvals, engineering changes, incidents, interventions, document updating, corrective actions, etc.) and their implementation, with the associated planned and real dates.



Date : 06/10/2017

Issue : 1.0

Making such information items always available and transferable,

Presenting them in the form of views or printouts.

For software maintained by the Consortium, each Partner will use a software versioning tool server (eq. SVN, GIT, CVS, etc.) to manage these software. This server is under the responsibility of each partner. Using these tools, the configuration manager will be able:

To keep track of all the source codes generated in the frame of the project To tag a version related to any milestone of the project To manage versions of the codes,

A document history page will trace the history of the document including validation tasks and related document status and updates following corrections. The revision history table will contain:

the edition number

the revision number

a. the issue date

b. a description of the status of the document, being

an indication of a change to the document

one of the statuses: draft, final or accepted

an indication Insert/Replace

the pages to which the Insert/Replace action applies.

II.5 Configuration status auditing

Comparisons will be made between the records and the current physical representation of the configured items to ensure that they match, as part of the quality assurance process.

The documents that allow for tracing the configuration during development and maintenance are:

a periodically updated document that gives the complete list of the configuration with version numbers and related changes

a status accounting report that includes a summary or periodically updated document giving the list of all changes, stating the status of each of them and giving the date of the change request, the date of the actual change and the name of the item with its old and new version number.



Date : 06/10/2017

Issue : 1.0



Date : 06/10/2017

Issue : 1.0

III RELEASE AND TRANSITION MANAGEMENT

Release and Transition Management is primarily concerned with the planned evolutions of the DU system that are to be implemented in the regular releases of the CMEMS. It has been built starting from the Release Management process implemented by the Consortium partners in the framework of CMEMS service (first phase).

[Image to be updated]

Figure 3: DU Release Management scheme

Since releases are essentially changes to the system and products, the process is closely linked to Change Management (cf. Chapter III). All evolutions to the DU overall/sub systems, product suite



Date : 06/10/2017

Issue : 1.0

integration (new, retired products) and significant changes impacting on the DU in general will first be planned and approved by Mercator Océan, either via the Annual Activity Plan or by RFC.

The DU operator will be capable of dissemination during the transition phase of the release, as described/requested by SoW and will follow Mercator Océan indications.

The following tables show the activity plan and delivery plan as requested by the committing authority for the first 12 months (full activity plan and delivery plan is presented in an attached excel file).



Date : 06/10/2017

Issue : 1.0

M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12Task 1: Initial Settings of the dissemination environment setting operational infrastructure

DU-001-005/010 DU-006 DU-007 DU-021

making exisitng products available DU-008 software installation and configuration DU-009 DU 011

DU-012-013

DU-014-017

DU-018-020

interface with production unit DU-022 Task 2: Maintenance and evolution of the dissemination environment maintenance of dissemination environment

DU-023-027

DU-023-027

evolution of the dissemination environment

DU-081-083

Task 3: Integration of the catalogue versions setting integration infrastructure

DU-028-031

DU-032-033

integration activities DU-034-036

Task 4: Operations Gathering and checking the input data & products

DU-037-038

DU-039-040

make products available on dissemination interfaces

DU-041-045

service desk DU-046



Date : 06/10/2017

Issue : 1.0

DU-052-057

double dissemination DU-204

DU-206-207

product delivering monitoring, dissemination monitoring and security monitoring

DU-058-059 DU-060

DU-062-065, DU-067

DU-061,DU-066

DU-069; DU-071-072 DU-068

DU-073

DU-074Task 5: Management and quality assurance Management of infrastrucutre activities DU-75-080management of integration activities

DU-084-085

management of operations DU-086-090

DU-100-108

DU-100-108

DU-100-108

DU-100-108

DU-100-108

DU-111-112

DU-113-115 DU-110

DU-116-121



Date : 06/10/2017

Issue : 1.0

DU-120 DU121-124

DU-125

global managementDU131-132

DU-133-136

DU-133-136

DU-133-136

DU-133-136

DU137-138

DU139-142

DU143-144

DU150 DU149

DU-151-152

DU-151-152

DU-151-152

DU-151-152

DU-023-027 trimonthly report based on monthly monitoring activities, updates etcDU-036 the report will provide the contracting authority with daily information for the period covering pre-integration and integration phase.DU-052-057 trimonthly report with monthly information as defined by the REQDU-207 is intended as the starting month from which the sample can be requested by the contracting authorityDU-058-059 are intended as the month from which the service will be in place. It includes DU-208-211 also.DU-062-065, DU-067-068. intended as the month from which the service will be in place. It includes DU-212-213 also.DU-069; DU-071-072 including DU-214-218DU-75-080 intended as the month from which the service is in place and accessible by means of the dashboardDU-091-97 is intende as the trimonthly report on the activites as specified by REQDU-109 is intended as the last month to be implementedDU-113-115 is intended as the starting month from which the service



Date : 06/10/2017

Issue : 1.0

will be in placeDU-125 is intended as the starting month from which the service is in place

Table 1.Activity Plan



Date : 06/10/2017

Issue : 1.0

month form the start of the project

Listo of the deliverables

M1 DU-001-005/010 DU-100-

108DU-111-112 DU-120 DU131-132 DU137-138 DU150

M2 DU-006

M3 DU-007 DU-009 DU-100-108

DU-133-136

DU-151-152

M4 DU 011

M5 DU-021 DU-008 DU-012-013

M6 DU-014-017

DU-100-108

DU-133-136

DU-151-152

M7

M8 DU-018-020 DU-022

M9 DU-023-027

DU-028-031

DU-037-038

DU-058-059

DU-062-065, DU-067

DU-069; DU-071-072

DU-100-108

DU-133-136

DU-151-152

M10 DU-034-036 DU-041-

045 DU-046 DU-113-115

M11 DU-204 DU-206-207

M12DU-023-027

DU-081-083

DU-032-033

DU-039-040

DU-052-057 DU-060

DU-061,DU-066

DU-068 DU-073 DU-074 DU-75-080 DU-084-085

DU-086-090 DU-100-108 DU-110 DU-116-121 DU121-124 DU-125 DU-133-136 DU139-142 DU143-144 DU149 DU-151-152

Table 2. Delivery Plan



Date : 06/10/2017

Issue : 1.0

Changes/Evolutions will be discussed among partners under the supervision of the DU Leader and the DU Change manager (if different). The DU operator will set up Test environment (and dismantled once completed) and will perform tests in order to check and assure the correct functionalities. The DU operator will provide Test Reports as proof of successful (or problematic) evolutions and will submit to Mercator Océan for the acceptance (Implementation “go/no-go”).

In addition to the system change itself, the DU operator will provide all required documentation and support to Mercator Océan for implementation of the change.

Changes impacting both DU and PCs will be performed thought dedicated interfaces (PU-DU) for easing the transition (tech constraint vs science constraint). The DU operator will provide technical testing, Test Design Documents, support to PCs/PUs, Tests Summary Reports. The DU operator will provide support to the CMEMS integration team for all verification activities performed by during the integration phase of each release.

The DU Leader (CNR) with the support of DU Deputy (ETT) will be the Release Manager and will be supported by the DU managers involved in that specific change. DU Technical Support will play a key role in ensuring the release is successful with no detrimental impact to the quality of service delivered.

If during the implementation the project is needing any change management the consortium will follow a defined procedure and workflow as described in the following section.



Date : 06/10/2017

Issue : 1.0

IV CHANGE MANAGEMENT

The DU Change Management process has been built starting from the process implemented by the Consortium partners in the framework of CMEMS service (first phase).

The Change Management process is described in the figure below.

ADAPT/UPDATE IMAGE

Figure 4: DU Change Management scheme.

All requests for change will be submitted to the DU Change Manager, who confers with the appropriate managers (e.g. specific sub-DU, SW evolutions, etc.); they are responsible for ensuring that changes within their product domains are analysed and justified. Appropriate manager will prepare [R|N|C]FC documents (Request/Notification/non-Conformity For Change) which will be recorded in the DU Change Management System. If approved internally, DU Change manager will interact with the CMEMS Top Level Change Mailbox, through the online system Mercator Océan will indicate (most probable: JIRA system in operation in first phase) .

The X Change Form will detail:

the nature of the change,

full details of the change being made,



Date : 06/10/2017

Issue : 1.0

who will perform the Change and when

the item it will impact (and how),

a risk assessment and an implementation plan to, including an estimated date and time for carrying out the change and a backout plan in the event of unexpected issues caused as a result of the change.

Every change will be assigned a unique reference and will be related to the subsystem infrastructure being changed wherever possible. If the change is associated with a significant level of risk, then the DU will provide evidence that all appropriate testing has been undertaken prior to the change to mitigate the risk. The CMEMS Service Desk will then be able to inform users and/or producers (depending on the change) as per the communication plan provided. Upon completion of the change, the DU will confirm to the Change Manager that the change has been successful with no remedial impacts before notifying the Change Mailbox. The DU Change Manager is responsible for overall oversight of changes in the DU systems and ensuring that the change procedures are followed.

Wherever possible, and for significant product changes, a period of parallel running will be implemented and test files offered to users to ensure that the change is successful.

There are three main sources of change:

System changes are raised locally by DU internal operations teams (DU Managers) and are generally responses to anomalies in ongoing operations or potential evolutions of the system (one or more components). The DU Change Manager will take into consideration whether the changes involve a system identified as having relational dependencies with other CMES services, and in this case will activate interfaces, and discuss about that timescales.

System changes may originate from outside the DU system, e.g., from user requests/feedbacks, from changes in the Central CMEMS System, etc., The externally initiated changes are raised at the request of the DU Leader and/or the DU Change Manager with the relevant DU Managers..

Major changes will tend to follow the Release Management process. They will be part of the Management Plan. All other changes will be handled by Request for Change (RFC). In either case, the change will ultimately be subject to approval by Mercator Océan.

All change requests and notifications will follow the forms provided by Mercator Océan. The Change manager will record and update the Change Form and will update the online system as requested by SoW. The DU Leader will be consulted on changes where there is likely to be significant service impact.



Date : 06/10/2017

Issue : 1.0

V PROBLEM MANAGEMENT

The CMEMS DU Problem Management process has been built on the process implemented by the Consortium partners in the framework of CMEMS projects.

The draft DU Problem Management process is described in the figure below.

[Image to be updated]

Figure 5: DU Problem Management scheme

Identification of a problem is commonly the result of escalation of an incident, usually a recurring incident, but may also be identified in the service provisioning. The DU Incident, Service or Operation Manager will, in concert with the relevant DU Manager, request escalation by the DU Service



Date : 06/10/2017

Issue : 1.0

Manager. Once a problem is identified, the Problem Manager creates (and owns) an appropriate ticket and informs the DU Service Manager. The Problem Manager appoints a problem task team whose task is to analyse the problem, propose a solution plan and pursue the task to resolution. The problem may be functionally escalated, if necessary, to a central CMEMS Element, e.g., CIS.

If a solution to the problem is found and it requires changes to the service, the DU Service Manager files an RFC (Request for Change) form. In some cases, a workaround may be employed as an interim solution. The CMEMS Service Desk solicits appropriate actions at top level and informs users, if necessary. When the changes are performed or if the workaround becomes the standard solution, the ticket is closed.

The process of problem investigation will generally follow that for incidents, but will tend to be slower in time. Collective discussion, investigation and testing with DU partners and internal support teams may also be required to resolve the underlying problem.

In the case of problems arising outside DU, the DU Leader will help identify the most appropriate people to assist with investigation. If appropriate, the DU Leader may delegate the responsibility for handling a problem to another (e.g., DU Problem/Incidents/Service/etc. Manager).
