167
Private Public Partnership Project (PPP) Large-scale Integrated Project (IP) D.21.1.2: Platform deployment, operations, analytics and support tools Project acronym: FI-Core Project full title: Future Internet - Core Contract No.: 632893 Strategic Objective: FI.ICT-2011.1.7 Technology foundation: Future Internet Core Platform Project Document Number: ICT-2013-FI-632893-WP21-D.21.1.2 Project Document Date: 2016-09-30 Deliverable Type and Security: PU Author: Silvio Cretti (CNET) Contributors: Miguel Jimenez (UPM), Sergio Vavassori (UPM), Joaquin Iranzo (ATOS), Fernando Lopez (TID), Pablo Rodríguez (TID), Jesús Pérez González (TID), Henar Muñoz Frutos (TID), José Ignacio Carretero Guarde (TID), Daniele Pizzolli (CNET), Daniele Santoro (CNET), Katarzyna Di Meo (CNET), Kenneth Nagin (IBM), Joaquin Salvachua (UPM), Gabriel Huecas (UPM), Pedro Verdugo (UPM)

forge.fiware.org · Private Public Partnership Project (PPP) Large-scale Integrated Project (IP) D.21.1.2: Platform deployment, operations, analytics and support tools Project acronym:

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

  • Private Public Partnership Project (PPP)

    Large-scale Integrated Project (IP)

    D.21.1.2: Platform deployment, operations, analytics and support tools

    Project acronym: FI-Core

    Project full title: Future Internet - Core

    Contract No.: 632893

    Strategic Objective: FI.ICT-2011.1.7 Technology foundation: Future Internet Core Platform

    Project Document Number: ICT-2013-FI-632893-WP21-D.21.1.2

    Project Document Date: 2016-09-30

    Deliverable Type and Security: PU

    Author: Silvio Cretti (CNET)

    Contributors: Miguel Jimenez (UPM), Sergio Vavassori (UPM), Joaquin Iranzo (ATOS),

    Fernando Lopez (TID), Pablo Rodríguez (TID), Jesús Pérez González (TID), Henar Muñoz

    Frutos (TID), José Ignacio Carretero Guarde (TID), Daniele Pizzolli (CNET), Daniele Santoro

    (CNET), Katarzyna Di Meo (CNET), Kenneth Nagin (IBM), Joaquin Salvachua (UPM), Gabriel

    Huecas (UPM), Pedro Verdugo (UPM)

  • D.21.1.2: Platform deployment, operations, analytics and support tools 2

    Table of Contents

    Table of Contents .................................................................................................................................... 2

    1 Introduction .................................................................................................................................... 8

    2 FI-Deploy ....................................................................................................................................... 10

    2.1 OPS-Deploy ........................................................................................................................... 10

    2.1.1 Description .................................................................................................................... 10

    2.1.2 Features implemented .................................................................................................. 11

    2.1.3 Installation manual ....................................................................................................... 11

    2.1.4 Installation verification ................................................................................................. 12

    2.1.5 User manual .................................................................................................................. 13

    2.1.6 API documentation ....................................................................................................... 26

    2.1.7 Known issues ................................................................................................................. 26

    2.1.8 Link to the software repository..................................................................................... 27

    2.1.9 License ........................................................................................................................... 27

    2.2 Keyrock to Keystone Federation ........................................................................................... 27

    2.2.1 Features Implemented .................................................................................................. 27

    2.2.2 Installation Manual ....................................................................................................... 28

    2.2.3 Installation Verification ................................................................................................. 29

    2.2.4 User Manual .................................................................................................................. 30

    2.2.5 API Description .............................................................................................................. 30

    2.2.6 Known Issues ................................................................................................................. 30

    2.2.7 Link to the Software Repository ................................................................................... 30

    2.2.8 License ........................................................................................................................... 30

    3 FI-Dash .......................................................................................................................................... 31

    3.1 FIDASH - FIWARE Lab Dashboard .......................................................................................... 31

    3.1.1 Description .................................................................................................................... 31

    3.1.2 Features implemented .................................................................................................. 31

    3.1.3 Installation manual ....................................................................................................... 33

    3.1.4 Installation verification ................................................................................................. 35

    3.1.5 User manual .................................................................................................................. 35

    3.1.6 API documentation ....................................................................................................... 57

    3.1.7 Known issues ................................................................................................................. 58

    3.1.8 Link to the software repository..................................................................................... 58

  • D.21.1.2: Platform deployment, operations, analytics and support tools 3

    3.1.9 License ........................................................................................................................... 58

    3.2 SLA Framework ..................................................................................................................... 58

    3.2.1 Description .................................................................................................................... 58

    3.2.2 Features implemented .................................................................................................. 59

    3.2.3 Installation manual ....................................................................................................... 59

    3.2.4 Installation verification ................................................................................................. 61

    3.2.5 User manual .................................................................................................................. 61

    3.2.6 API documentation ....................................................................................................... 61

    3.2.7 Known issues ................................................................................................................. 61

    3.2.8 Link to the software repository..................................................................................... 62

    3.2.9 License ........................................................................................................................... 62

    3.3 SLA Dashboard ...................................................................................................................... 62

    3.3.1 Description .................................................................................................................... 62

    3.3.2 Features implemented .................................................................................................. 63

    3.3.3 Installation manual ....................................................................................................... 63

    3.3.4 Installation verification ................................................................................................. 67

    3.3.5 User manual .................................................................................................................. 67

    3.3.6 API documentation ....................................................................................................... 75

    3.3.7 Known Issues ................................................................................................................. 75

    3.3.8 Link to the software repository..................................................................................... 75

    3.3.9 License ........................................................................................................................... 75

    4 FI-Health ........................................................................................................................................ 76

    4.1 Sanity Checks ........................................................................................................................ 76

    4.1.1 Description .................................................................................................................... 76

    4.1.2 Features implemented .................................................................................................. 76

    4.1.3 Installation manual ....................................................................................................... 76

    4.1.4 Installation verification ................................................................................................. 80

    4.1.5 User manual .................................................................................................................. 80

    4.1.6 API documentation ....................................................................................................... 81

    4.1.7 Known issues ................................................................................................................. 81

    4.1.8 Link to the software repository..................................................................................... 81

    4.1.9 License ........................................................................................................................... 81

    4.2 Sanity Checks Dashboard ...................................................................................................... 81

    4.2.1 Description .................................................................................................................... 81

  • D.21.1.2: Platform deployment, operations, analytics and support tools 4

    4.2.2 Features implemented .................................................................................................. 81

    4.2.3 Installation manual ....................................................................................................... 81

    4.2.4 Installation verification ................................................................................................. 83

    4.2.5 User manual .................................................................................................................. 83

    4.2.6 API documentation ....................................................................................................... 88

    4.2.7 Known issues ................................................................................................................. 88

    4.2.8 Link to the software repository..................................................................................... 88

    4.2.9 License ........................................................................................................................... 88

    4.3 Federation Monitoring .......................................................................................................... 88

    4.3.1 General overview .......................................................................................................... 88

    4.4 Ceilometer customization ..................................................................................................... 89

    4.4.1 Description .................................................................................................................... 89

    4.4.2 Features implemented .................................................................................................. 89

    4.4.3 Installation manual ....................................................................................................... 90

    4.4.4 Installation verification ................................................................................................. 92

    4.4.5 User manual .................................................................................................................. 93

    4.4.6 API documentation ....................................................................................................... 93

    4.4.7 Known issues ................................................................................................................. 93

    4.4.8 Link to the software repository..................................................................................... 93

    4.4.9 License ........................................................................................................................... 93

    4.5 Monasca customization ........................................................................................................ 94

    4.5.1 Description .................................................................................................................... 94

    4.5.2 Features implemented .................................................................................................. 94

    4.5.3 Installation manual ....................................................................................................... 94

    4.5.4 Installation verification ................................................................................................. 96

    4.5.5 User manual .................................................................................................................. 96

    4.5.6 API documentation ....................................................................................................... 96

    4.5.7 Known issues ................................................................................................................. 97

    4.5.8 Link to the software repository..................................................................................... 97

    4.5.9 License ........................................................................................................................... 97

    4.6 Nagios customization ............................................................................................................ 97

    4.6.1 Description .................................................................................................................... 97

    4.6.2 Features implemented .................................................................................................. 97

    4.6.3 Installation manual ....................................................................................................... 97

  • D.21.1.2: Platform deployment, operations, analytics and support tools 5

    4.6.4 Installation verification ................................................................................................. 99

    4.6.5 User manual .................................................................................................................. 99

    4.6.6 API documentation ....................................................................................................... 99

    4.6.7 Known issues ................................................................................................................. 99

    4.6.8 Link to the software repository..................................................................................... 99

    4.6.9 License ........................................................................................................................... 99

    4.7 Federation Monitoring API ................................................................................................... 99

    4.7.1 Description .................................................................................................................. 100

    4.7.2 Features Implemented ................................................................................................ 101

    4.7.3 Installation manual ..................................................................................................... 102

    4.7.4 Installation verification ............................................................................................... 103

    4.7.5 User manual ................................................................................................................ 104

    4.7.6 API documentation ..................................................................................................... 104

    4.7.7 Known issues ............................................................................................................... 104

    4.7.8 Link to software repository ...................................................................................... 104

    4.7.9 License ......................................................................................................................... 104

    4.8 Infographics and Status Pages ............................................................................................ 105

    4.8.1 Description .................................................................................................................. 105

    4.8.2 Features implemented ................................................................................................ 105

    4.8.3 Installation manual ..................................................................................................... 105

    4.8.4 Installation verification ............................................................................................... 106

    4.8.5 User manual ................................................................................................................ 106

    4.8.6 API documentation ..................................................................................................... 110

    4.8.7 Known issues ............................................................................................................... 110

    4.8.8 Link to the software repository................................................................................... 111

    4.8.9 License ......................................................................................................................... 111

    5 FI-Toolkit ..................................................................................................................................... 112

    5.1 Flavor Sync .......................................................................................................................... 112

    5.1.1 Description .................................................................................................................. 112

    5.1.2 Features implemented ................................................................................................ 112

    5.1.3 Installation Manual ..................................................................................................... 113

    5.1.4 Installation Verification ............................................................................................... 116

    5.1.5 User Manual ................................................................................................................ 117

    5.1.6 API Documentation ..................................................................................................... 117

  • D.21.1.2: Platform deployment, operations, analytics and support tools 6

    5.1.7 Known Issues ............................................................................................................... 117

    5.1.8 Link to the software repository................................................................................... 117

    5.1.9 License ......................................................................................................................... 117

    5.2 Glance Sync ......................................................................................................................... 117

    5.2.1 Description .................................................................................................................. 117

    5.2.2 Features implemented ................................................................................................ 117

    5.2.3 Installation manual ..................................................................................................... 120

    5.2.4 Installation verification ............................................................................................... 124

    5.2.5 User manual ................................................................................................................ 125

    5.2.6 API documentation ..................................................................................................... 129

    5.2.7 Known issues ............................................................................................................... 129

    5.2.8 Link to the software repository................................................................................... 129

    5.2.9 License ......................................................................................................................... 129

    5.3 Skuld (Users Management) ................................................................................................. 129

    5.3.1 Description .................................................................................................................. 129

    5.3.2 Features implemented ................................................................................................ 130

    5.3.3 Installation manual ..................................................................................................... 130

    5.3.4 Installation verification ............................................................................................... 131

    5.3.5 User manual ................................................................................................................ 132

    5.3.6 API documentation ..................................................................................................... 133

    5.3.7 Known issues ............................................................................................................... 133

    5.3.8 Link to the software repository................................................................................... 133

    5.3.9 License ......................................................................................................................... 133

    5.4 Maintenance Calendar ........................................................................................................ 133

    5.4.1 Description .................................................................................................................. 133

    5.4.2 Features implemented ................................................................................................ 135

    5.4.3 Installation .................................................................................................................. 135

    5.4.4 Installation verification ............................................................................................... 148

    5.4.5 User Manual ................................................................................................................ 151

    5.4.6 API Documentation ..................................................................................................... 151

    5.4.7 Known Issues ............................................................................................................... 151

    5.4.8 Link to the software repository................................................................................... 151

    5.4.9 License ......................................................................................................................... 151

    5.5 FIWARE-TESTBED-DEPLOY .................................................................................................. 152

  • D.21.1.2: Platform deployment, operations, analytics and support tools 7

    5.5.1 Description .................................................................................................................. 152

    5.5.2 Features implemented ................................................................................................ 152

    5.5.3 Installation manual ..................................................................................................... 152

    5.5.4 Installation verification ............................................................................................... 154

    5.5.5 User manual ................................................................................................................ 154

    5.5.6 API documentation ..................................................................................................... 154

    5.5.7 Known issues ............................................................................................................... 154

    5.5.8 Link to the software repository................................................................................... 154

    5.5.9 License ......................................................................................................................... 154

    5.6 Aiakos .................................................................................................................................. 154

    5.6.1 Description .................................................................................................................. 154

    5.6.2 Installation manual ..................................................................................................... 158

    5.6.3 Installation verification ............................................................................................... 159

    5.6.4 User manual ................................................................................................................ 159

    5.6.5 API documentation ..................................................................................................... 160

    5.6.6 Known issues ............................................................................................................... 160

    5.6.7 Link to the software repository................................................................................... 160

    5.6.8 License ......................................................................................................................... 160

    5.7 GEDepOrch (Generic Enabler Component for Deployment and Orchestration) ................ 160

    5.7.1 Description .................................................................................................................. 160

    5.7.2 Features implemented ................................................................................................ 161

    5.7.3 Installation manual ..................................................................................................... 161

    5.7.4 Installation verification ............................................................................................... 163

    5.7.5 User manual ................................................................................................................ 164

    5.7.6 API documentation ..................................................................................................... 165

    5.7.7 Link to software repository ......................................................................................... 165

    5.7.8 Known issues ............................................................................................................... 165

    5.7.9 License ......................................................................................................................... 165

    6 References .................................................................................................................................. 166

  • D.21.1.2: Platform deployment, operations, analytics and support tools 8

    1 Introduction

    This document should be considered as a descriptive report (i.e. the documentation) accompanying

    the software components released by the OPS Chapter during the two years1 of activity of the

    FIWARE continuation project. It is divided into four main sessions each one dedicated to describe the

    components developed in the different tasks defined in the OPS Chapter: FI-Deploy, FI-Dash, FI-

    Health and FI-Toolkit.

    OPS Chapter is devoted to provide tools for easing the operations and maintenance of the FIWARE

    Lab nodes. The most relevant results obtained in the last year are:

    1. Adoption of a plugin architecture in line with the new developments of OpenStack Fuel – see

    section 2.

    2. New widget and GUI that provide a front-end for visualizing monitoring data, status of the

    nodes, maintenance calendar and synchronization of OpenStack images – see section 3.

    3. Release of a new Federation Monitoring system based on Ceilometer and Monasca – see

    section 4.

    4. Enhancement of tools for image synchronization, user management and development of

    new tools for easing the FIWARE Lab operations (Maintenance Calendar, OpenStack installed

    version etc.) – see section 5.

    5. Generic Enabler Component for Deployment and Orchestration – see section 5.

    The following figure gives an overview of the FIWARE OPS components developed so far and of their

    main relationships in a sort of software stack.

    Figure 1: Component stack

    The software delivered by the OPS Chapter is present on GitHub [10] together with the relevant

    documentation reported in this document. The links to the GitHub repositories, where the software

    is hosted, are provided in the description of each component hereunder. Bear in mind that this

    report should not be considered as the whole “deliverable” provided by OPS Chapter: it is just a

    minimal part (the documentation) and not the most relevant one that consist of the software

    1 Note that the first version of this document (D.21.1.1) describes the results of the first year of activity.

  • D.21.1.2: Platform deployment, operations, analytics and support tools 9

    implemented. Nevertheless, it is important to give an overall description of the development

    activities and this is the objective of this document.

    Note that this document merges the four documents titled “Platform deployment tool”, “Platform

    operations tools”, “Platform analytics tools” and “Support tools” foreseen by the initial FIWARE

    continuation project Document of Work.

  • D.21.1.2: Platform deployment, operations, analytics and support tools 10

    2 FI-Deploy

    2.1 OPS-Deploy

    OPS-Deploy is an open source software component, based on OpenStack Fuel [2]. It provides a web UI through that a cloud administrator can intuitively deploy and manage an OpenStack environment. OPS-Deploy is used in FIWARE platform in order to deploy a more coherent and tested installation

    within the FIWARE Lab federation guaranteeing as well as a better deployment and a more

    manageable issues resolution.

    2.1.1 Description

    OPS-Deploy is a complex software composed by a set of Puppet [3] scripts, a task orchestrator

    (Nailgun [4]), a task executor (Astute [5]) and a UI. Its goal is to provide a user friendly tool for

    deploying a new FIWARE Lab node based on OpenStack. The tool has a double advantage: support a

    cloud infrastructure owner to set up a new node more quickly than a manual installation and build a

    more coherent and tested node within the FIWARE Lab federation. As said previously, OPS-Deploy is

    based on Fuel and obviously its architecture reflects the original structure. A high level architecture

    diagram is provided below. For any detailed information, please refer to the Fuel official

    documentation [6].

    Figure 2: High level architecture

    In OPS-Deploy several third-party components like Cobbler, Puppet, Mcollective live together to Fuel specific components (e.g. Astute) and FIWARE elements (e.g. monitoring). The original project has required some customizations or enhancements as adapt the GUI to FIWARE style guide or create the UI elements for enabling the monitoring components installation as well as to develop the installation scripts for each FIWARE component integrated.

  • D.21.1.2: Platform deployment, operations, analytics and support tools 11

    Starting with release 3.0, OPS-Deploy supports a pluggable architecture, inheriting the new solution from Fuel version 6.1. It enables users to install and configure additional capabilities for their environments in a flexible, repeatable and reliable manner. According to this new architecture approach, all the components previously developed, have been re-developed as a plugin. A plugin is composed by:

    deployment_script directory: it contains a set of shell or puppet scripts; environment_config.yaml: defines additional plugin UI fields. They will be shown into OPS-

    Deploy web UI (“Settings” tab); metadata.yaml: it contains name, version and compatibility definition for the plugin; repositories directory: it contains binary packages for CentOS and Ubuntu; task.yaml: it specifies when, where and how to run the installation scripts.

    As shown in Figure 2, through the Plugin framework, the plugin is integrated into UI and it is activated by Nailgun. Furthermore, properly setting the task.yaml file, it is possible to coordinate the workflow of concurrent plugin installations. Finally, in this release each plugin can only be installed before configuring and deploying the environment. Otherwise, the users should deploy again the environment to enable the plugin. In the same manner of the Release 2.x, the users are able to interact with OPS-Deploy using both GUI and CLI. They interact with Nailgun which implements REST API as well as deployment data management. It manages disk volumes configuration data, networks configuration data and any other environment specific data which are needed for finalize a deployment. Astute can be viewed as composed by Nailgun workers. Each of them runs certain actions according to the instructions provided from Nailgun. Nailgun uses SQL database to store its data and AMQP service to interact with workers whereas Cobbler is used as operating system provisioning service and DHCP service provider. Finally, Puppet is the deployment service and through MCollective agents are performed specific tasks like hard drives clearing, network connectivity probing on the discovered nodes.

    2.1.2 Features implemented

    The version 4.0 of OPS-Deploy is based on the stable branch of Fuel version 7.0 [7]. It installs OpenStack Kilo release 2015.1.0 on Ubuntu 14.04. The previous monitoring components are now installed by plugins. Currently the following plugins are available:

    Calamari (https://github.com/SmartInfrastructures/fuel-plugin-calamari) Nova-Docker (https://github.com/SmartInfrastructures/fuel-plugin-novadocker) OpenStack Data Collector (https://github.com/SmartInfrastructures/fuel-plugin-OpenStack-

    data-collector) NGSI Adapter 1.1.1 (https://github.com/SmartInfrastructures/fuel-plugin-ngsi-adapter) Context Broker 0.13 (https://github.com/SmartInfrastructures/fuel-plugin-context-broker)

    For any further information, please refer to the Fuel release plan [8].

    2.1.3 Installation manual

    The installation process is the same of the previous releases. The OPS-Deploy installer is available at https://github.com/SmartInfrastructures/fuel-main-dev/releases. It is distributed as an ISO image, which can be installed using a virtualization software package, such as VirtualBox, or on a bare-metal server. The first option is suggested only for testing purposes, whereas the second one is suggested for production environment. When installation is completed the system will be booted. Please pay attention to remove the installation media from the master node. Finally, the user opens with a browser the page http://10.20.0.2:8443 and log in using the admin credentials (by default they are admin/admin), whereas the default admin credentials for logging in the master node, by ssh, are

    https://github.com/SmartInfrastructures/fuel-plugin-calamarihttps://github.com/SmartInfrastructures/fuel-plugin-novadockerhttps://github.com/SmartInfrastructures/fuel-plugin-OpenStack-data-collectorhttps://github.com/SmartInfrastructures/fuel-plugin-OpenStack-data-collectorhttps://github.com/SmartInfrastructures/fuel-plugin-ngsi-adapterhttps://github.com/SmartInfrastructures/fuel-plugin-context-brokerhttps://github.com/SmartInfrastructures/fuel-main-dev/releaseshttp://10.20.0.2:8443/

  • D.21.1.2: Platform deployment, operations, analytics and support tools 12

    root/r00tme. It is highly recommended to change the password after you log in (using the passwd command). For any further information about the installation procedure, please refer to the Fuel User Guide [9].

    Prerequisites

    For testing scope, the suggested minimum hardware requirements for the OPS-Deploy node are: Dual-core CPU 4+ GB RAM 1 gigabit network port HDD 80 GB with dynamic disk expansion

    For a production environment, the suggested minimum hardware requirements are: Quad-core CPU 8+ GB RAM 1 gigabit network port HDD 512+ GB

    Network setup

    On the OPS-Deploy node (also named Master Node), the eth0 network interface is configured to reply to PXE requests. The default network is 10.20.0.2/24 and the gateway 10.20.0.1. After the OPS-Deploy Master Node is installed and booted, the user can power on all slave nodes (where the user is going to install OpenStack). First of all, the user should ensure that slave nodes are physically installed in the same network as the Master. After that, the user can boot each node in PXE boot mode (the user should enable it, modifying the BIOS boot order). Each node sends out DHCP discovery requests and gets the response from the OPS-Deploy node that runs the DHCP server (provided by Cobbler). When a node receives the response from the OPS-Deploy node, it fetches the PXE Linux bootloader and then the bootstrap image (CentOS based Linux in memory) from the OPS-Deploy node TFTP server and boots into it. When this image is loaded, it reports the node readiness and configuration to the master node. This could take a few minutes.

    Plugin installation

    In order to install a plugin, it is needed to build a rpm file first. This requires preparing the building

    environment following the instructions contained in the Fuel Plugin builder guide [9]. Consequently, the installation procedure consists of the following steps:

    git clone https://github.com/SmartInfrastructures/fuel-plugin-.git cd fuel-plugin- fuel-plugin-builder --build

    After that, the plugin file must be copied to the Fuel Master node:

    scp fuel-plugin-*rpm root@:

    The user installs the plugin using the fuel command line:

    ssh root@ fuel plugins --install fuel-plugin-*.rpm

    The user can verify that the plugin is installed correctly by the following command:

    fuel plugins list

    2.1.4 Installation verification

    In order to verify the correct installation of the OPS-Deploy, the user can use the following

    command:

  • D.21.1.2: Platform deployment, operations, analytics and support tools 13

    fuel release

    The answer should be as follows:

    id Name state operating_system version

    1 Kilo on CentOS 6.5 unavailable CentOS 2015.1.0-7.0

    2 Kilo on Ubuntu 14.04 available Ubuntu 2015.1.0-7.0

    2.1.5 User manual

    Homepage

    The OPS-Deploy UI is accessible at the default url: http://10.20.0.2:8443/. Using the admin credentials (by default they are admin/admin), the user is able to access to the Dashboard (see

    Figure 3).

    Figure 3: OPS-Deploy homepage

    The user sets bare-metal servers to boot from network via PXE and power them on. They will start automatically with a bootstrap operating system, based on Centos. The OPS-Deploy will notify discovered nodes on OPS-Deploy UI (see Figure 3 in the upper right corner) and consequently the user is able to create a new environment.

    http://10.20.0.2:8443/

  • D.21.1.2: Platform deployment, operations, analytics and support tools 14

    Figure 4: Creation of a new environment

    The first step that involves the user is the “New OpenStack Environment” creation ([Figure 4]), where the user inserts basic information about the environment as name, operating system, hypervisor and network manager (Nova-Network, Neutron with GRE, Neutron with VLAN). Thus, the environment is ready for deployment (Figure 5).

    Figure 5: The page of the created environment

    In environment creation process the user should design the architecture of his cloud infrastructure. The user assigns the role to every server, configures the network, defines the space allocated to hard disks and sets other OpenStack options (Figure 6).

  • D.21.1.2: Platform deployment, operations, analytics and support tools 15

    Figure 6: Environment definition

    Giving roles to servers

    In “Nodes” tab, the user can view the state of his environment, where the nodes are ordered by Roles. Thus, the user can view the node details and configure them properly. By clicking on “Add Nodes” button, the OPS-Deploy shows the list of available roles and the list of unallocated nodes. After selecting a role, other incompatible roles are automatically disabled. For example, a controller node cannot be together with a compute node simultaneously, and so on. Finally, the user applies changes (Figure 7).

    Figure 7: The list of available servers

  • D.21.1.2: Platform deployment, operations, analytics and support tools 16

    When the changes are applied, it is possible to tune the node, by clicking on the right button indicated by the gear icon. The OPS-Deploy shows a dialog where the user can configure network interfaces, define the space allocated to hard disks and view server information (e.g. Service tag, Mac addresses, hardware specifications, etc.) (Figure 8, Figure 9, Figure 10).

    Figure 8: Network interfaces configuration

    Figure 9: Hard disks configuration

    Figure 10: Detailed information about the selected server

  • D.21.1.2: Platform deployment, operations, analytics and support tools 17

    Network settings

    In the Network section, the user can manage configuration parameters. Based on the OpenStack network architecture, OPS-Deploy considers four networks: Public, Management, Private and Storage. Management and Storage sections indicate the network subnet in CIDR notation and VLAN tags, whereas the Public section allows to set the IPs pool and its VLAN tag (Figure 11). The Private Network (also named Fixed network) identifies the network between each tenant's VMs. Private network address spaces are not a part of the enterprise network address space; fixed IPs of virtual instances cannot be accessed directly from other networks and so, clearly, the private network should be isolated from other networks for security reasons.

    Figure 11: Infrastructure network settings

    The OPS-Deploy gives user the opportunity to manage the Neutron plugin and to define the L2

    connection tunnel ID range and the L3 floating IP range. Furthermore, the user can verify the

    network configuration by clicking the “Verify Network” button, which checks for connectivity

    between nodes using the configured VLANs. It also checks if some external DHCP interferes with the

    current deployment (Figure 12).

  • D.21.1.2: Platform deployment, operations, analytics and support tools 18

    Figure 12: L2/L3 Neutron configuration

    General Settings

    The "Settings" tab contains options useful to manage the current environment. Here, the user can select one or more plugins (e.g. Calamari, ContextBroker, NGSI Adapter, OpenStackDataCollector) and configure them. Moreover, the user can change the OpenStack admin account or can change the hypervisor type or the scheduler driver. The configuration will be applied only after the deploy is started (Figure 13, Figure 14, Figure 15, Figure 16).

    Figure 13: Infrastructure settings (Access, Additional Components)

  • D.21.1.2: Platform deployment, operations, analytics and support tools 19

    Figure 14: Infrastructure settings (Common)

    Figure 15: Infrastructure settings (Kernel parameters, Syslog and Mellanox Neutron Components)

  • D.21.1.2: Platform deployment, operations, analytics and support tools 20

    Figure 16: Infrastructure settings (Public network assignment, Storage)

    Plugins section

    Calamari

    The user selects the calamari plugin check-box and fill-in the required fields. The interface looks like

    the following:

    Figure 17: Calamari installation

    The user goes to the nodes page and adds a base-os node and rename the base-os host as

    “calamari”:

    Figure 18: Calamari installation

  • D.21.1.2: Platform deployment, operations, analytics and support tools 21

    Some minutes after the deploy is ready (currently the Fuel UI display "Ready" on the nodes, even if some plugin has not yet finished), the user can browse the calamari interface on the calamari server. He knows the address of the server from the network info pop-up (Figure 19):

    Figure 19: Network info popup

    The plugin is available at: https://github.com/SmartInfrastructures/fuel-plugin-calamari

    The plugin was also accepted in the OpenStack official repositories at: http://git.openstack.org/cgit/openstack/fuel-plugin-calamari/

    Nova Docker

    This plugin enables the Docker driver (https://wiki.OpenStack.org/wiki/Docker) for Nova compute. In the “Settings” tab:

    the user selects the Nova-Docker plugin check-box and writes the name of docker containers that automatically will be downloaded and loaded to glance. The official docker containers repository is https://wiki.OpenStack.org/wiki/Docker;

    if the user selects the 'Install on all compute' option, the plugins will configure all computes node consequently;

    enable one or more specific nodes named “docker”. The plugin is available at: https://github.com/SmartInfrastructures/fuel-plugin-novadocker

    Monitoring plugins

    The release 4.0 installs three monitoring components: Context Broker 0.13, NGSI Adapter 1.2.3 and OpenStackDataCollector (Release 24/04/2015). To install those monitoring plugins the user should build the plugin as usual, and copy the rpm files to the Master Node. Once the plugins have been installed, the user can select in the “Settings” tab the related checkbox for each of them. Finally, the user can start the environment deployment. The plugins are available at:

    https://github.com/SmartInfrastructures/fuel-plugin-context-broker https://github.com/SmartInfrastructures/fuel-plugin-ngsi-adapter/ https://github.com/SmartInfrastructures/fuel-plugin-OpenStack-data-collector

    https://github.com/SmartInfrastructures/fuel-plugin-calamarihttp://git.openstack.org/cgit/openstack/fuel-plugin-calamari/https://wiki.openstack.org/wiki/Dockerhttps://wiki.openstack.org/wiki/Dockerhttps://github.com/SmartInfrastructures/fuel-plugin-novadockerhttps://github.com/SmartInfrastructures/fuel-plugin-context-brokerhttps://github.com/SmartInfrastructures/fuel-plugin-ngsi-adapter/https://github.com/SmartInfrastructures/fuel-plugin-OpenStack-data-collector

  • D.21.1.2: Platform deployment, operations, analytics and support tools 22

    Other plugins (experimental)

    Detach swift2

    Plugin to separate swift object and proxy from controller.

    Create a new environment with the Fuel UI wizard. Click on the Settings tab of the Fuel web UI.

    Figure 20: Detach swift in fuel configuration overview

    Select the “Detach swift” tab, then enable the plugin by clicking on the “Detach swift” checkbox and fill-in the required fields

    2 https://github.com/SmartInfrastructures/fuel-plugin-detach-swift

    https://github.com/SmartInfrastructures/fuel-plugin-detach-swift

  • D.21.1.2: Platform deployment, operations, analytics and support tools 23

    For installing swift proxy on separated nodes check the "Install swift proxy on separate node(s)" checkbox.

    Figure 21: Detach swift in fuel configuration detail

    Go to the nodes page. Add computes and controllers node. Add one or more 'standalone-swift-object'. On this node are installed the storage part of

    swift.

    Figure 22: Detach swift configuration for nodes

    For the swift-proxy there are two cases depending on if "Install swift proxy on separate node(s)" is enable:

    o if enable add one or more 'standalone-swift-proxy'. On this node are installed the swift control part.

    o if not enable the swift proxies are install on controllers and there is no the need and neither the possibility to install 'standalone-swift-proxy'

  • D.21.1.2: Platform deployment, operations, analytics and support tools 24

    Figure 23: Detach swift configuration for nodes: error example

    Deploy the changes

    In both case, i.e. with "Install swift proxy on separate node(s)" is checked or not, swift is configured

    correctly. In the case where it is enabled the swift endpoint is set to swift proxy virtual ip, otherwise

    it is set to controllers.

    Storlets Installer3

    The storlets installer was developed with a joint effort with Doron Chen of IBM Haifa Lab, Tel-Aviv. This is an experimental fuel plugin to install the openstack/storlets from the official git repository

    (http://git.openstack.org/cgit/openstack/storlets/). It works with fuel version 9.0. More about the storlets can be found here: http://storlets.readthedocs.io/en/latest/index.html The plugin can be built and installed following the guide: https://wiki.openstack.org/wiki/Fuel/Plugins#How_To:_Build_and_install_a_plugin_from_source In the wizard, create a new OpenStack environment, select the OpenStack Release version Mitaka on

    Ubuntu+UCA 14.04 in order to include required package repositories.

    In the Settings tab of the environment, activate the plugin by clicking on the square box before the

    title and personalize the user name and password.

    3 https://github.com/SmartInfrastructures/fuel-plugin-storlets

    http://git.openstack.org/cgit/openstack/storlets/http://storlets.readthedocs.io/en/latest/index.htmlhttps://wiki.openstack.org/wiki/Fuel/Plugins#How_To:_Build_and_install_a_plugin_from_sourcehttps://wiki.openstack.org/wiki/Fuel/Plugins#How_To:_Build_and_install_a_plugin_from_sourcehttps://github.com/SmartInfrastructures/fuel-plugin-storlets

  • D.21.1.2: Platform deployment, operations, analytics and support tools 25

    Figure 24: Storlets settings in fuel web interface

    Logs

    The log section is designed to monitor the state of installation and support the troubleshooting. The

    user can select the node to monitor, the log level and the generator source.

    Health Check

    It is very useful, running a post deployment test, to see if the installation process is correctly

    finished. The Health check process runs a set of tests, and when it is done, the user will see green

    Thumbs Up sign if it was correct and a red Thumbs Down sign if something went wrong (Figure 25).

    Figure 25: Health check result

  • D.21.1.2: Platform deployment, operations, analytics and support tools 26

    Deployment

    When the user has finished setting the environment, he can start the deployment process, clicking

    on "Deploy changes" button (Figure 26).

    Figure 26: Installation in progress

    2.1.6 API documentation

    Not applicable.

    2.1.7 Known issues

    OPS-Deploy inherits some issues from Fuel 6.1. The main of them, are summarized below.

    OPS-Deploy requires a pingable default gateway in order to deploy

    OPS-Deploy must be able to ping the default gateway in order to deploy the environment. If your

    configuration does not include a pingable default gateway, you can work around it by specifying the

    Fuel Master node (or any other pingable host) as the default gateway. Alternatively, you can apply

    Patch 138448 [40] to disable the requirement to ping the default gateway. After applying this patch,

    you need to enable it following sequence of steps [8].

    Live migration

    Live migration with the specified destination host does not currently use the same memory

    oversubscription logic as during an instance boot. This may cause migrations to fail due to not

    enough amount of memory. See [41].

    Ceph OSD on controller nodes

    Placing Ceph OSD on Controllers is not recommended because it can severely degrade controller

    performance. It is better to use separate storage nodes if you have enough hardware.

  • D.21.1.2: Platform deployment, operations, analytics and support tools 27

    RPC clients are not able to find a RabbitMQ queue

    RPC clients cannot find a reply queue after the last RabbitMQ server restarts in the cluster. See [42].

    2.1.8 Link to the software repository

    https://github.com/SmartInfrastructures/fuel-main-dev/

    2.1.9 License

    Apache License, Version 2.0, January 2004

    2.2 Keyrock to Keystone Federation

    We developed a patch for the keystone for Kilo in order to support a simple federation.

    2.2.1 Features Implemented

    The main purpose is to:

    introduce a Keystone token driver that, installed in the remote cloud Keystone, to allow the token validation from the OpenStack services remotely with the Keyrock.

    thus, when a request from the Cloud Portal (authenticated with a token from the Keyrock) is sent to a remote cloud OpenStack service, the service tries to validate the token in the local Keystone and, if it fails, the validation of the FIWARE LAB user token is delegated transparently to the Keyrock.

    Figure 27: Use case

    When a FIWARE LAB user uses the remote Cloud resources, the new driver comes into play:

    Cloud Portal authenticates the user in Keyrock Cloud Portal sends a request (via APIs) to a remote Cloud OpenStack service Remote cloud OpenStack service tries to validate the token in Keystone. As the token

    validation does not succeed (the token is not stored in remote Keystone), the remote Keystone validates it with Keyrock acting as a gateway, and sends the response to the remote Cloud OS Service.

    See Figure 27, Figure 28 and Figure 29 for a graphical representation.

    https://github.com/SmartInfrastructures/fuel-main-dev/

  • D.21.1.2: Platform deployment, operations, analytics and support tools 28

    Figure 28: Sequence Diagram

    Figure 29: Current Implementation (UML)

    2.2.2 Installation Manual

    The installation of this component follows the procedure defined for the upstream Keystone version.

    For a detailed guide see: http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-

    install.html

    http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.htmlhttp://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html

  • D.21.1.2: Platform deployment, operations, analytics and support tools 29

    Configuration

    Add a section to keystone.conf as in the following example with remote idp and the remote idp service user and password:

    [simplefederation]

    idp=http://user1:[email protected]:35357

    idp=http://user2:[email protected]:35357

    2.2.3 Installation Verification

    In the keystone log search for the string "simplefederation". On a successful installation the

    validation of a remote token produces a log similar to the following:

    raise exception.TokenNotFound(token_id=token_id)

    TokenNotFound: Could not find token: c9a49ebe71754213b60011e7cede6045

    [-] simplefederation: trying to validate the token on remote idp:

    http://idp1:[email protected]:35357

    [-] simplefederation: auth url:

    http://192.168.50.102:35357/v3 get_simplefederation_token_data

    /opt/stack/keystone/keystone/auth/controllers.py:702

    [-] REQ: curl -g -i -X GET http://192.168.50.102:35357/v3 -H "Accept:

    application/json" -H "User-Agent: python-keystoneclient" _http_log_request

    /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:196

    [-] RESP: [200] content-length: 254 vary: X-Auth-Token server:

    Apache/2.4.7 (Ubuntu) date: Thu, 14 Apr 2016 09:40:06 GMT

    content-type: application/json x-openstack-request-id:

    req-e76a82f2-fa8e-42c5-95a9-4b20913cadcc

    {"status": "stable", "updated": "2015-03-30T00:00:00Z",

    "media-types": [{"base": "application/json", "type":

    "application/vnd.openstack.identity-v3+json"}], "id": "v3.4", "links":

    [{"href": "http://192.168.50.102:35357/v3/", "rel": "self"}]}}

    [-] Making authentication request to

    http://192.168.50.102:35357/v3/auth/tokens get_auth_ref

    /usr/local/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py:171

    [-] simplefederation: auth token:

    7c24aebdccdb458386cf3f635eaadf3c get_simplefederation_token_data

    /opt/stack/keystone/keystone/auth/controllers.py:716

    [-] simplefederation: Remote response:

    CaseInsensitiveDict({'content-length': '296', 'x-subject-token':

    'c9a49ebe71754213b60011e7cede6045', 'vary': 'X-Auth-Token', 'server':

    'Apache/2.4.7 (Ubuntu)', 'date': 'Thu, 14 Apr 2016 09:40:07 GMT',

    'content-type': 'application/json', 'x-openstack-request-id':

    'req-22c67fa4-2924-4d28-b750-4b4369f15dc5'})

    get_simplefederation_token_data

    /opt/stack/keystone/keystone/auth/controllers.py:731

    [-] simplefederation: Remote response: {"token": {"methods":

    ["password"], "expires_at": "2016-04-14T10:36:39.503300Z",

    "extras": {}, "user": {"domain": {"id": "default", "name":

    "Default"}, "id": "a704e3da1c364d78bdecf8d0d5356ef2", "name":

    "demo"}, "audit_ids": ["pAR7UvCPSkKrZMRqs3tRrw"], "issued_at":

    "2016-04-14T09:36:39.503323Z"}} get_simplefederation_token_data

    /opt/stack/keystone/keystone/auth/controllers.py:735

    [-] simplefederation: token validated on remote idp:

    http://idp1:[email protected]:35357

  • D.21.1.2: Platform deployment, operations, analytics and support tools 30

    2.2.4 User Manual

    Not applicable

    2.2.5 API Description

    Not applicable

    2.2.6 Known Issues

    Current Implementation is Limited to:

    UUID v3 tokens only Kilo only (Kilo end of life: 2016-05-02) not a plugin, it changes keystone a bit (all test pass or are skipped on a suggested keystone

    developer environment for python 2.7) need a service with admin role on the remote keystone (KeyRock)

    2.2.7 Link to the Software Repository

    https://github.com/SmartInfrastructures/keystone/

    2.2.8 License

    Apache License, Version 2.0, January 2004

    https://github.com/SmartInfrastructures/keystone/

  • D.21.1.2: Platform deployment, operations, analytics and support tools 31

    3 FI-Dash

    3.1 FIDASH - FIWARE Lab Dashboard

    FIDASH is an administration and management dashboard for FIWARE Lab. FIDASH is the dashboard component of the FIWARE FI-OPS tools, a tool suite created to monitor and administer FIWARE Lab nodes based on FIWARE Lab features such as: the underlying federated OpenStack-based nodes; the monitoring infrastructure; other administrative and operational tools created by FI-TOOLKIT and FI-HEALTH enablers.

    3.1.1 Description

    FIDASH is a tailored version of WireCloud [11] and as such is a highly-customizable mashup [12] environment that allows the user to easily define functionality and behavior of a dashboard. The dashboard is made up of multiple widgets that the user can choose, discard, set layout and modify their behavior. These widgets offer specific functionality such as display a list of VM (virtual machine) instances or synchronize the flavors of some FIWARE Lab nodes. Widgets rely on REST-based APIs offered by the different OpenStack services, SLA Manager tool (3.2), Federation Monitoring API (4.3) or other services created inside FI-Toolkit ([5]) to give support for certain actions. Some services are directly accessed, though others use some libraries to hide the complexity of the APIs, such as the jstack [13] library for accessing OpenStack services in FIWARE Lab deployments. Authentication is managed by the underlying WireCloud platform against the Identity Management (IdM) and the keystone Proxy present on all FIWARE Lab nodes, and authorization relies on the user rights present at the IdM. These widgets are connected together to perform higher level functions and to provide correct feedback based on actions done by other components; such integration is done through a mechanism, called wiring, that send asynchronous messages with data among themselves (events). User of FIDASH is in control of that wiring, and can connect/disconnect widgets at will, modifying the behavior of the dashboard. In addition, FIDASH comes with predefined dashboard set-ups (mashups with instantiated widgets and set up wiring) for an out-of-the-box usage; nonetheless the user can modify existing dashboard set-ups or create new ones from scratch, in a guided and intuitive manner. Both functionality and appearance can be customized by the user.

    3.1.2 Features implemented

    FIDASH, as FI-OPS dashboard, is the graphical front-end that offers FIWARE Lab administrators and users with a comprehensive access to different back-end services, namely:

    Monitor the usage of resources in live usage of resources at different levels: o Regions: display the free and used vCPUs, RAM, storage space, and IP addresses

    assigned and reserved o Hosts per region: display the percentage of CPU, RAM and storage used o VMs per region: display the percentage of CPU, RAM and storage used

    Maintenance Calendar, where maintenance periods can be defined per region so as interested people can be informed, as well as a mechanism to request for _no maintenance periods_ during certain periods where events are to be held

    Display the top tenants in terms of resource usage, showing and sorting by number of VMs, amount of RAM, number of virtual CPUs, percentage of RAM or CPU used

  • D.21.1.2: Platform deployment, operations, analytics and support tools 32

    Display the status of the generic enablers Global Instances Display the OpenStack component versions installed on the different nodes Synchronization of flavors across different regions Synchronization of images across different regions

    OpenStack services for managing offered resources in FIWARE Cloud. The elements managed at FIDASH are: running instances or VMs, that can be listed, detailed, searched, filtered, deleted and rebooted. Simple creation functionality is also provided

    images, that can be listed, detailed, searched, filtered, made public or protected, and created from file or remote URL. Simple launch functionality is also provided

    volumes, that can be listed, detailed, searched, filtered, and attached flavors, that can be listed, detailed, searched and filtered. Synchronization is also

    implemented but requires administrative privileges. Verification of the compliance with established Service Level Agreements.

    The widgets in FIDASH are: Detail Image: this widget displays all the attributes of an OpenStack Image available to the

    user in FIWARE Lab. The widget also allows the user to edit and delete the displayed image. Detail Instance: this widget displays all the attributes of an OpenStack Instance (VM)

    available to the user in FIWARE Lab. The widget also allows the user to reboot and delete the displayed instance.

    Detail Volume: this widget displays all the attributes of an OpenStack Volume available to the user in FIWARE lab. The widget also allows the user to attach, detach, edit and delete the displayed volume.

    List Flavors: this widget displays a list of flavors available to the user in FIWARE Lab. The widget also has multi-region support and allows the creation, edition and removal of flavors.

    List Images: this widget displays a list of images available to the user in FIWARE Lab in a table interface with searching and pagination capabilities. It also offers support to multi-region and allows the creation of new images in any region in which the user is allowed to do so.

    List Instances: this widget displays a list of instances available to the user in FIWARE Lab. The widget has also multi-region support and allows the creation of new instances.

    List Volumes: this widget displays a list of volumes available to the user in FIWARE Lab. The widget also has multi-region support and allows the creation of new volumes.

    Embedded SLA Manager: this widget displays the web interface for the SLA Manager (3.2) which allows users to handle agreements.

    Select Regions: this widget displays the list of regions and allows selecting them. It does send such list through wiring, and other widgets with region-based filters can be connected to it and therefore get the regions synchronized

    Monitor Regions: this widget displays monitoring data for RAM, disk, CPU and IP usage in any region in FIWARE's Cloud using the Federation Monitoring API.

    Monitor VMs: this widget displays monitoring data for RAM, disk, and CPU of every virtual machine in any region in FIWARE's Cloud using the Federation Monitoring API.

    Monitor Hosts: this widget displays monitoring data for RAM, disk, and CPU of every server in any region in FIWARE's Cloud using the Federation Monitoring API.

    Historical Monitor Regions: this widget displays historical monitoring data about RAM, disk, CPU and IP usage in any region in FIWARE's Cloud. The Federation Monitoring API did not finally offer such data, so no real usage yet.

    Historical Monitor VMs: this widget displays historical monitoring data about RAM, disk and CPU usage in any virtual machine in any region in FIWARE's Cloud. The Federation Monitoring API did not finally offer such data, so no real usage yet.

  • D.21.1.2: Platform deployment, operations, analytics and support tools 33

    Historical Monitor Hosts: this widget displays historical monitoring data about RAM, disk and CPU usage in any server in any region in FIWARE's Cloud. The Federation Monitoring API did not finally offer such data, so no real usage yet.

    Maintenance Calendar: this widget does show maintenance periods on any region, as well as allows adding or modifying them, and define requests of no-maintenance periods

    Show Difference Flavors: this widget emphasizes flavor differences in a comparison with “compare flavors” widget

    Compare Flavors: this widget does compare the widgets of a specific region and the reference flavors

    Manage Promoted Flavors: this widget does allow managing the list of reference flavors Synchronize Images: this widget does allow reviewing images in reference region (Spain2)

    and a specific region, and allows invocation of GlanceSync tool Top tenants (data-usage): this widget does show top tenants in terms of resource usage

    displaying number of VMs, number of vCPUs, RAM used, as well as percentages of CPU or RAM used in latest 24 hours

    Generic Enablers global instances: this widget displays the status of the services that make up the global instances of the different GEs

    OpenStack installed versions: this widget displays the version of the OpenStack components in any region

    3.1.3 Installation manual

    Introduction

    The document describes the installation of the FIDASH component, the modification and creation of dashboards, the permissions needed to operate with the different services, and other technical details relevant of the usage of the component.

    Installation of the environment

    FIDASH is based on WireCloud. Version 0.9.2 is recommended in case provided style wants to be used as is. Public instance of WireCloud4 might be used for testing some widgets, but roles are not defined, so not all functionality can be used.

    WireCloud installation

    A regular WireCloud instance is required. Please follow the instructions on the WireCloud installation

    guide or use a pre-created image.

    After the basic installation, integration with the IdM GE must be done, as described in such guide.

    Besides, Apache2-based running is also recommended.

    Theme

    It is not mandatory to modify the theme. A customized version of the WireCloud FIWARE theme has

    been created with specific logos and branding, and can be used by copying

    the https://github.com/fidash/fiware-fidash/tree/master/theme/fidashtheme folder on the

    WireCloud installation directory and modifying the THEME_ACTIVE directive

    on settings.py to fidashtheme.

    4 https://mashup.lab.fiware.org

    https://github.com/Wirecloud/wirecloud/blob/develop/docs/installation_guide.mdhttps://github.com/Wirecloud/wirecloud/blob/develop/docs/installation_guide.mdhttps://github.com/fidash/fiware-fidash/tree/master/theme/fidashthemehttps://mashup.lab.fiware.org/

  • D.21.1.2: Platform deployment, operations, analytics and support tools 34

    After installing and setting the theme, static content should be recreated. This is done using the following command inside the installation directory:

    python manage.py collectstatic

    Roles

    Different roles are used in Widgets and in backend services. These roles must be defined for certain

    users in the IdM application created for integrating the WireCloud instance with IdM.

    Some roles are general, such as UptimeRequester for calendar or InfrastructureManager for image

    synchronization. But the InfrastructureOwner is not to be granted to the user in the app, but to be

    granted to the user in an organization that matches the region name (adding the string " FIDASH").

    This role is for accounts allowed to manage elements inside a region, though that users should also

    have OpenStack permissions, which is out of the scope of this guide.

    Regions have been mapped as organizations in the IdM. For avoiding collisions, an organization with the name of every region ending in " FIDASH" has been created. Therefore, organizations such as Spain2 FIDASH or Crete FIDASH do represent regions Spain or Crete. For setting roles inside organizations, several steps must be carried out:

    1. create the region if not yet done 2. authorize the region to assign roles on the application. This is done by assigning the

    role purchaser to the region 3. switching to the region profile. Next steps are done on the region profile 4. add the user as member in the region 5. enter to the members configuration 6. authorize que user by granting him the desired role in the organization for the application.

    This is done only with the InfrastructureOwner role.

    Make a superuser

    Superusers can be created easily with the command python manage.py createsuperuser. However, when having activated the widget python-social-auth, local user database is no longer used. Therefore, the required steps are:

    1. login the platform with the IdM user to be made SuperUser. Please take note of the username assigned to that user, which is not his email. Username can be discovered on Wirecloud navigation bar (whose text is "username / workspace name"), or in the URL path, just after the IP address or domain-name, and before the dashboard name.

    2. Open a python shell with the command python manage.py shell on the WireCloud

    installation directory and write the following lines replacing USERNAME with the real username:

    from django.contrib.auth.models import User user = User.objects.get(username='USERNAME') user.is_superuser = True

    user.is_staff = True user.save()

    Uploading widgets to the new instance

  • D.21.1.2: Platform deployment, operations, analytics and support tools 35

    Each of the developed widgets, and the only one operator, is developed on a specific project inside

    fidash organization in GitHub.

    Widgets are configured using grunt, which is used for setting up the environment, testing, packaging

    and, optionally, uploading them to the instance. The default grunt action will package them, so

    leaving the .wgt file on the dist folder. More detailed instructions can be found on the developer

    guide, and indivitual compilation instructions are described in the README.md file of each widget.

    Uploading widgets automatically

    The wirecloud action will upload the widget and will set it as public to every user in the WireCloud

    instance. This action will require the URL of the instance and user credentials for carrying out such

    actions.

    Uploading widgets manually

    To upload widgets, go to the My Resources section, click on Upload button and then drag all the

    desired widgets.

    To make a widget public, open the Django Admin Console (it must be done through a superadmin user), click on Catalogue Resources, enter the desired resource and set on the tick on "Available to all users".

    3.1.4 Installation verification

    A correct installation of FIDASH must have all the FIDASH widgets available in the section My Resources. The permissions have to be tested by service:

    To check OpenStack-based services, deploy ListImages widget on any dashboard. It should list the public images.

    To check Monitoring services, deploy Monitor Regions widget and display information about different regions. Graphs showing the RAM, CPU, IP addresses and storage used should appear.

    To check SLA Management, deploy Embedded SLA Manager widget. The web interface of the manager should appear inside the widget

    3.1.5 User manual

    Users should be able to log in FIDASH platform at https://dash.lab.fiware.org using their own FIWARE Lab user. In case a user does not have a FIWARE Lab user, it should be requested at the IdM GE5. User roles and permissions are inherited from the ones they do have on the cloud portal or assigned on FIDASH application for administration purposes. Issues with user rights should be addressed to [email protected] clearly indicating FIDASH as the destination of the request.

    Using a pre-created dashboard

    5 https://account.lab.fiware.org/

    https://dash.lab.fiware.org/mailto:[email protected]

  • D.21.1.2: Platform deployment, operations, analytics and support tools 36

    FIDASH comes with pre-created dashboards that can be instantiated and used in seconds. To do this, the button

    shall be used to create a new dashboard. In the template-selection screen, click the search button,

    and all the available dashboards will appear.

    https://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/menu-small.png

    Figure 30: Create new workspace

    Select button of the desired dashboard is to be clicked. If no name is specified, new workspace will

    be created with the name of the dashboard.

    Dashboard customization

    An existing mashup or a new one (accessible from the mashup menu button menu button) shall be

    used.

    The first time the user tries to log into FIDASH she will be requested to allow access to the

    application FIDASH. Sometimes, this authorization request does not appear, and the access to

    FIDASH platform must be retried. And sometimes IdM does not return to FIDASH after authorization,

    so it has to be done by user browsing again to https://dash.lab.fiware.org.

    The main view of FIDASH is an empty dashboard where the user can create her own dashboard from

    scratch. However, the fastest way to start is instantiating a previously created dashboard, as

    explained below. The user is able to create multiple dashboards, to modify them (even the ones

    created instantiating a existing one), or to create them from scratch.

    In any case, the button for all of the dashboard-related tasks, including accessing existing ones,

    deleting, creating new ones or accessing their settings is this one:

    Instantiate dashboards

    In case a previously created dashboard is to be used, either by having been created by the user, uploaded by her or it is a publicly available dashboard, as they are on the platform.

    In FIDASH, dashboards appear as mashups in the My Resources section.

    https://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/menu-small.pnghttps://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/menu-small.pnghttps://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/menu-small.pnghttps://dash.lab.fiware.org/https://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/template-selection.pnghttps://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/menu-small.pnghttps://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/my-resources.png

  • D.21.1.2: Platform deployment, operations, analytics and support tools 37

    Clicking on My Resources button takes the user to the full list of resources, where some of them are

    labeled with the tag mashup. Those are the dashboards that can be instantiated.

    Figure 31: List of resources

    Documentation of widgets, operators and mashups is accessible on that screen by clicking on any of

    them.

    In case the user wants to deploy one of the dashboards (the mashups), she must back to the

    dashboard view, use the menu button ( ) and choose New workspace option. In next window, a

    look-up icon displays the list of mashups (the dashboards), and the user does only have to choose

    one of them by clicking on the select button.

    Figure 32: Select a mashup template

    https://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/my-resources-screen.pnghttps://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/menu-small.pnghttps://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/choose-mashup.png

  • D.21.1.2: Platform deployment, operations, analytics and support tools 38

    If no name is specified, the new dashboard will inherit the name of the mashup. In any case, it will

    appear on the list of workspaces accessible through the menu button.

    This dashboard can be customized as the user wishes:

    Size of the widgets can be changed dragging the lower-right corner of them

    Position of the widgets can be changed by clicking on the top bar of them

    Widgets may have some options accessible through their settings menu that appears when

    the mouse stops over the top-right corner of the widget User can also remove widgets (cross next to the options menu of the widget), add new widgets or

    modify the behavior through wiring. Next sections describe the latter options in detail.

    Instantiate widgets

    In the desired work-space, click the plus button for adding widgets.

    Figure 33: Add widget

    The available widgets will appear in a left-sliding window. Another plus button is available at each widget header so as to add it to the current workspace.

    Figure 34: List of widgets

    Widgets appear with a default size and in the best available empty space inside the workspace. Please be aware of the tab functionality at the bottom of the page, in case big dashboards are desired. Widget size and position can be changed and adjusted using its window controls (bottom-left and right corner to resize and title bar to drag).

    Defining behavior through wiring

    FIDASH widgets communicate among themselves so as to create a composite application made up from the collaboration of the different widgets instantiated in a dashboard. Widgets generate events containing data of items of interaction (mainly identifiers of VMs, volumes, etc.), and if other widgets receive such events and act accordingly (e.g. showing the details of the received VM id) is

    https://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/widget-menu.png

  • D.21.1.2: Platform deployment, operations, analytics and support tools 39

    decided by the user dashboard defining the wiring among the widgets. This is done on the wiring tool that is accessible through its icon:

    On the wiring tool, all the widgets deployed on the dashboard appear at the leftmost side of the screen. These elements are to be dropped on the middle panel so as to define the wiring (namely connect output endpoints with input endpoints). As they are at the center panel, input and output endpoints of the widgets are shown. Input endpoints are shown as orange circles at the left of the widgets, whereas output endpoints are green circles at the right side of them.

    Figure 35: Wiring endpoints

    If the user wants that an event generated at a given widget (for example, after she clicked on a VM image ID in a table) is reproduced on some other widget (e.g. showing the details of that image) she has to drag the corresponding output endpoint to the corresponding input endpoint. In case multiple input and output endpoints, or when their related actions are not easily understood by the simple description, the documentation of the widget describes precisely those actions and effects. It is noteworthy indicating that the selection of a certain endpoint highlights the compatible endpoints that can be connected with the one selected. In FIDASH this implies that choosing an endpoint outputting for example an image ID highlights the input endpoints where an image ID is expected. Due to the open nature of the WireCloud wiring technology, it is not a constraint, and a user can decide to connect different types of events at his own risk (since they do only send text data), but the result might not be satisfactory.

  • D.21.1.2: Platform deployment, operations, analytics and support tools 40

    Figure 36: Compatible endpoints

    Finally, the created wire appears as a pipe linking both endpoints that implies that whenever the widget source of the event produces it, the event with the associated data is immediately taken by the platform to the consumer of that event. The input and output endpoints can have as many wires connected as desired, allowing the user to completely define the behavior of her dashboard.

    Basic widget connections

    Created widgets for FIDASH are designed maximizing the input and output endpoints that those widgets can handle. This is done so as to provide the maximum flexibility to FIDASH users. However, there are typical connections that are the basis for most dashboards. All the widgets related to OpenStack services are divided into two categories, listing widgets and detail widgets. Listing widgets show list of images, volumes, etc, plus some basic information, and rely on detail widgets for showing further information. These pairs of widgets are to be connected together (in case a user wants to deploy both on her dashboard). Nonetheless some of them offer some further functionality (e.g. ListInstance or DetailInstance widgets can generate image ID events that the user can wire to the same or other instance of DetailImage in case she wants to make use of that option). An example of connections of a simple dashboard containing one instance of every widget would be the one shown in Figure 37. In this case, Detail Image would display the details of an image clicked on the details of an instance, on the list of instances (has an image column) or on the list of images.

  • D.21.1.2: Platform deployment, operations, analytics and support tools 41

    Figure 37: Example of connections

    Publishing your own dashboard

    Once a dashboard is created, having settled the desired layout, the wiring, the properties, etc., it can be saved for the future, of for sharing with other users. This can be done by clicking on the menu button (menu button) and choosing Upload to my resources. User must indicate vendor, version number, email, and some brief description. On other tabs an icon can be added (170x80 px), and some other options such as block the widgets in the mashup, block the connections, or embed the widgets or operators inside the dashboard (not only linking them by reference).

    Figure 38: Publishing your own dashboard

    https://github.com/fidash/fiware-fidash/blob/master/docs/user_guide/images/upload-dashboard.png

  • D.21.1.2: Platform deployment, operations, analytics and support tools 42

    After uploading the dashboard, My Resources section will display a new resource with the chosen name. This dashboard can be instantiated or downloaded (to be shared