whale Documentation

whale DocumentationRelease

Mirantis

Jun 30, 2017

Contents

1 Annotation 1

2 Architecture 3

3 How to install 5

4 How to run tests 7

5 How to debug tests 9

6 Deep to structure 11

Python Module Index 25

i

ii

CHAPTER 1

Annotation

Whale is intended to provide the community with a testing toolkit based on Stepler framework that is capable toperform testing of Decapod (a tool to manage a life cycle of Ceph clusters).

1

https://github.com/Mirantis/stepler

whale Documentation, Release

2 Chapter 1. Annotation

CHAPTER 2

Architecture

Architecture has following abstraction levels, where code lives (from higher to less):

• clients are able to manipulate resources: users, roles, servers, etc. For example: keystone client, nova client,node client, etc.

• steps are actions, that we want to make over resources via clients: create, delete, update, migrate, etc. Theyshould end with check, that step was finished correct.

• fixtures manage resources construction, destruction, etc via steps.

• tests combine steps and fixtures according to scenario.

Detailed information about auto-tests construction is available in our guideline.

Sometimes it needs to have code for ssh connection, proxy server, etc. They are not related with clients, steps, fixturesand tests and are considered as third party helpers and must be implemented based on its purpose with OOP and designprinciples.

Whale uses py.test as test runner and tox for routine operations. Be sure you know them.

3

http://autotests-guideline.readthedocs.io/

http://doc.pytest.org/

https://tox.readthedocs.io/


4 Chapter 2. Architecture

CHAPTER 3

How to install

Make following commands in terminal:

$ git clone https://github.com/Mirantis/whale.git$ cd whale$ virtualenv .venv$ . .venv/bin/activate$ pip install -U pip$ git submodule update --init --recursive$ pip install -r requirements.txt

5


6 Chapter 3. How to install

CHAPTER 4

How to run tests

If you know how to launch tests with py.test, you may skip this section.

Before launching you should export some openstack environment variables:

• DECAPOD_URL (default value 'default')

• DECAPOD_LOGIN (default value 'user')

• DECAPOD_PASSWORD (default value 'password')

To get details look into whale/config.py.

Let’s view typical commands to launch test in different ways:

• If you want to launch all tests (-v is used to show full name and status of each test):

$ py.test whale -v

• For example, you write the test test_create_user and want to launch it only:

$ py.test whale -k test_create_user

• If your test was failed and you want to debug it, you should disable stdout capture:

$ py.test whale -k test_create_user -s

• Full information about py.test is obtainable with:

$ py.test -h

7


8 Chapter 4. How to run tests

CHAPTER 5

How to debug tests

We recommend to use ipdb to set up break points inside code. Just put following chunk before code line where youwant to debug (don’t forget about -s to disable py.test stdout capture):

import ipdb; ipdb.set_trace()

9


10 Chapter 5. How to debug tests

CHAPTER 6

Deep to structure

Decapod

Decapod tests

Cluster tests

whale.decapod.tests.test_clusters.test_create_delete_cluster(cluster_steps)Scenario: Check cluster deletion.

Steps:

1.Create cluster

2.Delete cluster

whale.decapod.tests.test_clusters.test_get_cluster(cluster_steps, cluster)Scenario: Check that cluster was got.

Setup:

1.Create cluster

Steps:

1.Get cluster

Teardown:

1.Delete cluster

whale.decapod.tests.test_clusters.test_list_clusters(cluster_steps, cluster)Scenario: Check that we got list of clusters.

**Setup:

1.Create cluster

Steps:

1.Get list of clusters

Teardown:

1.Delete cluster

11


whale.decapod.tests.test_clusters.test_update_cluster(cluster_steps, cluster)Scenario: Check cluster update.

Setup:

1.Create cluster

Steps:

1.Update cluster

Teardown:

1.Delete cluster

Execution tests

whale.decapod.tests.test_executions.test_deploy_cluster_add_osd_monitor_telegraf(deploy_cluster,server_steps,play-book_config_steps,ex-e-cu-tion_steps)

Scenario: Deploy Ceph cluster, add OSD and monitor hosts, Telegraf.

Setup:

1.Deploy cluster

Steps:

1.Get all servers

2.Create new configuration, using the cluster and “Add OSD to Ceph cluster” playbook and 1 vacant server

3.Execute created configuration

4.Create new configuration, using the cluster and “Add monitor to the cluster” playbook and 1 vacant server


6.Create new configuration, using the cluster and “Telegraf Integration Plugin for Decapod” playbook andall used servers


8.Create new configuration, using the cluster and “Telegraf removal plugin for Decapod” playbook and allused servers


10.Create new configuration, using the cluster and “Remove monitor host from Ceph cluster” playbook andmonitor server


12.Create new configuration, using the cluster and “Remove OSD host from Ceph cluster” playbook and OSDserver


Teardown:

12 Chapter 6. Deep to structure


1.Delete cluster via “purge_cluster” playbook execution.

whale.decapod.tests.test_executions.test_deploy_cluster_add_rest_api_rados_gateway(deploy_cluster,server_steps,play-book_config_steps,ex-e-cu-tion_steps)

Scenario: Deploy Ceph cluster, add REST API and Rados Gateway hosts.

Setup:

1.Deploy cluster

Steps:

1.Get all cluster servers

2.Create new configuration, using the cluster and “Add Ceph REST API host” playbook, and 2 servers (onecluster server and one vacant server)


4.Create new configuration, using the cluster and “Add Rados Gateway to the cluster” playbook, and 2servers (one cluster server and one vacant server)


6.Create new configuration, using the cluster and “Remove Rados Gateway host from cluster” playbook, andRados Gateway servers


8.Create new configuration, using the cluster and “Remove Ceph REST API host” playbook, and REST APIservers


Teardown:


whale.decapod.tests.test_executions.test_deploy_cluster_integrate_cinder_upgrade_ceph(deploy_cluster,play-book_config_steps,ex-e-cu-tion_steps)

Scenario: Deploy Ceph cluster with Cinder integration, upgrade Ceph.

Setup:

1.Deploy cluster with OSD collocated journals and Ceph RestAPI

Steps:

1.Create new configuration, using the cluster and “Cinder Integration” playbook with “Cinder with Cephbackend”, “Glance with Ceph backend” and “Nova with Ceph backend”


6.1. Decapod 13


3.Create new configuration, using the cluster and “Upgrade Ceph” playbook with force time sync on Cephnodes


Teardown:


Playbook configuration tests

whale.decapod.tests.test_playbook_configs.test_create_delete_playbook_config(playbook_config_steps,clus-ter,server_steps)

Scenario: Check playbook configuration creation and deletion.

Setup:

1.Create cluster

Steps:

1.Get list of all servers

2.Create playbook configuration

3.Delete playbook configuration

Teardown:

1.Delete cluster

whale.decapod.tests.test_playbook_configs.test_get_playbook_config(playbook_config_steps,play-book_config_deploy)

Scenario: Check playbook configuration getting.

Setup:


Steps:

1.Get playbook configuration by its id

Teardown:


whale.decapod.tests.test_playbook_configs.test_list_playbook_configs(playbook_config_deploy,play-book_config_steps)

Scenario: Check getting of all playbook configurations.

Setup:


Steps:

1.Check playbook configs is not empty

Teardown:




whale.decapod.tests.test_playbook_configs.test_update_playbook_config(playbook_config_steps,play-book_config_deploy)

Scenario: Check playbook configuration updating.

Setup:


Steps:

1.Update playbook configuration with new name

Teardown:


Playbook tests

whale.decapod.tests.test_playbooks.test_list_playbooks(playbook_steps)Scenario: Get list of all playbooks.

Steps:

1.Get all playbooks

Role tests

whale.decapod.tests.test_roles.test_create_and_delete_role(role_steps)Scenario: Check creating and deleting of role.

Steps:

1.Create role

2.Delete role

whale.decapod.tests.test_roles.test_get_role(role, role_steps)Scenario: Check role getting.

Setup:

1.Create role

Steps:

1.Get role by id

Teardown:

1.Delete role

whale.decapod.tests.test_roles.test_list_permissions(role_steps)Scenario: Check getting all role permissions.

Steps:

1.List all role permissions

whale.decapod.tests.test_roles.test_list_roles(role_steps)Scenario: Check getting of all roles.

Steps:

1.List all roles

6.1. Decapod 15


whale.decapod.tests.test_roles.test_update_role(role, role_steps)Scenario: Check role updating.

Setup:

1.Create role

Steps:

1.Update role with the new name

Teardown:

1.Delete role

Server tests

whale.decapod.tests.test_servers.test_delete_create_server(server_steps)Scenario: Check that server was deleted and created.

Steps:

1.Delete server

2.Create server

whale.decapod.tests.test_servers.test_get_server(server_steps)Scenario: Check that we got server.

Steps:

1.Get server

whale.decapod.tests.test_servers.test_list_servers(server_steps)Scenario: Check that we got list of servers.

Steps:

1.Get list of servers

whale.decapod.tests.test_servers.test_update_server(server_steps)Scenario: Check that server was updated.

Steps:

1.Update server

User tests

whale.decapod.tests.test_users.test_create_delete_user(user_steps)Scenario: Check user creation and deletion.

Steps:

1.Create user

2.Delete user

whale.decapod.tests.test_users.test_get_user(user, user_steps)Scenario: Check getting of user.

Setup:

1.Create user



Steps:

1.Get user by id

Teardown:

1.Delete user

whale.decapod.tests.test_users.test_list_users(user_steps)Scenario:** Check getting of all users.

Steps:

1.List all users

whale.decapod.tests.test_users.test_update_user(user, user_steps)Scenario: Check updating of user.

Setup:

1.Create user

Steps:

1.Update user with new login, email and full name

Teardown:

1.Delete user

Decapod UI

Decapod conftest

Contains fixtures specific for Decapod (UI testing).

whale.decapod_ui.conftest.auth_steps(decapod)Get auth steps to login or logout in decapod.

whale.decapod_ui.conftest.decapod(video_capture)Initial fixture to start.

whale.decapod_ui.conftest.login(auth_steps)Login to decapod.

Majority of tests requires user login. Logs out after test.

whale.decapod_ui.conftest.report_dir(request)Create report directory to put test logs.

whale.decapod_ui.conftest.ui_cluster_steps(decapod, login)Function fixture to get cluster steps.

Parameters

• decapod (Decapod) – instatiated decapod application

• login (None) – user should log in before cluster actions

Returns instantiated cluster steps

Return type ClusterSteps

6.2. Decapod UI 17


whale.decapod_ui.conftest.ui_configuration_steps(decapod, login)Function fixture to get configuration steps.

Parameters

• decapod (Decapod) – instantiated decapod web application

• login (None) – should log in decapod before steps using

Returns instantiated configuration steps

Return type ConfigurationSteps

whale.decapod_ui.conftest.ui_role_steps(decapod, login)Function fixture to get role steps.

Parameters



Returns Instantiated role steps

Return type RoleSteps

whale.decapod_ui.conftest.ui_user_steps(decapod, login)Function fixture to get user steps.

Parameters



Returns Instantiated user steps

Return type UserSteps

whale.decapod_ui.conftest.video_capture(virtual_display, report_dir)Capture video of test.

whale.decapod_ui.conftest.virtual_display(request)Run test in virtual X server if env var is defined.

Decapod steps

Contains steps specific for decapod (UI testing).

class whale.decapod_ui.steps.AuthSteps(app)Authentication steps.

check_alert_present()Step to check alert message is present.

login(username=’login’, password=’password’, check=True)Step to log in user account.

Parameters

• username (-) – string, user name.

• password (-) – string, user password.

logout(check=True)Step to log out user account.



class whale.decapod_ui.steps.ClusterSteps(app)Cluster steps.

create_cluster(name=None, check=True)Step to create cluster.

Parameters

• name (str, optional) – name of cluster

• check (bool, optional) – flag whether to check step or not

Returns cluster name

Return type str

Raises Exception – if cluster is not present on page

update_cluster(name, new_name=None, check=True)Step to create cluster.

Parameters

• name (str) – name of cluster to be updated

• new_name (str, optional) – name of cluster after update


Returns cluster name

Return type str

Raises Exception – if cluster name was not changed

class whale.decapod_ui.steps.RoleSteps(app)Role steps.

create_role(role_name=None, check=True)Step to create role.

Parameters

• role_name (str) – role name


Returns role name

Return type str

Raises Exception – if role is not present on page

delete_role(role_name, check=True)Step to delete role.

Parameters



Raises Exception – if role is present on page

update_role(role_name, new_role_name=None, api_permissions=None, play-book_permissions=None, check=True)

Step to update role.

Parameters

6.2. Decapod UI 19



• new_role_name (str|None) – new role name

• api_permissions (list|None) – list of api permissions

• playbook_permissions (list|None) – list of playbook permissions


Returns role name

Return type str

Raises Exception – if role is not present on page

class whale.decapod_ui.steps.UserSteps(app)User steps.

create_user(role_name, login=None, full_name=None, check=True)Step to create user.

Parameters

• role_name (str) – role name of user

• login (str, optional) – login of user

• full_name (str, optional) – full name of user


Returns user login

Return type str

Raises Exception – if user is not present on page

delete_user(login, check=True)Step to delete user.

Parameters

• login (str) – login of user to be deleted


Raises Exception – if user is present on page

update_user(login, new_login=None, new_full_name=None, new_email=None,new_role_name=None, check=True)

Step to update user.

Parameters

• login (str, optional) – login of user to be changed

• new_login (str, optional) – new login of user

• new_full_name (str, optional) – new full name of user

• new_email (str, optional) – new email of user

• new_role_name (str, optional) – new role of user


Returns user login

Return type str



Raises Exception – if check failed

Decapod tests

Auth tests

whale.decapod_ui.tests.test_auth.test_login_logout(auth_steps)Scenario: User can log in and log out to/from Decapod UI.

Steps:

1.User logs in

2.User logs out

Cluster tests

whale.decapod_ui.tests.test_clusters.test_create_cluster(ui_cluster_steps, clus-ter_steps)

Scenario: Cluster may be created in UI.

Steps:

1.Create cluster via UI

2.Find cluster by name via API

Teardown:

1.Delete cluster via API

whale.decapod_ui.tests.test_clusters.test_update_cluster(cluster, ui_cluster_steps,cluster_steps)

Scenario: Cluster may be updated in UI.

Setup:

1.Create cluster via API

Steps:

1.Update cluster name via UI

2.Find cluster by name via API

Teardown:


Playbook configuration tests

whale.decapod_ui.tests.test_configuration.test_create_config_deploy_cluster(cluster,play-book_steps,server_steps,play-book_config_steps,ui_configuration_steps)

Scenario: Playbook configuration may be created in UI.

Setup:

6.2. Decapod UI 21



Steps:

1.Get servers list via API

2.Find playbook name by ID using API

3.Create playbook cluster deploy configuration via UI

Teardown:

1.Delete playbook configuration via API


whale.decapod_ui.tests.test_configuration.test_delete_config(playbook_config_deploy,ui_configuration_steps)

Scenario: Playbook configuration may be deleted in UI.

Setup:


2.Create playbook cluster deploy configuration via API

Steps:

1.Delete playbook cluster deploy configuration via UI

Teardown:


whale.decapod_ui.tests.test_configuration.test_update_config_cluster_name(playbook_config_deploy,clus-ter_steps,ui_configuration_steps)

Scenario: Playbook configuration may be updated in UI.

Setup:



Steps:

1.Create second cluster via API

2.Update cluster name of playbook configuration using UI

Teardown:


2.Delete clusters via API



Execution tests

whale.decapod_ui.tests.test_execution.test_create_execution_deploy_cluster(playbook_config_deploy,ex-e-cu-tion_steps,ui_configuration_steps)

Scenario: Execution for cluster deployment may be created in UI.

Setup:



Steps:

1.Create execution via UI

2.Check execution was successfully completed via API

Teardown:



whale.decapod_ui.tests.test_execution.test_create_execution_purge_cluster(cluster_steps,de-ploy_cluster,play-book_config_steps,ex-e-cu-tion_steps,ui_configuration_steps)

Scenario: Execution for cluster deletion may be created in UI.

Setup:


2.Create playbook configuration for cluster deployment via API

3.Create execution for cluster deployment via API

Steps:

1.Create playbook configuration for cluster deletion via API

2.Create execution for cluster deletion via UI

3.Check cluster has been deleted

Teardown:

1.Delete playbook configurations

User UI tests

whale.decapod_ui.tests.test_user.test_create_user(role, user_steps, ui_user_steps)Scenario: User may be created in UI.

6.2. Decapod UI 23


Setup:

1.Create role using API

Steps:

1.Create user via UI

2.Find user by login via API

Teardown:

1.Delete user via API

2.Delete role using API

whale.decapod_ui.tests.test_user.test_delete_user(user, ui_user_steps)Scenario: User may be deleted in UI.

Setup:


2.Create user via API

Steps:

1.Delete user via UI

Teardown:

1.Delete role using API

whale.decapod_ui.tests.test_user.test_update_user(user, role_steps, user_steps,ui_user_steps)

Scenario: User may be updated in UI.

Setup:


2.Create user via API

Steps:

1.Create new role using API

2.Update user login, full name, email and role via UI

3.Find user by login via API

Teardown:

1.Delete user via API

2.Delete created roles using API


Python Module Index

wwhale.decapod.tests.test_clusters, 11whale.decapod.tests.test_executions, 12whale.decapod.tests.test_playbook_configs,

14whale.decapod.tests.test_playbooks, 15whale.decapod.tests.test_roles, 15whale.decapod.tests.test_servers, 16whale.decapod.tests.test_users, 16whale.decapod_ui.conftest, 17whale.decapod_ui.steps, 18whale.decapod_ui.tests.test_auth, 21whale.decapod_ui.tests.test_clusters,

21whale.decapod_ui.tests.test_configuration,

21whale.decapod_ui.tests.test_execution,

22whale.decapod_ui.tests.test_user, 23

25


26 Python Module Index

Index

Aauth_steps() (in module whale.decapod_ui.conftest), 17AuthSteps (class in whale.decapod_ui.steps), 18

Ccheck_alert_present() (whale.decapod_ui.steps.AuthSteps

method), 18ClusterSteps (class in whale.decapod_ui.steps), 18create_cluster() (whale.decapod_ui.steps.ClusterSteps

method), 19create_role() (whale.decapod_ui.steps.RoleSteps

method), 19create_user() (whale.decapod_ui.steps.UserSteps

method), 20

Ddecapod() (in module whale.decapod_ui.conftest), 17delete_role() (whale.decapod_ui.steps.RoleSteps

method), 19delete_user() (whale.decapod_ui.steps.UserSteps

method), 20

Llogin() (in module whale.decapod_ui.conftest), 17login() (whale.decapod_ui.steps.AuthSteps method), 18logout() (whale.decapod_ui.steps.AuthSteps method), 18

Rreport_dir() (in module whale.decapod_ui.conftest), 17RoleSteps (class in whale.decapod_ui.steps), 19

Ttest_create_and_delete_role() (in module

whale.decapod.tests.test_roles), 15test_create_cluster() (in module

whale.decapod_ui.tests.test_clusters), 21test_create_config_deploy_cluster() (in module

whale.decapod_ui.tests.test_configuration),21

test_create_delete_cluster() (in modulewhale.decapod.tests.test_clusters), 11

test_create_delete_playbook_config() (in modulewhale.decapod.tests.test_playbook_configs),14

test_create_delete_user() (in modulewhale.decapod.tests.test_users), 16

test_create_execution_deploy_cluster() (in modulewhale.decapod_ui.tests.test_execution), 23

test_create_execution_purge_cluster() (in modulewhale.decapod_ui.tests.test_execution), 23

test_create_user() (in modulewhale.decapod_ui.tests.test_user), 23

test_delete_config() (in modulewhale.decapod_ui.tests.test_configuration),22

test_delete_create_server() (in modulewhale.decapod.tests.test_servers), 16

test_delete_user() (in modulewhale.decapod_ui.tests.test_user), 24

test_deploy_cluster_add_osd_monitor_telegraf() (inmodule whale.decapod.tests.test_executions),12

test_deploy_cluster_add_rest_api_rados_gateway() (inmodule whale.decapod.tests.test_executions),13

test_deploy_cluster_integrate_cinder_upgrade_ceph() (inmodule whale.decapod.tests.test_executions),13

test_get_cluster() (in modulewhale.decapod.tests.test_clusters), 11

test_get_playbook_config() (in modulewhale.decapod.tests.test_playbook_configs),14

test_get_role() (in modulewhale.decapod.tests.test_roles), 15

test_get_server() (in modulewhale.decapod.tests.test_servers), 16

test_get_user() (in modulewhale.decapod.tests.test_users), 16

test_list_clusters() (in module

27


whale.decapod.tests.test_clusters), 11test_list_permissions() (in module

whale.decapod.tests.test_roles), 15test_list_playbook_configs() (in module

whale.decapod.tests.test_playbook_configs),14

test_list_playbooks() (in modulewhale.decapod.tests.test_playbooks), 15

test_list_roles() (in modulewhale.decapod.tests.test_roles), 15

test_list_servers() (in modulewhale.decapod.tests.test_servers), 16

test_list_users() (in modulewhale.decapod.tests.test_users), 17

test_login_logout() (in modulewhale.decapod_ui.tests.test_auth), 21

test_update_cluster() (in modulewhale.decapod.tests.test_clusters), 11

test_update_cluster() (in modulewhale.decapod_ui.tests.test_clusters), 21

test_update_config_cluster_name() (in modulewhale.decapod_ui.tests.test_configuration),22

test_update_playbook_config() (in modulewhale.decapod.tests.test_playbook_configs),14

test_update_role() (in modulewhale.decapod.tests.test_roles), 15

test_update_server() (in modulewhale.decapod.tests.test_servers), 16

test_update_user() (in modulewhale.decapod.tests.test_users), 17

test_update_user() (in modulewhale.decapod_ui.tests.test_user), 24

Uui_cluster_steps() (in module

whale.decapod_ui.conftest), 17ui_configuration_steps() (in module

whale.decapod_ui.conftest), 17ui_role_steps() (in module whale.decapod_ui.conftest),

18ui_user_steps() (in module whale.decapod_ui.conftest),

18update_cluster() (whale.decapod_ui.steps.ClusterSteps

method), 19update_role() (whale.decapod_ui.steps.RoleSteps

method), 19update_user() (whale.decapod_ui.steps.UserSteps

method), 20UserSteps (class in whale.decapod_ui.steps), 20

Vvideo_capture() (in module whale.decapod_ui.conftest),

18virtual_display() (in module whale.decapod_ui.conftest),

18

Wwhale.decapod.tests.test_clusters (module), 11whale.decapod.tests.test_executions (module), 12whale.decapod.tests.test_playbook_configs (module), 14whale.decapod.tests.test_playbooks (module), 15whale.decapod.tests.test_roles (module), 15whale.decapod.tests.test_servers (module), 16whale.decapod.tests.test_users (module), 16whale.decapod_ui.conftest (module), 17whale.decapod_ui.steps (module), 18whale.decapod_ui.tests.test_auth (module), 21whale.decapod_ui.tests.test_clusters (module), 21whale.decapod_ui.tests.test_configuration (module), 21whale.decapod_ui.tests.test_execution (module), 22whale.decapod_ui.tests.test_user (module), 23

28 Index

Documents

whale Documentation