path: root/docs/testing/user/userguide/runfunctest.rst

.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0

Executing the functest suites
=============================

Manual testing
--------------

This section assumes the following:
 * The Functest Docker container is running
 * The docker prompt is shown
 * The Functest environment is ready (Functest CLI command 'functest env prepare'
   has been executed)

If any of the above steps are missing please refer to the Functest Config Guide
as they are a prerequisite and all the commands explained in this section **must** be
performed **inside the container**.

The Functest CLI offers two commands (functest tier ...) and (functest testcase ... )
for the execution of Test Tiers or Test Cases::

  root@22e436918db0:~/repos/functest/ci# functest tier --help
  Usage: functest tier [OPTIONS] COMMAND [ARGS]...

  Options:
    -h, --help  Show this message and exit.

  Commands:
    get-tests  Prints the tests in a tier.
    list       Lists the available tiers.
    run        Executes all the tests within a tier.
    show       Shows information about a tier.
  root@22e436918db0:~/repos/functest/ci# functest testcase --help

  Usage: functest testcase [OPTIONS] COMMAND [ARGS]...

  Options:
    -h, --help  Show this message and exit.

  Commands:
    list  Lists the available testcases.
    run   Executes a test case.
    show  Shows information about a test case.

More details on the existing Tiers and Test Cases can be seen with the 'list'
command::

  root@22e436918db0:~/repos/functest/ci# functest tier list
      - 0. healthcheck:
             ['connection_check', 'api_check', 'snaps_health_check',]
      - 1. smoke:
             ['vping_ssh', 'vping_userdata', 'tempest_smoke_serial', 'odl', 'rally_sanity', 'refstack_defcore', 'snaps_smoke']
      - 2. features:
             ['doctor', 'domino', 'promise', security_scan']
      - 3. components:
             ['tempest_full_parallel', 'rally_full']
      - 4. vnf:
             ['cloudify_ims', 'orchestra_ims', 'vyos_vrouter']

  and

  root@22e436918db0:~/repos/functest/ci# functest testcase list
  api_check
  connection_check
  snaps_health_check
  vping_ssh
  vping_userdata
  snaps_smoke
  refstack_defcore
  tempest_smoke_serial
  rally_sanity
  odl
  tempest_full_parallel
  rally_full
  vyos_vrouter

Note the list of test cases depend on the installer and the scenario.

More specific details on specific Tiers or Test Cases can be seen wih the
'show' command::

  root@22e436918db0:~/repos/functest/ci# functest tier show smoke
  +======================================================================+
  | Tier:  smoke                                                         |
  +======================================================================+
  | Order: 1                                                             |
  | CI Loop: (daily)|(weekly)                                            |
  | Description:                                                         |
  |    Set of basic Functional tests to validate the OpenStack           |
  |    deployment.                                                       |
  | Test cases:                                                          |
  |    - vping_ssh                                                       |
  |    - vping_userdata                                                  |
  |    - tempest_smoke_serial                                            |
  |    - rally_sanity                                                    |
  |                                                                      |
  +----------------------------------------------------------------------+

  and

  root@22e436918db0:~/repos/functest/ci# functest testcase  show tempest_smoke_serial
  +======================================================================+
  | Testcase:  tempest_smoke_serial                                      |
  +======================================================================+
  | Description:                                                         |
  |    This test case runs the smoke subset of the OpenStack Tempest     |
  |    suite. The list of test cases is generated by Tempest             |
  |    automatically and depends on the parameters of the OpenStack      |
  |    deplopyment.                                                      |
  | Dependencies:                                                        |
  |   - Installer:                                                       |
  |   - Scenario :                                                       |
  |                                                                      |
  +----------------------------------------------------------------------+


To execute a Test Tier or Test Case, the 'run' command is used::

  root@22e436918db0:~/repos/functest/ci# functest tier run healthcheck
  2017-03-30 05:36:19,752 - run_tests - INFO - ############################################
  2017-03-30 05:36:19,752 - run_tests - INFO - Running tier 'healthcheck'
  2017-03-30 05:36:19,753 - run_tests - INFO - ############################################
  2017-03-30 05:36:19,753 - run_tests - INFO -

  2017-03-30 05:36:19,753 - run_tests - INFO - ============================================
  2017-03-30 05:36:19,753 - run_tests - INFO - Running test case 'connection_check'...
  2017-03-30 05:36:19,753 - run_tests - INFO - ============================================
  2017-03-30 05:36:20,046 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,046 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,775 - functest - INFO - Using flavor metatdata '{'hw:mem_page_size': 'any'}'
  2017-03-30 05:36:20,777 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,777 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,777 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,778 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,778 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,779 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,779 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,779 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,780 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,780 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,781 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,781 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,781 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,782 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,782 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,783 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:20,783 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:20,784 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  test_glance_connect_fail (snaps.openstack.utils.tests.glance_utils_tests.GlanceSmokeTests) ... ok
  test_glance_connect_success (snaps.openstack.utils.tests.glance_utils_tests.GlanceSmokeTests) ... ok
  test_keystone_connect_fail (snaps.openstack.utils.tests.keystone_utils_tests.KeystoneSmokeTests) ... ok
  test_keystone_connect_success (snaps.openstack.utils.tests.keystone_utils_tests.KeystoneSmokeTests) ... ok
  test_neutron_connect_fail (snaps.openstack.utils.tests.neutron_utils_tests.NeutronSmokeTests) ... ok
  test_neutron_connect_success (snaps.openstack.utils.tests.neutron_utils_tests.NeutronSmokeTests) ... ok
  test_retrieve_ext_network_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronSmokeTests) ... ok
  test_nova_connect_fail (snaps.openstack.utils.tests.nova_utils_tests.NovaSmokeTests) ... ok
  test_nova_connect_success (snaps.openstack.utils.tests.nova_utils_tests.NovaSmokeTests) ... ok

  ----------------------------------------------------------------------
  Ran 9 tests in 1.332s

  OK
  2017-03-30 05:36:22,116 - functest - INFO - connection_check OK
  2017-03-30 05:36:22,483 - functest - INFO - The results were successfully pushed to DB
  2017-03-30 05:36:22,483 - run_tests - INFO - Test execution time: 00:02
  2017-03-30 05:36:22,484 - run_tests - INFO -

  2017-03-30 05:36:22,484 - run_tests - INFO - ============================================
  2017-03-30 05:36:22,484 - run_tests - INFO - Running test case 'api_check'...
  2017-03-30 05:36:22,484 - run_tests - INFO - ============================================
  2017-03-30 05:36:22,590 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:22,591 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,361 - functest - INFO - Using flavor metatdata '{'hw:mem_page_size': 'any'}'
  2017-03-30 05:36:23,362 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,362 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,363 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,364 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,364 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,365 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,365 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,365 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,366 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,366 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,367 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,367 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,368 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,368 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,368 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,369 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,370 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,370 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,370 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,371 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,372 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,372 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,372 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,373 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,373 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,374 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,374 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,374 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,375 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,376 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,376 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,376 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,376 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,377 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,377 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,377 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,377 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,377 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,378 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,378 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,378 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,378 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,379 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,379 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,379 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,380 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,380 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,380 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,380 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,380 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,381 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,381 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,381 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,381 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,381 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,382 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,382 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,382 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,382 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,382 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,383 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,383 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,383 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,384 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,384 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,384 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,384 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,384 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,385 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,385 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,385 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,385 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,386 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,386 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,386 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,386 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,387 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,387 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,387 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,388 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,388 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,388 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,388 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,388 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,420 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,420 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,420 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,420 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,421 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,421 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,421 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,421 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,422 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,422 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:36:23,422 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:36:23,422 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  test_create_project_minimal (snaps.openstack.utils.tests.keystone_utils_tests.KeystoneUtilsTests) ... ok
  test_create_user_minimal (snaps.openstack.utils.tests.keystone_utils_tests.KeystoneUtilsTests) ... ok
  test_create_delete_user (snaps.openstack.tests.create_user_tests.CreateUserSuccessTests) ... ok
  test_create_user (snaps.openstack.tests.create_user_tests.CreateUserSuccessTests) ... ok
  test_create_user_2x (snaps.openstack.tests.create_user_tests.CreateUserSuccessTests) ... 2017-03-30 05:36:27,257 - create_user - INFO - Found user with name - CreateUserSuccessTests-5ff765c1-56bd-4c4-name
  ok
  test_create_delete_project (snaps.openstack.tests.create_project_tests.CreateProjectSuccessTests) ... ok
  test_create_project (snaps.openstack.tests.create_project_tests.CreateProjectSuccessTests) ... ok
  test_create_project_2x (snaps.openstack.tests.create_project_tests.CreateProjectSuccessTests) ... 2017-03-30 05:36:29,798 - create_image - INFO - Found project with name - CreateProjectSuccessTests-1b2fce89-dd5e-471-name
  ok
  test_create_project_sec_grp_one_user (snaps.openstack.tests.create_project_tests.CreateProjectUserTests) ... 2017-03-30 05:36:31,327 - OpenStackSecurityGroup - INFO - Creating security group CreateProjectUserTests-34aa7d96-f19c-4db-name...
  2017-03-30 05:36:31,327 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-34aa7d96-f19c-4db-name
  2017-03-30 05:36:31,705 - neutron_utils - INFO - Creating security group with name - CreateProjectUserTests-34aa7d96-f19c-4db-name
  2017-03-30 05:36:31,878 - neutron_utils - INFO - Retrieving security group rules associate with the security group - CreateProjectUserTests-34aa7d96-f19c-4db-name
  2017-03-30 05:36:31,915 - neutron_utils - INFO - Retrieving security group with ID - 4dc3e8e4-3dc8-4dda-9c7e-03d08171e17a
  2017-03-30 05:36:31,980 - neutron_utils - INFO - Retrieving security group with ID - 4dc3e8e4-3dc8-4dda-9c7e-03d08171e17a
  2017-03-30 05:36:32,048 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-34aa7d96-f19c-4db-name
  2017-03-30 05:36:32,108 - neutron_utils - INFO - Deleting security group rule with ID - 81cca252-45fe-4052-adb9-819191693618
  2017-03-30 05:36:32,257 - neutron_utils - INFO - Deleting security group rule with ID - 8900647f-1587-4068-bd2e-7b77677d12ed
  2017-03-30 05:36:32,367 - neutron_utils - INFO - Deleting security group with name - CreateProjectUserTests-34aa7d96-f19c-4db-name
  ok
  test_create_project_sec_grp_two_users (snaps.openstack.tests.create_project_tests.CreateProjectUserTests) ... 2017-03-30 05:36:34,950 - OpenStackSecurityGroup - INFO - Creating security group CreateProjectUserTests-6664b595-4657-4f9-name...
  2017-03-30 05:36:34,950 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-6664b595-4657-4f9-name
  2017-03-30 05:36:35,337 - neutron_utils - INFO - Creating security group with name - CreateProjectUserTests-6664b595-4657-4f9-name
  2017-03-30 05:36:35,528 - neutron_utils - INFO - Retrieving security group rules associate with the security group - CreateProjectUserTests-6664b595-4657-4f9-name
  2017-03-30 05:36:35,566 - neutron_utils - INFO - Retrieving security group with ID - ea8b1da3-0e3d-45aa-b63f-68d2e7b57e48
  2017-03-30 05:36:35,831 - neutron_utils - INFO - Retrieving security group with ID - ea8b1da3-0e3d-45aa-b63f-68d2e7b57e48
  2017-03-30 05:36:36,118 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-6664b595-4657-4f9-name
  2017-03-30 05:36:36,474 - OpenStackSecurityGroup - INFO - Creating security group CreateProjectUserTests-6664b595-4657-4f9-name...
  2017-03-30 05:36:36,475 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-6664b595-4657-4f9-name
  2017-03-30 05:36:36,717 - neutron_utils - INFO - Retrieving security group rules associate with the security group - CreateProjectUserTests-6664b595-4657-4f9-name
  2017-03-30 05:36:36,768 - neutron_utils - INFO - Retrieving security group with ID - ea8b1da3-0e3d-45aa-b63f-68d2e7b57e48
  2017-03-30 05:36:36,831 - neutron_utils - INFO - Retrieving security group with ID - ea8b1da3-0e3d-45aa-b63f-68d2e7b57e48
  2017-03-30 05:36:36,902 - neutron_utils - INFO - Deleting security group rule with ID - f6e50aea-e6d2-4ba9-ab78-0674cdcd5415
  2017-03-30 05:36:37,054 - neutron_utils - INFO - Deleting security group rule with ID - a32d9c1d-7ae0-4fe9-b4c9-8b039008f836
  2017-03-30 05:36:37,204 - neutron_utils - INFO - Deleting security group with name - CreateProjectUserTests-6664b595-4657-4f9-name
  2017-03-30 05:36:37,350 - neutron_utils - INFO - Deleting security group rule with ID - f6e50aea-e6d2-4ba9-ab78-0674cdcd5415
  2017-03-30 05:36:37,387 - OpenStackSecurityGroup - WARNING - Rule not found, cannot delete - Security group rule f6e50aea-e6d2-4ba9-ab78-0674cdcd5415 does not exist
  Neutron server returns request_ids: ['req-e740871d-34c4-4b95-a76c-6b84028954e6']
  2017-03-30 05:36:37,387 - neutron_utils - INFO - Deleting security group rule with ID - a32d9c1d-7ae0-4fe9-b4c9-8b039008f836
  2017-03-30 05:36:37,426 - OpenStackSecurityGroup - WARNING - Rule not found, cannot delete - Security group rule a32d9c1d-7ae0-4fe9-b4c9-8b039008f836 does not exist
  Neutron server returns request_ids: ['req-8121308e-d7d3-4ccc-961f-5fa794fccc83']
  2017-03-30 05:36:37,427 - neutron_utils - INFO - Deleting security group with name - CreateProjectUserTests-6664b595-4657-4f9-name
  2017-03-30 05:36:37,470 - OpenStackSecurityGroup - WARNING - Security Group not found, cannot delete - Security group ea8b1da3-0e3d-45aa-b63f-68d2e7b57e48 does not exist
  Neutron server returns request_ids: ['req-09424914-a32d-4bcb-9d90-0ad307ec4c56']
  ok
  test_create_image_minimal_file (snaps.openstack.utils.tests.glance_utils_tests.GlanceUtilsTests) ... ok
  test_create_image_minimal_url (snaps.openstack.utils.tests.glance_utils_tests.GlanceUtilsTests) ... ok
  test_create_network (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsNetworkTests) ... 2017-03-30 05:37:02,330 - neutron_utils - INFO - Creating network with name NeutronUtilsNetworkTests-9dc31d5e-be87-480d-af6e-d89f0608e459-pub-net
  2017-03-30 05:37:04,307 - neutron_utils - INFO - Deleting network with name NeutronUtilsNetworkTests-9dc31d5e-be87-480d-af6e-d89f0608e459-pub-net
  ok
  test_create_network_empty_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsNetworkTests) ... ok
  test_create_network_null_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsNetworkTests) ... ok
  test_create_subnet (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... 2017-03-30 05:37:04,953 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-127e135b-f050-4e85-8c9f-b4f72fb1b028-pub-net
  2017-03-30 05:37:06,108 - neutron_utils - INFO - Creating subnet with name NeutronUtilsSubnetTests-127e135b-f050-4e85-8c9f-b4f72fb1b028-pub-subnet
  2017-03-30 05:37:07,544 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsSubnetTests-127e135b-f050-4e85-8c9f-b4f72fb1b028-pub-subnet
  2017-03-30 05:37:07,944 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-127e135b-f050-4e85-8c9f-b4f72fb1b028-pub-net
  ok
  test_create_subnet_empty_cidr (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... 2017-03-30 05:37:08,594 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-4edb48fe-2532-409b-8dc7-dcb344068a20-pub-net
  2017-03-30 05:37:09,862 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-4edb48fe-2532-409b-8dc7-dcb344068a20-pub-net
  ok
  test_create_subnet_empty_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... 2017-03-30 05:37:10,962 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-bb15908a-a475-45e9-9777-8b5d3faaaea8-pub-net
  2017-03-30 05:37:11,973 - neutron_utils - INFO - Creating subnet with name NeutronUtilsSubnetTests-bb15908a-a475-45e9-9777-8b5d3faaaea8-pub-subnet
  2017-03-30 05:37:13,088 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-bb15908a-a475-45e9-9777-8b5d3faaaea8-pub-net
  ok
  test_create_subnet_null_cidr (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... 2017-03-30 05:37:14,032 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-701278ff-f4b6-478f-b16f-1d3fdfb43ed7-pub-net
  2017-03-30 05:37:15,100 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-701278ff-f4b6-478f-b16f-1d3fdfb43ed7-pub-net
  ok
  test_create_subnet_null_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... 2017-03-30 05:37:15,658 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-2dad9c37-c892-494b-a8dc-51963ce11cd8-pub-net
  2017-03-30 05:37:16,184 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-2dad9c37-c892-494b-a8dc-51963ce11cd8-pub-net
  ok
  test_add_interface_router (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:17,438 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-5f1f0b29-1148-4628-8626-f2aa63b17914-pub-net
  2017-03-30 05:37:18,624 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-5f1f0b29-1148-4628-8626-f2aa63b17914-pub-subnet
  2017-03-30 05:37:20,041 - neutron_utils - INFO - Creating router with name - NeutronUtilsRouterTests-5f1f0b29-1148-4628-8626-f2aa63b17914-pub-router
  2017-03-30 05:37:22,518 - neutron_utils - INFO - Adding interface to router with name NeutronUtilsRouterTests-5f1f0b29-1148-4628-8626-f2aa63b17914-pub-router
  2017-03-30 05:37:23,883 - neutron_utils - INFO - Removing router interface from router named NeutronUtilsRouterTests-5f1f0b29-1148-4628-8626-f2aa63b17914-pub-router
  2017-03-30 05:37:25,345 - neutron_utils - INFO - Deleting router with name - NeutronUtilsRouterTests-5f1f0b29-1148-4628-8626-f2aa63b17914-pub-router
  2017-03-30 05:37:27,019 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-5f1f0b29-1148-4628-8626-f2aa63b17914-pub-subnet
  2017-03-30 05:37:28,570 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-5f1f0b29-1148-4628-8626-f2aa63b17914-pub-net
  ok
  test_add_interface_router_null_router (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:29,583 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-021a41c7-e2b2-45df-bb6d-630ddba2b776-pub-net
  2017-03-30 05:37:30,234 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-021a41c7-e2b2-45df-bb6d-630ddba2b776-pub-subnet
  2017-03-30 05:37:30,724 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-021a41c7-e2b2-45df-bb6d-630ddba2b776-pub-subnet
  2017-03-30 05:37:31,128 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-021a41c7-e2b2-45df-bb6d-630ddba2b776-pub-net
  ok
  test_add_interface_router_null_subnet (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:32,229 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-e52f96b6-92c9-49a3-ac7f-6a4a61a82c7e-pub-net
  2017-03-30 05:37:32,833 - neutron_utils - INFO - Creating router with name - NeutronUtilsRouterTests-e52f96b6-92c9-49a3-ac7f-6a4a61a82c7e-pub-router
  2017-03-30 05:37:34,002 - neutron_utils - INFO - Adding interface to router with name NeutronUtilsRouterTests-e52f96b6-92c9-49a3-ac7f-6a4a61a82c7e-pub-router
  2017-03-30 05:37:34,003 - neutron_utils - INFO - Deleting router with name - NeutronUtilsRouterTests-e52f96b6-92c9-49a3-ac7f-6a4a61a82c7e-pub-router
  2017-03-30 05:37:35,238 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-e52f96b6-92c9-49a3-ac7f-6a4a61a82c7e-pub-net
  ok
  test_create_port (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:35,898 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-842cf533-4886-4539-86e4-15bcd8c77b63-pub-net
  2017-03-30 05:37:36,426 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-842cf533-4886-4539-86e4-15bcd8c77b63-pub-subnet
  2017-03-30 05:37:37,725 - neutron_utils - INFO - Creating port for network with name - NeutronUtilsRouterTests-842cf533-4886-4539-86e4-15bcd8c77b63-pub-net
  2017-03-30 05:37:38,511 - neutron_utils - INFO - Deleting port with name NeutronUtilsRouterTests-842cf533-4886-4539-86e4-15bcd8c77b63-port
  2017-03-30 05:37:39,036 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-842cf533-4886-4539-86e4-15bcd8c77b63-pub-subnet
  2017-03-30 05:37:41,326 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-842cf533-4886-4539-86e4-15bcd8c77b63-pub-net
  ok
  test_create_port_empty_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:42,725 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-303b5eae-374a-4da9-a905-aa39a7d5f026-pub-net
  2017-03-30 05:37:43,246 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-303b5eae-374a-4da9-a905-aa39a7d5f026-pub-subnet
  2017-03-30 05:37:43,945 - neutron_utils - INFO - Creating port for network with name - NeutronUtilsRouterTests-303b5eae-374a-4da9-a905-aa39a7d5f026-pub-net
  2017-03-30 05:37:45,674 - neutron_utils - INFO - Deleting port with name NeutronUtilsRouterTests-303b5eae-374a-4da9-a905-aa39a7d5f026-port
  2017-03-30 05:37:46,197 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-303b5eae-374a-4da9-a905-aa39a7d5f026-pub-subnet
  2017-03-30 05:37:48,252 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-303b5eae-374a-4da9-a905-aa39a7d5f026-pub-net
  ok
  test_create_port_invalid_ip (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:49,864 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-bd7a4489-79e8-4328-8519-5ad1951c0b5d-pub-net
  2017-03-30 05:37:50,322 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-bd7a4489-79e8-4328-8519-5ad1951c0b5d-pub-subnet
  2017-03-30 05:37:50,803 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-bd7a4489-79e8-4328-8519-5ad1951c0b5d-pub-subnet
  2017-03-30 05:37:51,240 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-bd7a4489-79e8-4328-8519-5ad1951c0b5d-pub-net
  ok
  test_create_port_invalid_ip_to_subnet (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:51,767 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-ef2c3474-7b51-483b-b269-05fc4532f294-pub-net
  2017-03-30 05:37:52,246 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-ef2c3474-7b51-483b-b269-05fc4532f294-pub-subnet
  2017-03-30 05:37:52,795 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-ef2c3474-7b51-483b-b269-05fc4532f294-pub-subnet
  2017-03-30 05:37:53,199 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-ef2c3474-7b51-483b-b269-05fc4532f294-pub-net
  ok
  test_create_port_null_ip (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:53,806 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-f5829764-e1ff-4a43-b24e-52c0107f12b2-pub-net
  2017-03-30 05:37:54,326 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-f5829764-e1ff-4a43-b24e-52c0107f12b2-pub-subnet
  2017-03-30 05:37:55,475 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-f5829764-e1ff-4a43-b24e-52c0107f12b2-pub-subnet
  2017-03-30 05:37:55,932 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-f5829764-e1ff-4a43-b24e-52c0107f12b2-pub-net
  ok
  test_create_port_null_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:37:57,121 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-194aff34-e0e6-4218-8c17-33a9d9b34816-pub-net
  2017-03-30 05:37:57,611 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-194aff34-e0e6-4218-8c17-33a9d9b34816-pub-subnet
  2017-03-30 05:37:58,880 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-194aff34-e0e6-4218-8c17-33a9d9b34816-pub-subnet
  2017-03-30 05:37:59,638 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-194aff34-e0e6-4218-8c17-33a9d9b34816-pub-net
  ok
  test_create_port_null_network_object (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:38:01,037 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-3a49f710-5292-411b-83df-42bb176a5020-pub-net
  2017-03-30 05:38:01,609 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-3a49f710-5292-411b-83df-42bb176a5020-pub-subnet
  2017-03-30 05:38:02,095 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-3a49f710-5292-411b-83df-42bb176a5020-pub-subnet
  2017-03-30 05:38:03,034 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-3a49f710-5292-411b-83df-42bb176a5020-pub-net
  ok
  test_create_router_empty_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... ok
  test_create_router_null_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... ok
  test_create_router_simple (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:38:03,973 - neutron_utils - INFO - Creating router with name - NeutronUtilsRouterTests-54c3eaf0-00c8-4726-a248-b57f98a37999-pub-router
  2017-03-30 05:38:05,749 - neutron_utils - INFO - Deleting router with name - NeutronUtilsRouterTests-54c3eaf0-00c8-4726-a248-b57f98a37999-pub-router
  ok
  test_create_router_with_public_interface (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... 2017-03-30 05:38:07,392 - neutron_utils - INFO - Creating router with name - NeutronUtilsRouterTests-a4e93ee5-781e-4e9d-9b55-b4d8fb3f0e7b-pub-router
  2017-03-30 05:38:09,164 - neutron_utils - INFO - Deleting router with name - NeutronUtilsRouterTests-a4e93ee5-781e-4e9d-9b55-b4d8fb3f0e7b-pub-router
  ok
  test_create_delete_simple_sec_grp (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSecurityGroupTests) ... 2017-03-30 05:38:10,643 - neutron_utils - INFO - Creating security group with name - NeutronUtilsSecurityGroupTests-0b62acfe-fc43-4c9f-bd30-2ce350c73c57name
  2017-03-30 05:38:11,020 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-0b62acfe-fc43-4c9f-bd30-2ce350c73c57name
  2017-03-30 05:38:11,084 - neutron_utils - INFO - Deleting security group with name - NeutronUtilsSecurityGroupTests-0b62acfe-fc43-4c9f-bd30-2ce350c73c57name
  2017-03-30 05:38:11,230 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-0b62acfe-fc43-4c9f-bd30-2ce350c73c57name
  ok
  test_create_sec_grp_no_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSecurityGroupTests) ... ok
  test_create_sec_grp_no_rules (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSecurityGroupTests) ... 2017-03-30 05:38:11,653 - neutron_utils - INFO - Creating security group with name - NeutronUtilsSecurityGroupTests-19dc66ba-630d-4ef5-87e1-b0461971ef8ename
  2017-03-30 05:38:11,960 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-19dc66ba-630d-4ef5-87e1-b0461971ef8ename
  2017-03-30 05:38:12,047 - neutron_utils - INFO - Deleting security group with name - NeutronUtilsSecurityGroupTests-19dc66ba-630d-4ef5-87e1-b0461971ef8ename
  ok
  test_create_sec_grp_one_rule (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSecurityGroupTests) ... 2017-03-30 05:38:12,321 - neutron_utils - INFO - Creating security group with name - NeutronUtilsSecurityGroupTests-cf3022e5-dc6e-4cc1-8fe0-41c8c1c56defname
  2017-03-30 05:38:12,676 - neutron_utils - INFO - Retrieving security group rules associate with the security group - NeutronUtilsSecurityGroupTests-cf3022e5-dc6e-4cc1-8fe0-41c8c1c56defname
  2017-03-30 05:38:12,735 - neutron_utils - INFO - Creating security group to security group - NeutronUtilsSecurityGroupTests-cf3022e5-dc6e-4cc1-8fe0-41c8c1c56defname
  2017-03-30 05:38:12,736 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-cf3022e5-dc6e-4cc1-8fe0-41c8c1c56defname
  2017-03-30 05:38:12,948 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-cf3022e5-dc6e-4cc1-8fe0-41c8c1c56defname
  2017-03-30 05:38:13,024 - neutron_utils - INFO - Retrieving security group rules associate with the security group - NeutronUtilsSecurityGroupTests-cf3022e5-dc6e-4cc1-8fe0-41c8c1c56defname
  2017-03-30 05:38:13,054 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-cf3022e5-dc6e-4cc1-8fe0-41c8c1c56defname
  2017-03-30 05:38:13,121 - neutron_utils - INFO - Deleting security group rule with ID - 07b4bfbe-c632-496b-95f7-b42de9293519
  2017-03-30 05:38:13,238 - neutron_utils - INFO - Deleting security group rule with ID - c5e58c9f-6cc8-4543-ae39-aa1960b9a3e1
  2017-03-30 05:38:13,387 - neutron_utils - INFO - Deleting security group rule with ID - 184d29e8-b460-4f80-858f-7915b9bafe9b
  2017-03-30 05:38:13,492 - neutron_utils - INFO - Deleting security group with name - NeutronUtilsSecurityGroupTests-cf3022e5-dc6e-4cc1-8fe0-41c8c1c56defname
  ok
  test_create_delete_keypair (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsKeypairTests) ... 2017-03-30 05:38:13,664 - nova_utils - INFO - Creating keypair with name - NovaUtilsKeypairTests-cb36f8f9-ceca-4802-8735-a1dc846ad2a8
  ok
  test_create_key_from_file (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsKeypairTests) ... 2017-03-30 05:38:18,337 - nova_utils - INFO - Saved public key to - tmp/NovaUtilsKeypairTests-a7eba01b-9615-4271-b5f9-8fe915972f16.pub
  2017-03-30 05:38:18,338 - nova_utils - INFO - Saved private key to - tmp/NovaUtilsKeypairTests-a7eba01b-9615-4271-b5f9-8fe915972f16
  2017-03-30 05:38:18,338 - nova_utils - INFO - Saving keypair to - tmp/NovaUtilsKeypairTests-a7eba01b-9615-4271-b5f9-8fe915972f16.pub
  2017-03-30 05:38:18,338 - nova_utils - INFO - Creating keypair with name - NovaUtilsKeypairTests-a7eba01b-9615-4271-b5f9-8fe915972f16
  ok
  test_create_keypair (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsKeypairTests) ... 2017-03-30 05:38:21,492 - nova_utils - INFO - Creating keypair with name - NovaUtilsKeypairTests-74535dab-d450-47b1-8814-c0b3f48b7643
  ok
  test_floating_ips (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsKeypairTests) ... 2017-03-30 05:38:23,509 - nova_utils - INFO - Creating floating ip to external network - admin_floating_net
  ok
  test_create_delete_flavor (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsFlavorTests) ... ok
  test_create_flavor (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsFlavorTests) ... ok
  test_create_clean_flavor (snaps.openstack.tests.create_flavor_tests.CreateFlavorTests) ... ok
  test_create_delete_flavor (snaps.openstack.tests.create_flavor_tests.CreateFlavorTests) ... ok
  test_create_flavor (snaps.openstack.tests.create_flavor_tests.CreateFlavorTests) ... ok
  test_create_flavor_existing (snaps.openstack.tests.create_flavor_tests.CreateFlavorTests) ... 2017-03-30 05:38:34,855 - create_image - INFO - Found flavor with name - CreateFlavorTests-3e389b6c-ee95-4f2d-bf74-78d324722ef2name
  ok

  ----------------------------------------------------------------------
  Ran 48 tests in 131.483s

  OK
  2017-03-30 05:38:34,905 - functest - INFO - api_check OK
  2017-03-30 05:38:35,259 - functest - INFO - The results were successfully pushed to DB
  2017-03-30 05:38:35,259 - run_tests - INFO - Test execution time: 02:12
  2017-03-30 05:38:35,267 - run_tests - INFO -

  2017-03-30 05:38:35,268 - run_tests - INFO - ============================================
  2017-03-30 05:38:35,268 - run_tests - INFO - Running test case 'snaps_health_check'...
  2017-03-30 05:38:35,268 - run_tests - INFO - ============================================
  2017-03-30 05:38:35,383 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:38:35,384 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  2017-03-30 05:38:35,855 - functest - INFO - Using flavor metatdata '{'hw:mem_page_size': 'any'}'
  2017-03-30 05:38:35,856 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds
  2017-03-30 05:38:35,856 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.2:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None
  test_check_vm_ip_dhcp (snaps.openstack.tests.create_instance_tests.SimpleHealthCheck) ... 2017-03-30 05:38:39,896 - create_image - INFO - Creating image
  2017-03-30 05:38:41,843 - create_image - INFO - Image is active with name - SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-image
  2017-03-30 05:38:41,843 - create_image - INFO - Image is now active with name - SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-image
  2017-03-30 05:38:41,845 - OpenStackNetwork - INFO - Creating neutron network SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-priv-net...
  2017-03-30 05:38:42,140 - neutron_utils - INFO - Creating network with name SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-priv-net
  2017-03-30 05:38:42,480 - neutron_utils - INFO - Creating subnet with name SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-priv-subnet
  2017-03-30 05:38:44,166 - neutron_utils - INFO - Creating port for network with name - SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-priv-net
  2017-03-30 05:38:45,173 - create_instance - INFO - Creating VM with name - SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-inst
  2017-03-30 05:38:48,419 - create_instance - INFO - Created instance with name - SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-inst
  2017-03-30 05:39:05,164 - create_instance - INFO - VM is - ACTIVE
  2017-03-30 05:39:05,164 - create_instance_tests - INFO - Looking for expression Lease of.*obtained in the console log
  2017-03-30 05:39:06,547 - create_instance_tests - INFO - DHCP lease obtained logged in console
  2017-03-30 05:39:06,548 - create_instance_tests - INFO - With correct IP address
  2017-03-30 05:39:06,548 - create_instance - INFO - Deleting Port - SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181port-1
  2017-03-30 05:39:06,548 - neutron_utils - INFO - Deleting port with name SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181port-1
  2017-03-30 05:39:07,178 - create_instance - INFO - Deleting VM instance - SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-inst
  2017-03-30 05:39:07,693 - create_instance - INFO - Checking deletion status
  2017-03-30 05:39:11,088 - create_instance - INFO - VM has been properly deleted VM with name - SimpleHealthCheck-85a41a34-a9b3-463d-a12c-4bd057d70181-inst
  ok

  ----------------------------------------------------------------------
  Ran 1 test in 36.376s

  OK
  2017-03-30 05:39:12,233 - functest - INFO - snaps_health_check OK
  2017-03-30 05:39:12,598 - functest - INFO - The results were successfully pushed to DB
  2017-03-30 05:39:12,598 - run_tests - INFO - Test execution time: 00:37
  2017-03-30 05:39:12,599 - run_tests - INFO -

  and

  root@22e436918db0:~/repos/functest/ci# functest testcase run vping_ssh
  Executing command: 'python /home/opnfv/repos/functest/ci/run_tests.py -t vping_ssh'
  2016-06-30 11:50:31,861 - run_tests - INFO - Sourcing the OpenStack RC file...
  2016-06-30 11:50:31,865 - run_tests - INFO - ============================================
  2016-06-30 11:50:31,865 - run_tests - INFO - Running test case 'vping_ssh'...
  2016-06-30 11:50:31,865 - run_tests - INFO - ============================================
  2016-06-30 11:50:32,977 - vping_ssh - INFO - Creating image 'functest-vping' from '/home/opnfv/functest/data/cirros-0.3.5-x86_64-disk.img'...
  2016-06-30 11:50:45,470 - vping_ssh - INFO - Creating neutron network vping-net...
  2016-06-30 11:50:47,645 - vping_ssh - INFO - Creating security group  'vPing-sg'...
  2016-06-30 11:50:48,843 - vping_ssh - INFO - Using existing Flavor 'm1.small'...
  2016-06-30 11:50:48,927 - vping_ssh - INFO - vPing Start Time:'2016-06-30 11:50:48'
  2016-06-30 11:50:48,927 - vping_ssh - INFO - Creating instance 'opnfv-vping-1'...
  2016-06-30 11:51:34,664 - vping_ssh - INFO - Instance 'opnfv-vping-1' is ACTIVE.
  2016-06-30 11:51:34,818 - vping_ssh - INFO - Adding 'opnfv-vping-1' to security group 'vPing-sg'...
  2016-06-30 11:51:35,209 - vping_ssh - INFO - Creating instance 'opnfv-vping-2'...
  2016-06-30 11:52:01,439 - vping_ssh - INFO - Instance 'opnfv-vping-2' is ACTIVE.
  2016-06-30 11:52:01,439 - vping_ssh - INFO - Adding 'opnfv-vping-2' to security group 'vPing-sg'...
  2016-06-30 11:52:01,754 - vping_ssh - INFO - Creating floating IP for VM 'opnfv-vping-2'...
  2016-06-30 11:52:01,969 - vping_ssh - INFO - Floating IP created: '10.17.94.140'
  2016-06-30 11:52:01,969 - vping_ssh - INFO - Associating floating ip: '10.17.94.140' to VM 'opnfv-vping-2'
  2016-06-30 11:52:02,792 - vping_ssh - INFO - Trying to establish SSH connection to 10.17.94.140...
  2016-06-30 11:52:19,915 - vping_ssh - INFO - Waiting for ping...
  2016-06-30 11:52:21,108 - vping_ssh - INFO - vPing detected!
  2016-06-30 11:52:21,108 - vping_ssh - INFO - vPing duration:'92.2' s.
  2016-06-30 11:52:21,109 - vping_ssh - INFO - vPing OK
  2016-06-30 11:52:21,153 - clean_openstack - INFO - +++++++++++++++++++++++++++++++
  2016-06-30 11:52:21,153 - clean_openstack - INFO - Cleaning OpenStack resources...
  2016-06-30 11:52:21,153 - clean_openstack - INFO - +++++++++++++++++++++++++++++++
  Version 1 is deprecated, use alternative version 2 instead.
  :
  :
  etc.

To list the test cases which are part of a specific Test Tier, the 'get-tests'
command is used with 'functest tier'::

  root@22e436918db0:~/repos/functest/ci# functest tier get-tests healthcheck
  Test cases in tier 'healthcheck':
   ['connection_check', 'api_check', 'snaps_health_check']


Please note that for some scenarios some test cases might not be launched.
For example, the last example displayed only the 'odl' testcase for the given
environment. In this particular system the deployment does not support the 'ocl' SDN
Controller Test Case; for example.

**Important** If you use the command 'functest tier run <tier_name>', then the
Functest CLI utility will call **all valid Test Cases**, which belong to the
specified Test Tier, as relevant to scenarios deployed to the SUT environment.
Thus, the Functest CLI utility calculates automatically which tests can be
executed and which cannot, given the environment variable **DEPLOY_SCENARIO**,
which is passed in to the Functest docker container.

Currently, the Functest CLI command 'functest testcase run <testcase_name>', supports
two possibilities::

 *  Run a single Test Case, specified by a valid choice of <testcase_name>
 *  Run ALL test Test Cases (for all Tiers) by specifying <testcase_name> = 'all'

Functest includes a cleaning mechanism in order to remove all the OpenStack
resources except those present before running any test. The script
*$REPOS_DIR/functest/functest/utils/openstack_snapshot.py* is called once when setting up
the Functest environment (i.e. CLI command 'functest env prepare') to snapshot
all the OpenStack resources (images, networks, volumes, security groups, tenants,
users) so that an eventual cleanup does not remove any of these defaults.

It is also called before running a test except if it is disabled by configuration
in the testcases.yaml file (clean_flag=false). This flag has been added as some
upstream tests already include their own cleaning mechanism (e.g. Rally).

The script **openstack_clean.py** which is located in
*$REPOS_DIR/functest/functest/utils/* is normally called after a test execution. It is
in charge of cleaning the OpenStack resources that are not specified in the
defaults file generated previously which is stored in
*/home/opnfv/functest/conf/openstack_snapshot.yaml* in the Functest docker container.

It is important to mention that if there are new OpenStack resources created
manually after the snapshot done before running the tests, they will be removed,
unless you use the special method of invoking the test case with specific
suppression of clean up. (See the `Troubleshooting`_ section).

The reason to include this cleanup meachanism in Functest is because some
test suites create a lot of resources (users, tenants, networks, volumes etc.)
that are not always properly cleaned, so this function has been set to keep the
system as clean as it was before a full Functest execution.

Although the Functest CLI provides an easy way to run any test, it is possible to
do a direct call to the desired test script. For example:

    python $REPOS_DIR/functest/functest/opnfv_tests/openstack/vping/vping_ssh.py


Automated testing
-----------------

As mentioned previously, the Functest Docker container preparation as well as
invocation of Test Cases can be called within the container from the Jenkins CI
system. There are 3 jobs that automate the whole process. The first job runs all
the tests referenced in the daily loop (i.e. that must been run daily), the second
job runs the tests referenced in the weekly loop (usually long duration tests run
once a week maximum) and the third job allows testing test suite by test suite specifying
the test suite name. The user may also use either of these Jenkins jobs to execute
the desired test suites.

One of the most challenging task in the Danube release consists
in dealing with lots of scenarios and installers. Thus, when the tests are
automatically started from CI, a basic algorithm has been created in order to
detect whether a given test is runnable or not on the given scenario.
Some Functest test suites cannot be systematically run (e.g. ODL suite can not
be run on an ONOS scenario). The daily/weekly notion has been introduces in
Colorado in order to save CI time and avoid running systematically
long duration tests. It was not used in Colorado due to CI resource shortage.
The mechanism remains however as part of the CI evolution.

CI provides some useful information passed to the container as environment
variables:

 * Installer (apex|compass|fuel|joid), stored in INSTALLER_TYPE
 * Installer IP of the engine or VM running the actual deployment, stored in INSTALLER_IP
 * The scenario [controller]-[feature]-[mode], stored in DEPLOY_SCENARIO with

   * controller = (odl|ocl|nosdn|onos)
   * feature = (ovs(dpdk)|kvm|sfc|bgpvpn|multisites|netready|ovs_dpdk_bar)
   * mode = (ha|noha)

The constraints per test case are defined in the Functest configuration file
*/home/opnfv/repos/functest/functest/ci/testcases.yaml*::

 tiers:
   -
        name: smoke
        order: 1
        ci_loop: '(daily)|(weekly)'
        description : >-
            Set of basic Functional tests to validate the OpenStack deployment.
        testcases:
            -
                name: vping_ssh
                criteria: 'status == "PASS"'
                blocking: true
                description: >-
                    This test case verifies: 1) SSH to an instance using floating
                    IPs over the public network. 2) Connectivity between 2 instances
                    over a private network.
                dependencies:
                    installer: ''
                    scenario: '^((?!bgpvpn|odl_l3).)*$'
                run:
                    module: 'functest.opnfv_tests.openstack.vping.vping_ssh'
                    class: 'VPingSSH'
        ....

We may distinguish 2 levels in the test case description:
  * Tier level
  * Test case level

At the tier level, we define the following parameters:

 * ci_loop: indicate if in automated mode, the test case must be run in dail and/or weekly jobs
 * description: a high level view of the test case

For a given test case we defined:
  * the name of the test case
  * the criteria (experimental): a criteria used to declare the test case as PASS or FAIL
  * blocking: if set to true, if the test is failed, the execution of the following tests is canceled
  * clean_flag: shall the functect internal mechanism be invoked after the test
  * the description of the test case
  * the dependencies: a combination of 2 regex on the scenario and the installer name
  * run: In Danube we introduced the notion of abstract class in order to harmonize the way to run internal, feature or vnf tests

For further details on abstraction classes, see developper guide.

Additional parameters have been added in the desription in the Database.
The target is to use the configuration stored in the Database and consider the
local file as backup if the Database is not reachable.
The additional fields related to a test case are:
  * trust: we introduced this notion to put in place a mechanism of scenario promotion.
  * Version: it indicates since which version you can run this test
  * domains: the main domain covered by the test suite
  * tags: a list of tags related to the test suite

The order of execution is the one defined in the file if all test cases are selected.

In CI daily job the tests are executed in the following order:

  1) healthcheck (blocking)
  2) smoke: both vPings are blocking
  3) Feature project tests cases

In CI weekly job we add 2 tiers:

  4) VNFs (vIMS)
  5) Components (Rally and Tempest long duration suites)

As explained before, at the end of an automated execution, the OpenStack resources
might be eventually removed.
Please note that a system snapshot is taken before any test case execution.

This testcase.yaml file is used for CI, for the CLI and for the automatic reporting.