diff options
Diffstat (limited to 'docs/testing/user')
| -rw-r--r-- | docs/testing/user/configguide/configguide.rst | 17 | ||||
| -rw-r--r-- | docs/testing/user/configguide/index.rst | 44 | ||||
| -rw-r--r-- | docs/testing/user/userguide/index.rst | 211 | ||||
| -rw-r--r-- | docs/testing/user/userguide/introduction.rst | 131 | ||||
| -rw-r--r-- | docs/testing/user/userguide/runfunctest.rst | 378 | ||||
| -rw-r--r-- | docs/testing/user/userguide/troubleshooting.rst | 54 |
6 files changed, 487 insertions, 348 deletions
diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index 56a414d36..e3485be40 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -1,5 +1,5 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 +.. SPDX-License-Identifier: CC-BY-4.0 Pulling the Docker image @@ -16,8 +16,8 @@ created automatically by the CI mechanisms, and you must ensure you pull an image with the **correct tag** to match the OPNFV software release installed in your environment. All available tagged images can be seen from location [FunctestDockerTags_]. For example, when running -on the first official release of the OPNFV Colorado system platform, -tag "colorado.1.0" is needed. Pulling other tags might cause some +on the first official release of the OPNFV Danube system platform, +tag "danube.1.0" is needed. Pulling other tags might cause some problems while running the tests. Docker images pulled without a tag specifier bear the implicitly assigned label "latest". If you need to specifically pull the latest @@ -201,7 +201,7 @@ when performing manual test scenarios:: tests are activated under Jenkins CI control. It enables the correlation of test results, which - are independently pushed to the results datbase + are independently pushed to the results database from different Jenkins jobs. DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. <DebugTraceValue> = "true" or "false" @@ -405,9 +405,9 @@ follows:: | `-- setup.py |-- core | |-- __init__.py - | |-- feature_base.py + | |-- feature.py | |-- pytest_suite_runner.py - | |-- testcase_base.py + | |-- testcase.py | |-- vnf_base.py |-- opnfv_tests | |-- __init__.py @@ -444,8 +444,7 @@ We may distinguish several directories, the first level has 4 directories: * **docker**: This directory includes the needed files and tools to build the Funtest Docker image. * **docs**: This directory includes documentation: Release Notes, - User Guide, Configuration Guide and Developer Guide. Test results - are also located in a sub--directory called 'results'. + User Guide, Configuration Guide and Developer Guide. * **functest**: This directory contains all the code needed to run functest internal cases and OPNFV onboarded feature or VNF test cases. @@ -592,7 +591,7 @@ prompt:: - here = "false" NOTE: In Jenkins CI runs, an additional item "BUILD TAG" - would also be listed. The valaue is set by Jenkins CI. + would also be listed. The value is set by Jenkins CI. Finally, the **functest** CLI has a **--help** options: diff --git a/docs/testing/user/configguide/index.rst b/docs/testing/user/configguide/index.rst index f12739e31..737f0ba7c 100644 --- a/docs/testing/user/configguide/index.rst +++ b/docs/testing/user/configguide/index.rst @@ -27,8 +27,8 @@ Version history Introduction ============ This document describes how to install and configure Functest in OPNFV. -The Functest CLI is utilized during the Functest environment preparation -step. The given example commands should work in both virtual and bare +The Functest CLI is used during the Functest environment preparation +phase. The given example commands should work in both virtual and bare metal cases alike. High level architecture @@ -52,21 +52,12 @@ follows:: | | | - Rally | | | | | | | | | | - Robot | | | | | | | | | | - TestON | | | | | | | + | | | - RefStack | | | | | | | | | | | |-------------------------+ | | | | | Testcases | | | | | | | | | | - VIM | | | | | | | - | | | -- healthcheck | | | | | | | - | | | -- vPing_ssh | | | | | | | - | | | -- vPing_userdata | | | | | | | - | | | -- SNAPS_cases | | | | | | | - | | | -- Tempest_smoke | | | | | | | - | | | -- Rally_sanity | | | | | | | - | | | -- Tempest_full | | | | | | | - | | | -- Rally_full | | | | | | | | | | | | | | | | | | | | - SDN Controller | | | | | | | - | | | -- odl | | | | | | | - | | | -- onos | | | | | | | | | | | | | | | | | | | | - Features | | | | | | | | | | | | | | | | | @@ -109,16 +100,16 @@ on any platform on any Operating System. The automated mechanisms inside the Functest Docker container will: * Retrieve OpenStack credentials - * Prepare the environment according to the SUT + * Prepare the environment according to the System Under Test (SUT) * Perform the appropriate functional tests * Push the test results into the OPNFV test result database This Docker image can be integrated into CI or deployed independently. -Please note that the Functest Docker container has been designed for -OPNFV, however, it would be possible to adapt it to any VIM + controller -environment, since most of the test cases are integrated from upstream -communities. +Please note that the Functest Docker container has been designed for OPNFV, +however, it would be possible to adapt it to any OpenStack based VIM + +controller environment, since most of the test cases are integrated from +upstream communities. The functional test cases are described in the Functest User Guide `[2]`_ @@ -127,8 +118,7 @@ Prerequisites ============= The OPNFV deployment is out of the scope of this document but it can be found in http://docs.opnfv.org. -The OPNFV platform is considered as the System Under Test (SUT) in this -document. +The OPNFV platform is considered as the SUT in this document. Several prerequisites are needed for Functest: @@ -149,7 +139,7 @@ requirements. Normally it is the same server from where the OPNFV deployment has been triggered previously. NOTE: If your Jumphost is operating behind a company http proxy and/or -Firewall, please consult first the section `Proxy Support`_, towards +firewall, please consult first the section `Proxy Support`_, towards the end of this document. The section details some tips/tricks which *may* be of help in a proxified environment. @@ -158,7 +148,7 @@ Docker installation Docker installation and configuration is only needed to be done once through the life cycle of Jumphost. -If your Jumphost is based on Ubuntu, RHEL or CentOS linux, please +If your Jumphost is based on Ubuntu, SUSE, RHEL or CentOS linux, please consult the references below for more detailed instructions. The commands below are offered as a short reference. @@ -182,7 +172,7 @@ Add your user to docker group to be able to run commands without sudo:: sudo usermod -aG docker <your_user> -A reconnect is needed. There are 2 ways for this: +A reconnection is needed. There are 2 ways for this: #. Re-login to your account #. su - <username> @@ -190,10 +180,12 @@ References - Installing Docker Engine on different Linux Operating Systems: * Ubuntu_ * RHEL_ * CentOS_ + * SUSE_ .. _Ubuntu: https://docs.docker.com/engine/installation/linux/ubuntulinux/ .. _RHEL: https://docs.docker.com/engine/installation/linux/rhel/ .. _CentOS: https://docs.docker.com/engine/installation/linux/centos/ +.. _SUSE: https://docs.docker.com/engine/installation/linux/suse/ Public/External network on SUT ------------------------------ @@ -286,11 +278,11 @@ References .. _`[4]`: http://artifacts.opnfv.org/functest/danube/docs/configguide/index.html -OPNFV main site: opnfvmain_. +`OPNFV main site`_ -OPNFV functional test page: opnfvfunctest_. +`Functest page`_ IRC support channel: #opnfv-functest -.. _opnfvmain: http://www.opnfv.org -.. _opnfvfunctest: https://wiki.opnfv.org/functest +.. _`OPNFV main site`: http://www.opnfv.org +.. _`Functest page`: https://wiki.opnfv.org/functest diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index 9436de2b9..0459ce5f1 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -1,5 +1,5 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 +.. SPDX-License-Identifier: CC-BY-4.0 ========================= OPNFV FUNCTEST user guide @@ -28,7 +28,7 @@ Introduction The goal of this document is to describe the OPNFV Functest test cases and to provide a procedure to execute them. In the OPNFV Danube system release, -a Functest CLI utility is introduced for easier execution of test procedures. +a Functest CLI utility is introduced for an easier execution of test procedures. **IMPORTANT**: It is assumed here that the Functest Docker container is already properly deployed and that all instructions described in this guide are to be @@ -41,21 +41,37 @@ The different test cases are described in the remaining sections of this documen VIM (Virtualized Infrastructure Manager) ---------------------------------------- -Healthcheck -^^^^^^^^^^^ -In Colorado release a new Tier 'healthcheck' with one testcase 'healthcheck' -was introduced. The healthcheck testcase verifies that some basic IP connectivity -and essential operations of OpenStack functionality over the command line are -working correctly. +Healthcheck tests +^^^^^^^^^^^^^^^^^ +In Danube, healthcheck tests have been refactored and rely on SNAPS, a +OPNFV middleware project. + +SNAPS stands for "SDN/NFV Application development Platform and Stack". +SNAPS is an object-oriented OpenStack library packaged with tests that exercise +OpenStack. +More information on SNAPS can be found in `[13]`_ + +Three tests are declared as healthcheck tests and can be used for gating by the +installer, they cover functionally the tests previously done by healthcheck +test case. + +The tests are: -In particular, the following verifications are performed: - * DHCP agent functionality for IP address allocation - * Openstack Authentication management functionality via the Keystone API - * OpenStack Image management functionality via the Glance API - * OpenStack Block Storage management functionality via the Cinder API - * OpenStack Networking management functionality via the Neutron API - * Openstack Compute management functionality via the NOVA API + * *connection_check* + * *api_check* + * *snaps_health_check* + +Connection_check consists in 9 test cases (test duration < 5s) checking the +connectivity with Glance, Keystone, Neutron, Nova and the external network. + +Api_check verifies the retrieval of OpenStack clients: Keystone, Glance, +Neutron and Nova and may perform some simple queries. When the config value of +snaps.use_keystone is True, functest must have access to the cloud's private +network. This suite consists in 49 tests (test duration < 2 minutes). + +snaps_health_check creates instance, allocate floating IP, connect to the VM. +This test replaced the previous Colorado healthcheck test. Self-obviously, successful completion of the 'healthcheck' testcase is a necessary pre-requisite for the execution of all other test Tiers. @@ -229,30 +245,9 @@ The Rally testcases are distributed accross two Tiers: NOTE: Test case 'rally_sanity' executes a limited number of Rally smoke test cases. Test case 'rally_full' executes the full defined set of Rally tests. -SNAPS ------ - -SNAPS stands for "SNA/NFV Application development Platform and Stack". -This project seeks to develop baseline OpenStack NFV installations. It has been -developed by Steven Pisarski and provided an object oriented library to perform -functional and performance tests. It has been declined in several test suites in -Functest, 2 are part of healthcheck tier, one belongs to smoke tier. - -connection check -^^^^^^^^^^^^^^^^ -Connection_check consists in 9 test cases (test duration < 5s) checking the -connectivity with Glance, Keystone, Neutron, Nova and the external network. - -api_check -^^^^^^^^^ -This test case verifies the retrieval of OpenStack clients: Keystone, Glance, -Neutron and Nova and may perform some simple queries. When the config value of -snaps.use_keystone is True, functest must have access to the cloud's private -network. -This suite consists in 49 tests (test duration< 2 minutes) - snaps_smoke -^^^^^^^^^^^ +------------ + This test case contains tests that setup and destroy environments with VMs with and without Floating IPs with a newly created user and project. Set the config value snaps.use_floating_ips (True|False) to toggle this functionality. When @@ -260,16 +255,15 @@ the config value of snaps.use_keystone is True, functest must have access the cloud's private network. This suite consists in 38 tests (test duration < 10 minutes) -More information on SNAPS can be found in `[13]`_ - SDN Controllers --------------- -There are currently 2 available controllers: +There are currently 3 available controllers: * OpenDaylight (ODL) * ONOS + * OpenContrail (OCL) OpenDaylight ^^^^^^^^^^^^ @@ -366,60 +360,30 @@ The test cases are described as follows: Features -------- -Please refer to the dedicated feature user guides for details: - - * bgpvpn: http://artifacts.opnfv.org/sdnvpn/danube/docs/userguide/index.html - * copper: http://artifacts.opnfv.org/copper/danube/docs/userguide/index.html - * doctor: http://artifacts.opnfv.org/doctor/danube/userguide/index.html - * domino: http://artifacts.opnfv.org/domino/docs/userguide-single/index.html - * multisites: http://artifacts.opnfv.org/multisite/docs/userguide/index.html - * onos-sfc: http://artifacts.opnfv.org/onosfw/danube/userguide/index.html - * odl-sfc: http://artifacts.opnfv.org/sfc/danube/userguide/index.html - * promise: http://artifacts.opnfv.org/danube/colorado/docs/userguide/index.html - * security_scan: http://artifacts.opnfv.org/security_scan/colorado/docs/userguide/index.html - * TODO +In Danube, functest supports the integration of: -security_scan -^^^^^^^^^^^^^ + * barometer + * bgpvpn + * doctor + * domino + * fds + * multisite + * netready + * odl-sfc + * promise + * security_scan -Security Scanning, is a project to insure security compliance and vulnerability -checks, as part of an automated CI / CD platform delivery process. - -The project makes use of the existing SCAP format `[6]`_ to perform deep -scanning of NFVI nodes, to insure they are hardened and free of known CVE -reported vulnerabilities. - -The SCAP content itself, is then consumed and run using an upstream opensource tool -known as OpenSCAP `[7]`_. - -The OPNFV Security Group have developed the code that will called by the OPNFV Jenkins -build platform, to perform a complete scan. Resulting reports are then copied to the -OPNFV functest dashboard. - -The current work flow is as follows: - - * Jenkins Build Initiated - * security_scan.py script is called, and a config file is passed to the script as - an argument. - * The IP addresses of each NFVi node (compute / control) are gathered - * A scan profile is matched to the node type. - * The OpenSCAP application is remotely installed onto each target node gathered - on step 3, using upstream packaging (rpm and .deb). - * A scan is made against each node gathered within step 3. - * HTML Reports are downloaded for rendering on a dashboard. - * If the config file value 'clean' is set to 'True' then the application installed in - step 5 is removed, and all reports created at step 6 are deleted. - -Security scan is supported by Apex, TODO.... +Note: copper is not supported in Danube. +Please refer to the dedicated feature user guides for details. VNF --- -vIMS -^^^^ +cloudify_ims +^^^^^^^^^^^^ The IP Multimedia Subsystem or IP Multimedia Core Network Subsystem (IMS) is an architectural framework for delivering IP multimedia services. @@ -437,10 +401,14 @@ The goal of this test suite consists of: The Clearwater architecture is described as follows: -.. figure:: ../images/clearwater-architecture.png +.. figure:: ../../../images/clearwater-architecture.png :align: center :alt: vIMS architecture +orchestra_ims +^^^^^^^^^^^^^ +Orchestra test case deals with the deployment of OpenIMS with OpenBaton +orchestrator. parser ^^^^^^ @@ -448,6 +416,28 @@ parser See parser user guide for details: `[12]`_ +vyos-vrouter +^^^^^^^^^^^^ + +This test case deals with the deployment and the test of vyos vrouter with +Cloudify orchestrator. The test case can do testing for interchangeability of +BGP Protocol using vyos. + +The Workflow is as follows: + * Deploy + Deploy VNF Testing topology by Cloudify using blueprint. + * Configuration + Setting configuration to Target VNF and reference VNF using ssh + * Run + Execution of test command for test item written YAML format file. + Check VNF status and behavior. + * Reporting + Output of report based on result using JSON format. + +The vyos-vrouter architecture is described in `[14]`_ + + + .. include:: ./runfunctest.rst @@ -479,20 +469,27 @@ at the end of the job and can be described as follow:: +=========================+===============+============+===============+===========================================================================+ | TEST CASE | TIER | DURATION | RESULT | URL | +=========================+===============+============+===============+===========================================================================+ - | healthcheck | healthcheck | 03:07 | PASS | | + | connection_check | healthcheck | 00:02 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb62b34079ac000a42e3fe | + +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ + | api_check | healthcheck | 01:15 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb62fe4079ac000a42e3ff | + +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ + | snaps_health_check | healthcheck | 00:50 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb63314079ac000a42e400 | +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ - | vping_ssh | smoke | 00:56 | PASS | http://testresults.opnfv.org/test/api/v1/results/57ac13d79377c54b278bd4c1 | + | vping_ssh | smoke | 01:10 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb63654079ac000a42e401 | +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ - | vping_userdata | smoke | 00:41 | PASS | http://testresults.opnfv.org/test/api/v1/results/57ac14019377c54b278bd4c2 | + | vping_userdata | smoke | 00:59 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb63a14079ac000a42e403 | +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ - | tempest_smoke_serial | smoke | 16:05 | FAIL | http://testresults.opnfv.org/test/api/v1/results/57ac17ca9377c54b278bd4c3 | + | tempest_smoke_serial | smoke | 12:57 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb66bd4079ac000a42e408 | +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ - | rally_sanity | smoke | 12:19 | PASS | http://testresults.opnfv.org/test/api/v1/results/57ac1aad9377c54b278bd4cd | + | rally_sanity | smoke | 10:22 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb692b4079ac000a42e40a | +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ - | odl | sdn_suites | 00:24 | PASS | http://testresults.opnfv.org/test/api/v1/results/57ac1ad09377c54b278bd4ce | + | refstack_defcore | smoke | 12:28 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb6c184079ac000a42e40c | +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ - | promise | features | 00:41 | PASS | http://testresults.opnfv.org/test/api/v1/results/57ac1ae59377c54b278bd4cf | + | snaps_smoke | smoke | 12:04 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb6eec4079ac000a42e40e | +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ + | domino | features | 00:29 | PASS | http://testresults.opnfv.org/test/api/v1/results/58cb6f044079ac000a42e40f | + +-------------------------+---------------+------------+---------------+---------------------------------------------------------------------------+ + Results are automatically pushed to the test results database, some additional result files are pushed to OPNFV artifact web sites. @@ -504,17 +501,10 @@ portal is also automatically updated. This portal provides information on: * Tempest: Tempest test case including reported errors per scenario and installer * vIMS: vIMS details per scenario and installer -.. figure:: ../images/functest-reporting-status.png +.. figure:: ../../../images/functest-reporting-status.png :align: center :alt: Functest reporting portal Fuel status page - -Test Dashboard -============== - -Based on results collected in CI, a test dashboard is dynamically generated. - - .. include:: ./troubleshooting.rst @@ -528,21 +518,20 @@ References .. _`[5]`: https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/blob/master/openstack-blueprint.yaml .. _`[6]`: https://scap.nist.gov/ .. _`[7]`: https://github.com/OpenSCAP/openscap -.. _`[9]`: https://git.opnfv.org/cgit/functest/tree/testcases/VIM/OpenStack/CI/libraries/os_defaults.yaml .. _`[11]`: http://robotframework.org/ .. _`[12]`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html -.. _`[13]`: TODO URL doc SNAPS +.. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO +.. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter -OPNFV main site: opnfvmain_. +`OPNFV main site`_ -OPNFV functional test page: opnfvfunctest_. +`Functest page`_ IRC support chan: #opnfv-testperf -.. _opnfvmain: http://www.opnfv.org -.. _opnfvfunctest: https://wiki.opnfv.org/opnfv_functional_testing +.. _`OPNFV main site`: http://www.opnfv.org +.. _`Functest page`: https://wiki.opnfv.org/functest .. _`OpenRC`: http://docs.openstack.org/user-guide/common/cli_set_environment_variables_using_openstack_rc.html .. _`Rally installation procedure`: https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html -.. _`config_test.py` : https://git.opnfv.org/cgit/functest/tree/testcases/config_functest.py -.. _`config_functest.yaml` : https://git.opnfv.org/cgit/functest/tree/testcases/config_functest.yaml -.. _`Functest reporting`: http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-fuel.html +.. _`config_functest.yaml` : https://git.opnfv.org/cgit/functest/tree/functest/ci/config_functest.yaml +.. _`Functest reporting`: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-fuel.html diff --git a/docs/testing/user/userguide/introduction.rst b/docs/testing/user/userguide/introduction.rst index 4dfe79375..d1b7d23e0 100644 --- a/docs/testing/user/userguide/introduction.rst +++ b/docs/testing/user/userguide/introduction.rst @@ -22,15 +22,16 @@ validate the scenario for the release. +-------------+---------------+----------------+----------------------------------+ | Domain | Tier | Test case | Comments | +=============+===============+================+==================================+ -| VIM | healthcheck | healthcheck | Verify basic operation in VIM | -| | +----------------+----------------------------------+ -| | | connection | Check OpenStack connectivity | +| VIM | healthcheck | connection | Check OpenStack connectivity | | | | _check | through SNAPS framework | | | +----------------+----------------------------------+ | | | api_check | Check OpenStack API through | | | | | SNAPS framework | +| | +----------------+----------------------------------+ +| | | snaps_health | basic instance creation, check | +| | | \_check | DHCP | | +---------------+----------------+----------------------------------+ -| | smoke | vPing_SSH | NFV "Hello World" using an SSH | +| | smoke | vping_ssh | NFV "Hello World" using an SSH | | | | | connection to a destination VM | | | | | over a created floating IP | | | | | address on the SUT Public / | @@ -42,7 +43,7 @@ validate the scenario for the release. | | | | VM on a specified IP address over| | | | | the SUT Private Tenant network. | | | +----------------+----------------------------------+ -| | | vPing_userdata | Uses Ping with given userdata | +| | | vping_userdata | Uses Ping with given userdata | | | | | to test intra-VM connectivity | | | | | over the SUT Private Tenant | | | | | network. The correct operation | @@ -60,6 +61,9 @@ validate the scenario for the release. | | +----------------+----------------------------------+ | | | snaps_smoke | Run a subset of the OpenStack | | | | | Rally Test Suite in smoke mode | +| | +----------------+----------------------------------+ +| | | refstack | Reference RefStack suite | +| | | \_defcore | tempest selection for NFV | | +---------------+----------------+----------------------------------+ | | components | tempest_full | Generate and run a full set of | | | | \_parallel | the OpenStack Tempest Test Suite.| @@ -71,6 +75,9 @@ validate the scenario for the release. | | | rally_full | Run the OpenStack testing tool | | | | | benchmarking OpenStack modules | | | | | See the Rally documents `[3]`_. | +| | +----------------+----------------------------------+ +| | | tempest_custom | Allow to run a customized list | +| | | | of Tempest cases | +-------------+---------------+----------------+----------------------------------+ | Controllers | smoke | odl | Opendaylight Test suite | | | | | Limited test suite to check the | @@ -83,17 +90,27 @@ validate the scenario for the release. | | | | functions. | | | | | See `ONOSFW User Guide`_ for | | | | | details. | +| | +----------------+----------------------------------+ +| | | odl_netvirt | Test Suite for the OpenDaylight | +| | | | SDN Controller when the NetVirt | +| | | | features are installed. It | +| | | | integrates some test suites from | +| | | | upstream using Robot as the test | +| | | | framework | +| | +----------------+----------------------------------+ +| | | fds | Test Suite for the OpenDaylight | +| | | | SDN Controller when the GBP | +| | | | features are installed. It | +| | | | integrates some test suites from | +| | | | upstream using Robot as the test | +| | | | framework | +-------------+---------------+----------------+----------------------------------+ -| Features | features | promise | Resource reservation and | -| | | | management project to identify | -| | | | NFV related requirements and | -| | | | realize resource reservation for | -| | | | future usage by capacity | -| | | | management of resource pools | -| | | | regarding compute, network and | -| | | | storage. | -| | | | See `Promise User Guide`_ for | -| | | | details. | +| Features | features | bgpvpn | Implementation of the OpenStack | +| | | | bgpvpn API from the SDNVPN | +| | | | feature project. It allows for | +| | | | the creation of BGP VPNs. | +| | | | See `SDNVPN User Guide`_ for | +| | | | details | | | +----------------+----------------------------------+ | | | doctor | Doctor platform, as of Colorado | | | | | release, provides the three | @@ -106,25 +123,6 @@ validate the scenario for the release. | | | | See `Doctor User Guide`_ for | | | | | details | | | +----------------+----------------------------------+ -| | | bgpvpn | Implementation of the OpenStack | -| | | | bgpvpn API from the SDNVPN | -| | | | feature project. It allows for | -| | | | the creation of BGP VPNs. | -| | | | See `SDNVPN User Guide`_ for | -| | | | details | -| | +----------------+----------------------------------+ -| | | security_scan | Implementation of a simple | -| | | | security scan. (Currently | -| | | | available only for the Apex | -| | | | installer environment) | -| | +----------------+----------------------------------+ -| | | onos-sfc | SFC testing for onos scenarios | -| | | | See `ONOSFW User Guide`_ for | -| | | | details | -| | +----------------+----------------------------------+ -| | | odl-sfc | SFC testing for odl scenarios | -| | | | See `SFC User Guide`_ for details| -| | +----------------+----------------------------------+ | | | domino | Domino provides TOSCA template | | | | | distribution service for network | | | | | service and VNF descriptors | @@ -134,21 +132,41 @@ validate the scenario for the release. | | | | See `Domino User Guide`_ for | | | | | details | | | +----------------+----------------------------------+ -| | | copper | Copper develops OPNFV platform | -| | | | support for policy management, | -| | | | using open source projects such | -| | | | as OpenStack Congress, focused | -| | | | on helping ensure that virtual | -| | | | infrastructure and the apps that | -| | | | execute on it comply with the | -| | | | configuration policy intent of | -| | | | service providers, developers, | -| | | | and end users. See more detail | -| | | | in the `Copper User Guide`_. | -| | +----------------+----------------------------------+ -| | | multisites | Multisites | +| | | multisite | Multisite | | | | | See `Multisite User Guide`_ for | | | | | details | +| | +----------------+----------------------------------+ +| | | netready | Testing from netready project | +| | | | ping using gluon | +| | +----------------+----------------------------------+ +| | | odl-sfc | SFC testing for odl scenarios | +| | | | See `SFC User Guide`_ for details| +| | +----------------+----------------------------------+ +| | | parser | Parser is an integration project | +| | | | which aims to provide | +| | | | placement/deployment templates | +| | | | translation for OPNFV platform, | +| | | | including TOSCA -> HOT, POLICY ->| +| | | | TOSCA and YANG -> TOSCA. it | +| | | | deals with a fake vRNC. | +| | | | See `Parser User Guide`_ for | +| | | | details | +| | +----------------+----------------------------------+ +| | | promise | Resource reservation and | +| | | | management project to identify | +| | | | NFV related requirements and | +| | | | realize resource reservation for | +| | | | future usage by capacity | +| | | | management of resource pools | +| | | | regarding compute, network and | +| | | | storage. | +| | | | See `Promise User Guide`_ for | +| | | | details. | +| | +----------------+----------------------------------+ +| | | security_scan | Implementation of a simple | +| | | | security scan. (Currently | +| | | | available only for the Apex | +| | | | installer environment) | +-------------+---------------+----------------+----------------------------------+ | VNF | vnf | cloudify_ims | Example of a real VNF deployment | | | | | to show the NFV capabilities of | @@ -158,18 +176,10 @@ validate the scenario for the release. | | | | It provides a fully functional | | | | | VoIP System | | | +----------------+----------------------------------+ -| | | opera_ims | vIMS deployment using openBaton | +| | | orchestra_ims | OpenIMS deployment using | +| | | | Openbaton orchestrator | | | +----------------+----------------------------------+ -| | | orchestra_ims | vIMS deployment using open-O | -+ +---------------+----------------+----------------------------------+ -| | | parser | Parser is an integration project | -| | | | which aims to provide | -| | | | placement/deployment templates | -| | | | translation for OPNFV platform, | -| | | | including TOSCA -> HOT, POLICY ->| -| | | | TOSCA and YANG -> TOSCA. | -| | | | See `Parser User Guide`_ for | -| | | | details | +| | | vyos_vrouter | vRouter testing | +-------------+---------------+----------------+----------------------------------+ @@ -185,7 +195,7 @@ In Danube, we merged smoke and sdn controller tiers in smoke tier. An overview of the Functest Structural Concept is depicted graphically below: -.. figure:: ../images/concepts_mapping_final.png +.. figure:: ../../../images/concepts_mapping_final.png :align: center :alt: Functest Concepts Structure @@ -203,8 +213,7 @@ NoSQL database. The goal is to populate the database with results from different sources and scenarios and to show them on a `Functest Dashboard`_. A screenshot of a live Functest Dashboard is shown below: -** TODO ** -.. figure:: ../images/FunctestDashboardDanube.png +.. figure:: ../../../images/FunctestDashboardDanube.png :align: center :alt: Functest Dashboard diff --git a/docs/testing/user/userguide/runfunctest.rst b/docs/testing/user/userguide/runfunctest.rst index e7ab84b26..8a3ba8c73 100644 --- a/docs/testing/user/userguide/runfunctest.rst +++ b/docs/testing/user/userguide/runfunctest.rst @@ -48,33 +48,34 @@ command:: root@22e436918db0:~/repos/functest/ci# functest tier list - 0. healthcheck: - ['healthcheck', 'connection_check', 'api_check',] + ['connection_check', 'api_check', 'snaps_health_check',] - 1. smoke: - ['vping_ssh', 'vping_userdata', 'tempest_smoke_serial', 'rally_sanity', 'snaps_smoke', 'odl'] + ['vping_ssh', 'vping_userdata', 'tempest_smoke_serial', 'odl', 'rally_sanity', 'refstack_defcore', 'snaps_smoke'] - 2. features: - ['doctor', 'security_scan'] + ['doctor', 'domino', 'promise', security_scan'] - 3. components: ['tempest_full_parallel', 'rally_full'] - 4. vnf: - ['cloudify_ims'] + ['cloudify_ims', 'orchestra_ims', 'vyos_vrouter'] and root@22e436918db0:~/repos/functest/ci# functest testcase list - healthcheck api_check connection_check + snaps_health_check vping_ssh vping_userdata snaps_smoke + refstack_defcore tempest_smoke_serial rally_sanity odl - doctor - security_scan tempest_full_parallel rally_full - cloudify_ims + vyos_vrouter + +Note the list of test cases depend on the installer and the scenario. More specific details on specific Tiers or Test Cases can be seen wih the 'show' command:: @@ -117,55 +118,260 @@ More specific details on specific Tiers or Test Cases can be seen wih the To execute a Test Tier or Test Case, the 'run' command is used:: root@22e436918db0:~/repos/functest/ci# functest tier run healthcheck - Executing command: 'python /home/opnfv/repos/functest/ci/run_tests.py -t healthcheck' - 2016-06-30 11:44:56,933 - run_tests - INFO - Sourcing the OpenStack RC file... - 2016-06-30 11:44:56,937 - run_tests - INFO - ############################################ - 2016-06-30 11:44:56,938 - run_tests - INFO - Running tier 'healthcheck' - 2016-06-30 11:44:56,938 - run_tests - INFO - ############################################ - 2016-06-30 11:44:56,938 - run_tests - INFO - ============================================ - 2016-06-30 11:44:56,938 - run_tests - INFO - Running test case 'healthcheck'... - 2016-06-30 11:44:56,938 - run_tests - INFO - ============================================ - 2016-06-30 11:44:56,953 - healtcheck - INFO - Testing Keystone API... - 2016-06-30 11:45:05,351 - healtcheck - INFO - ...Keystone OK! - 2016-06-30 11:45:05,354 - healtcheck - INFO - Testing Glance API... - 2016-06-30 11:45:29,746 - healtcheck - INFO - ... Glance OK! - 2016-06-30 11:45:29,749 - healtcheck - INFO - Testing Cinder API... - 2016-06-30 11:45:37,502 - healtcheck - INFO - ...Cinder OK! - 2016-06-30 11:45:37,505 - healtcheck - INFO - Testing Neutron API... - 2016-06-30 11:45:39,664 - healtcheck - INFO - External network found. ccd98ad6-d34a-4768-b03c-e28ecfcd51ca - 2016-06-30 11:45:39,667 - healtcheck - INFO - 1. Create Networks... - 2016-06-30 11:45:44,227 - healtcheck - INFO - 2. Create subnets... - 2016-06-30 11:45:46,805 - healtcheck - INFO - 4. Create Routers... - 2016-06-30 11:45:54,261 - healtcheck - INFO - ...Neutron OK! - 2016-06-30 11:45:54,264 - healtcheck - INFO - Testing Nova API... - 2016-06-30 11:47:12,272 - healtcheck - INFO - ...Nova OK! - 2016-06-30 11:47:12,274 - healtcheck - INFO - Checking if instances get an IP from DHCP... - : - : - 2016-06-30 11:48:17,832 - healtcheck - INFO - ...DHCP OK! - 2016-06-30 11:48:17,835 - healtcheck - INFO - Health check passed! - 2016-06-30 11:48:17,837 - clean_openstack - INFO - +++++++++++++++++++++++++++++++ - 2016-06-30 11:48:17,837 - clean_openstack - INFO - Cleaning OpenStack resources... - 2016-06-30 11:48:17,837 - clean_openstack - INFO - +++++++++++++++++++++++++++++++ - Version 1 is deprecated, use alternative version 2 instead. - WARNING:cinderclient.api_versions:Version 1 is deprecated, use alternative version 2 instead. - 2016-06-30 11:48:18,272 - clean_openstack - INFO - Removing Nova instances... - 2016-06-30 11:48:24,439 - clean_openstack - INFO - ------------------------------------------- - 2016-06-30 11:48:24,440 - clean_openstack - INFO - Removing Glance images... - 2016-06-30 11:48:35,853 - clean_openstack - INFO - ------------------------------------------- - 2016-06-30 11:48:35,854 - clean_openstack - INFO - Removing Cinder volumes... - 2016-06-30 11:48:37,344 - clean_openstack - INFO - ------------------------------------------- - 2016-06-30 11:48:37,344 - clean_openstack - INFO - Removing floating IPs... - 2016-06-30 11:48:37,467 - clean_openstack - INFO - ------------------------------------------- - 2016-06-30 11:48:37,467 - clean_openstack - INFO - Removing Neutron objects - 2016-06-30 11:48:53,633 - clean_openstack - INFO - ------------------------------------------- - 2016-06-30 11:48:53,633 - clean_openstack - INFO - Removing Security groups... - 2016-06-30 11:48:53,689 - clean_openstack - INFO - ------------------------------------------- - 2016-06-30 11:48:53,689 - clean_openstack - INFO - Removing Users... - 2016-06-30 11:48:54,444 - clean_openstack - INFO - ------------------------------------------- - 2016-06-30 11:48:54,444 - clean_openstack - INFO - Removing Tenants... - 2016-06-30 11:48:54,711 - clean_openstack - INFO - ------------------------------------------- - + 2017-03-21 13:34:21,400 - run_tests - INFO - ############################################ + 2017-03-21 13:34:21,400 - run_tests - INFO - Running tier 'healthcheck' + 2017-03-21 13:34:21,400 - run_tests - INFO - ############################################ + 2017-03-21 13:34:21,401 - run_tests - INFO - + + 2017-03-21 13:34:21,401 - run_tests - INFO - ============================================ + 2017-03-21 13:34:21,401 - run_tests - INFO - Running test case 'connection_check'... + 2017-03-21 13:34:21,401 - run_tests - INFO - ============================================ + test_glance_connect_fail (snaps.openstack.utils.tests.glance_utils_tests.GlanceSmokeTests) ... ok + test_glance_connect_success (snaps.openstack.utils.tests.glance_utils_tests.GlanceSmokeTests) ... ok + test_keystone_connect_fail (snaps.openstack.utils.tests.keystone_utils_tests.KeystoneSmokeTests) ... ok + test_keystone_connect_success (snaps.openstack.utils.tests.keystone_utils_tests.KeystoneSmokeTests) ... ok + test_neutron_connect_fail (snaps.openstack.utils.tests.neutron_utils_tests.NeutronSmokeTests) ... ok + test_neutron_connect_success (snaps.openstack.utils.tests.neutron_utils_tests.NeutronSmokeTests) ... ok + test_retrieve_ext_network_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronSmokeTests) ... ok + test_nova_connect_fail (snaps.openstack.utils.tests.nova_utils_tests.NovaSmokeTests) ... ok + test_nova_connect_success (snaps.openstack.utils.tests.nova_utils_tests.NovaSmokeTests) ... ok + + ---------------------------------------------------------------------- + Ran 9 tests in 3.768s + + OK + 2017-03-21 13:34:26,570 - functest.core.testcase_base - INFO - connection_check OK + 2017-03-21 13:34:26,918 - functest.core.testcase_base - INFO - The results were successfully pushed to DB + 2017-03-21 13:34:26,918 - run_tests - INFO - Test execution time: 00:05 + 2017-03-21 13:34:26,918 - run_tests - INFO - + + 2017-03-21 13:34:26,918 - run_tests - INFO - ============================================ + 2017-03-21 13:34:26,918 - run_tests - INFO - Running test case 'api_check'... + 2017-03-21 13:34:26,919 - run_tests - INFO - ============================================ + test_create_project_minimal (snaps.openstack.utils.tests.keystone_utils_tests.KeystoneUtilsTests) ... ok + test_create_user_minimal (snaps.openstack.utils.tests.keystone_utils_tests.KeystoneUtilsTests) ... ok + test_create_delete_user (snaps.openstack.tests.create_user_tests.CreateUserSuccessTests) ... ok + test_create_user (snaps.openstack.tests.create_user_tests.CreateUserSuccessTests) ... ok + test_create_user_2x (snaps.openstack.tests.create_user_tests.CreateUserSuccessTests) ... + 2017-03-21 13:34:32,684 - create_user - INFO - Found user with name - CreateUserSuccessTests-7e741e11-c9fd-489-name ok + test_create_delete_project (snaps.openstack.tests.create_project_tests.CreateProjectSuccessTests) ... ok + test_create_project (snaps.openstack.tests.create_project_tests.CreateProjectSuccessTests) ... ok + test_create_project_2x (snaps.openstack.tests.create_project_tests.CreateProjectSuccessTests) ... + 2017-03-21 13:34:35,922 - create_image - INFO - Found project with name - CreateProjectSuccessTests-b38e08ce-2862-48a-name ok + test_create_project_sec_grp_one_user (snaps.openstack.tests.create_project_tests.CreateProjectUserTests) ... + 2017-03-21 13:34:37,907 - OpenStackSecurityGroup - INFO - Creating security group CreateProjectUserTests-ab8801f6-dad8-4f9-name... + 2017-03-21 13:34:37,907 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-ab8801f6-dad8-4f9-name + 2017-03-21 13:34:38,376 - neutron_utils - INFO - Creating security group with name - CreateProjectUserTests-ab8801f6-dad8-4f9-name + 2017-03-21 13:34:38,716 - neutron_utils - INFO - Retrieving security group rules associate with the security group - CreateProjectUserTests-ab8801f6-dad8-4f9-name + 2017-03-21 13:34:38,762 - neutron_utils - INFO - Retrieving security group with ID - 821419cb-c54c-41b4-a61b-fb30e5dd2ec5 + 2017-03-21 13:34:38,886 - neutron_utils - INFO - Retrieving security group with ID - 821419cb-c54c-41b4-a61b-fb30e5dd2ec5 + 2017-03-21 13:34:39,000 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-ab8801f6-dad8-4f9-name + 2017-03-21 13:34:39,307 - neutron_utils - INFO - Deleting security group rule with ID - d85fafc0-9649-45c9-a00e-452f3d5c09a6 + 2017-03-21 13:34:39,531 - neutron_utils - INFO - Deleting security group rule with ID - 69d79c09-bc3b-4975-9353-5f43aca51237 + 2017-03-21 13:34:39,762 - neutron_utils - INFO - Deleting security group with name - CreateProjectUserTests-ab8801f6-dad8-4f9-name ok + test_create_project_sec_grp_two_users (snaps.openstack.tests.create_project_tests.CreateProjectUserTests) ... + 2017-03-21 13:34:43,511 - OpenStackSecurityGroup - INFO - Creating security group CreateProjectUserTests-4d9261a6-e008-44b-name... + 2017-03-21 13:34:43,511 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-4d9261a6-e008-44b-name + 2017-03-21 13:34:44,090 - neutron_utils - INFO - Creating security group with name - CreateProjectUserTests-4d9261a6-e008-44b-name + 2017-03-21 13:34:44,784 - neutron_utils - INFO - Retrieving security group rules associate with the security group - CreateProjectUserTests-4d9261a6-e008-44b-name + 2017-03-21 13:34:44,864 - neutron_utils - INFO - Retrieving security group with ID - 780193e4-9bd2-4f2e-a14d-b01abf74c832 + 2017-03-21 13:34:45,233 - neutron_utils - INFO - Retrieving security group with ID - 780193e4-9bd2-4f2e-a14d-b01abf74c832 + 2017-03-21 13:34:45,332 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-4d9261a6-e008-44b-name + 2017-03-21 13:34:45,779 - OpenStackSecurityGroup - INFO - Creating security group CreateProjectUserTests-4d9261a6-e008-44b-name... + 2017-03-21 13:34:45,779 - neutron_utils - INFO - Retrieving security group with name - CreateProjectUserTests-4d9261a6-e008-44b-name + 2017-03-21 13:34:46,112 - neutron_utils - INFO - Retrieving security group rules associate with the security group - CreateProjectUserTests-4d9261a6-e008-44b-name + 2017-03-21 13:34:46,184 - neutron_utils - INFO - Retrieving security group with ID - 780193e4-9bd2-4f2e-a14d-b01abf74c832 + 2017-03-21 13:34:46,296 - neutron_utils - INFO - Retrieving security group with ID - 780193e4-9bd2-4f2e-a14d-b01abf74c832 + 2017-03-21 13:34:46,387 - neutron_utils - INFO - Deleting security group rule with ID - 2320a573-ec56-47c5-a1ba-ec514d30114b + 2017-03-21 13:34:46,636 - neutron_utils - INFO - Deleting security group rule with ID - 6186282b-db37-4e47-becc-a3886079c069 + 2017-03-21 13:34:46,780 - neutron_utils - INFO - Deleting security group with name - CreateProjectUserTests-4d9261a6-e008-44b-name + 2017-03-21 13:34:47,006 - neutron_utils - INFO - Deleting security group rule with ID - 2320a573-ec56-47c5-a1ba-ec514d30114b + 2017-03-21 13:34:47,072 - OpenStackSecurityGroup - WARNING - Rule not found, cannot delete - Security group rule 2320a573-ec56-47c5-a1ba-ec514d30114b does not exist + Neutron server returns request_ids: ['req-d74eb2e2-b26f-4236-87dc-7255866141d9'] + 2017-03-21 13:34:47,072 - neutron_utils - INFO - Deleting security group rule with ID - 6186282b-db37-4e47-becc-a3886079c069 + 2017-03-21 13:34:47,118 - OpenStackSecurityGroup - WARNING - Rule not found, cannot delete - Security group rule 6186282b-db37-4e47-becc-a3886079c069 does not exist + Neutron server returns request_ids: ['req-8c0a5a24-be90-4844-a9ed-2a85cc6f59a5'] + 2017-03-21 13:34:47,118 - neutron_utils - INFO - Deleting security group with name - CreateProjectUserTests-4d9261a6-e008-44b-name + 2017-03-21 13:34:47,172 - OpenStackSecurityGroup - WARNING - Security Group not found, cannot delete - Security group 780193e4-9bd2-4f2e-a14d-b01abf74c832 does not exist + Neutron server returns request_ids: ['req-c6e1a6b5-43e0-4d46-bb68-c2e1672d4d21'] ok + test_create_image_minimal_file (snaps.openstack.utils.tests.glance_utils_tests.GlanceUtilsTests) ... ok + test_create_image_minimal_url (snaps.openstack.utils.tests.glance_utils_tests.GlanceUtilsTests) ... ok + test_create_network (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsNetworkTests) ... + 2017-03-21 13:35:22,275 - neutron_utils - INFO - Creating network with name NeutronUtilsNetworkTests-c06c20e0-d78f-4fa4-8401-099a7a6cab2e-pub-net + 2017-03-21 13:35:23,965 - neutron_utils - INFO - Deleting network with name NeutronUtilsNetworkTests-c06c20e0-d78f-4fa4-8401-099a7a6cab2e-pub-net ok + test_create_network_empty_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsNetworkTests) ... ok + test_create_network_null_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsNetworkTests) ... ok + test_create_subnet (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... + 2017-03-21 13:35:25,495 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-4f440a5f-54e3-4455-ab9b-39dfe06f6d21-pub-net + 2017-03-21 13:35:26,841 - neutron_utils - INFO - Creating subnet with name NeutronUtilsSubnetTests-4f440a5f-54e3-4455-ab9b-39dfe06f6d21-pub-subnet + 2017-03-21 13:35:28,311 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsSubnetTests-4f440a5f-54e3-4455-ab9b-39dfe06f6d21-pub-subnet + 2017-03-21 13:35:29,585 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-4f440a5f-54e3-4455-ab9b-39dfe06f6d21-pub-net ok + test_create_subnet_empty_cidr (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... + 2017-03-21 13:35:31,013 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-41fc0db4-71ee-47e6-bec9-316273e5bcc0-pub-net + 2017-03-21 13:35:31,652 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-41fc0db4-71ee-47e6-bec9-316273e5bcc0-pub-net ok + test_create_subnet_empty_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... + 2017-03-21 13:35:32,379 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-1030e0cb-1714-4d18-8619-a03bac0d0257-pub-net + 2017-03-21 13:35:33,516 - neutron_utils - INFO - Creating subnet with name NeutronUtilsSubnetTests-1030e0cb-1714-4d18-8619-a03bac0d0257-pub-subnet + 2017-03-21 13:35:34,160 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-1030e0cb-1714-4d18-8619-a03bac0d0257-pub-net ok + test_create_subnet_null_cidr (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... + 2017-03-21 13:35:35,784 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-1d7522fd-3fb5-4b1c-8741-97d7c47a5f7d-pub-net + 2017-03-21 13:35:36,367 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-1d7522fd-3fb5-4b1c-8741-97d7c47a5f7d-pub-net ok + test_create_subnet_null_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... + 2017-03-21 13:35:37,055 - neutron_utils - INFO - Creating network with name NeutronUtilsSubnetTests-0a8ac1b2-e5d4-4522-a079-7e17945e482e-pub-net + 2017-03-21 13:35:37,691 - neutron_utils - INFO - Deleting network with name NeutronUtilsSubnetTests-0a8ac1b2-e5d4-4522-a079-7e17945e482e-pub-net ok + test_add_interface_router (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:35:38,994 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-433818c9-4472-49a8-9241-791ad0a71d3f-pub-net + 2017-03-21 13:35:40,311 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-433818c9-4472-49a8-9241-791ad0a71d3f-pub-subnet + 2017-03-21 13:35:41,713 - neutron_utils - INFO - Creating router with name - NeutronUtilsRouterTests-433818c9-4472-49a8-9241-791ad0a71d3f-pub-router + 2017-03-21 13:35:44,131 - neutron_utils - INFO - Adding interface to router with name NeutronUtilsRouterTests-433818c9-4472-49a8-9241-791ad0a71d3f-pub-router + 2017-03-21 13:35:45,725 - neutron_utils - INFO - Removing router interface from router named NeutronUtilsRouterTests-433818c9-4472-49a8-9241-791ad0a71d3f-pub-router + 2017-03-21 13:35:47,464 - neutron_utils - INFO - Deleting router with name - NeutronUtilsRouterTests-433818c9-4472-49a8-9241-791ad0a71d3f-pub-router + 2017-03-21 13:35:48,670 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-433818c9-4472-49a8-9241-791ad0a71d3f-pub-subnet + 2017-03-21 13:35:50,921 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-433818c9-4472-49a8-9241-791ad0a71d3f-pub-net ok + test_add_interface_router_null_router (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:35:52,230 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-1fc2de16-2d3e-497b-b947-022b1bf9d90c-pub-net + 2017-03-21 13:35:53,662 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-1fc2de16-2d3e-497b-b947-022b1bf9d90c-pub-subnet + 2017-03-21 13:35:55,203 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-1fc2de16-2d3e-497b-b947-022b1bf9d90c-pub-subnet + 2017-03-21 13:35:55,694 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-1fc2de16-2d3e-497b-b947-022b1bf9d90c-pub-net ok + test_add_interface_router_null_subnet (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:35:57,392 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-2e4fb9f3-312b-4954-8015-435464fdc8b0-pub-net + 2017-03-21 13:35:58,215 - neutron_utils - INFO - Creating router with name - NeutronUtilsRouterTests-2e4fb9f3-312b-4954-8015-435464fdc8b0-pub-router + 2017-03-21 13:36:00,369 - neutron_utils - INFO - Adding interface to router with name NeutronUtilsRouterTests-2e4fb9f3-312b-4954-8015-435464fdc8b0-pub-router + 2017-03-21 13:36:00,369 - neutron_utils - INFO - Deleting router with name - NeutronUtilsRouterTests-2e4fb9f3-312b-4954-8015-435464fdc8b0-pub-router + 2017-03-21 13:36:02,742 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-2e4fb9f3-312b-4954-8015-435464fdc8b0-pub-net ok + test_create_port (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:05,010 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-dde05ce1-a2f8-4c5e-a028-e1ca0e11a05b-pub-net + 2017-03-21 13:36:05,996 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-dde05ce1-a2f8-4c5e-a028-e1ca0e11a05b-pub-subnet + 2017-03-21 13:36:09,103 - neutron_utils - INFO - Creating port for network with name - NeutronUtilsRouterTests-dde05ce1-a2f8-4c5e-a028-e1ca0e11a05b-pub-net + 2017-03-21 13:36:10,312 - neutron_utils - INFO - Deleting port with name NeutronUtilsRouterTests-dde05ce1-a2f8-4c5e-a028-e1ca0e11a05b-port + 2017-03-21 13:36:11,045 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-dde05ce1-a2f8-4c5e-a028-e1ca0e11a05b-pub-subnet + 2017-03-21 13:36:14,265 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-dde05ce1-a2f8-4c5e-a028-e1ca0e11a05b-pub-net ok + test_create_port_empty_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:16,250 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-b986a259-e873-431c-bde4-b2771ace4549-pub-net + 2017-03-21 13:36:16,950 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-b986a259-e873-431c-bde4-b2771ace4549-pub-subnet + 2017-03-21 13:36:17,798 - neutron_utils - INFO - Creating port for network with name - NeutronUtilsRouterTests-b986a259-e873-431c-bde4-b2771ace4549-pub-net + 2017-03-21 13:36:18,544 - neutron_utils - INFO - Deleting port with name NeutronUtilsRouterTests-b986a259-e873-431c-bde4-b2771ace4549-port + 2017-03-21 13:36:19,582 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-b986a259-e873-431c-bde4-b2771ace4549-pub-subnet + 2017-03-21 13:36:21,606 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-b986a259-e873-431c-bde4-b2771ace4549-pub-net ok + test_create_port_invalid_ip (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:23,779 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-7ab3a329-9dd8-4e6f-9d52-aafb47ea5122-pub-net + 2017-03-21 13:36:25,201 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-7ab3a329-9dd8-4e6f-9d52-aafb47ea5122-pub-subnet + 2017-03-21 13:36:25,599 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-7ab3a329-9dd8-4e6f-9d52-aafb47ea5122-pub-subnet + 2017-03-21 13:36:26,220 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-7ab3a329-9dd8-4e6f-9d52-aafb47ea5122-pub-net ok + test_create_port_invalid_ip_to_subnet (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:27,112 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-c016821d-cd4f-4e0f-8f8c-d5cef3392e64-pub-net + 2017-03-21 13:36:28,720 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-c016821d-cd4f-4e0f-8f8c-d5cef3392e64-pub-subnet + 2017-03-21 13:36:29,457 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-c016821d-cd4f-4e0f-8f8c-d5cef3392e64-pub-subnet + 2017-03-21 13:36:29,909 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-c016821d-cd4f-4e0f-8f8c-d5cef3392e64-pub-net ok + test_create_port_null_ip (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:31,037 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-9a86227f-6041-4b04-86a7-1701fb86baa3-pub-net + 2017-03-21 13:36:31,695 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-9a86227f-6041-4b04-86a7-1701fb86baa3-pub-subnet + 2017-03-21 13:36:32,305 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-9a86227f-6041-4b04-86a7-1701fb86baa3-pub-subnet + 2017-03-21 13:36:33,553 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-9a86227f-6041-4b04-86a7-1701fb86baa3-pub-net ok + test_create_port_null_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:34,593 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-42efa897-4f65-4d9b-b19d-fbc61f97c966-pub-net + 2017-03-21 13:36:35,217 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-42efa897-4f65-4d9b-b19d-fbc61f97c966-pub-subnet + 2017-03-21 13:36:36,648 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-42efa897-4f65-4d9b-b19d-fbc61f97c966-pub-subnet + 2017-03-21 13:36:37,251 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-42efa897-4f65-4d9b-b19d-fbc61f97c966-pub-net ok + test_create_port_null_network_object (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:37,885 - neutron_utils - INFO - Creating network with name NeutronUtilsRouterTests-617f4110-45c1-4900-bad1-a6204f34dd64-pub-net + 2017-03-21 13:36:38,468 - neutron_utils - INFO - Creating subnet with name NeutronUtilsRouterTests-617f4110-45c1-4900-bad1-a6204f34dd64-pub-subnet + 2017-03-21 13:36:40,005 - neutron_utils - INFO - Deleting subnet with name NeutronUtilsRouterTests-617f4110-45c1-4900-bad1-a6204f34dd64-pub-subnet + 2017-03-21 13:36:41,637 - neutron_utils - INFO - Deleting network with name NeutronUtilsRouterTests-617f4110-45c1-4900-bad1-a6204f34dd64-pub-net ok + test_create_router_empty_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... ok + test_create_router_null_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... ok + test_create_router_simple (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:43,424 - neutron_utils - INFO - Creating router with name - NeutronUtilsRouterTests-b6a2dafc-38d4-4c46-bb41-2ba9e1c0084e-pub-router + 2017-03-21 13:36:45,013 - neutron_utils - INFO - Deleting router with name - NeutronUtilsRouterTests-b6a2dafc-38d4-4c46-bb41-2ba9e1c0084e-pub-router ok + test_create_router_with_public_interface (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... + 2017-03-21 13:36:47,829 - neutron_utils - INFO - Creating router with name - NeutronUtilsRouterTests-d268dda2-7a30-4d3d-a008-e5aa4592637d-pub-router + 2017-03-21 13:36:49,448 - neutron_utils - INFO - Deleting router with name - NeutronUtilsRouterTests-d268dda2-7a30-4d3d-a008-e5aa4592637d-pub-router ok + test_create_delete_simple_sec_grp (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSecurityGroupTests) ... + 2017-03-21 13:36:51,067 - neutron_utils - INFO - Creating security group with name - NeutronUtilsSecurityGroupTests-1543e861-ea38-4fbe-9723-c27552e3eb7aname + 2017-03-21 13:36:51,493 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-1543e861-ea38-4fbe-9723-c27552e3eb7aname + 2017-03-21 13:36:51,568 - neutron_utils - INFO - Deleting security group with name - NeutronUtilsSecurityGroupTests-1543e861-ea38-4fbe-9723-c27552e3eb7aname + 2017-03-21 13:36:51,772 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-1543e861-ea38-4fbe-9723-c27552e3eb7aname ok + test_create_sec_grp_no_name (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSecurityGroupTests) ... ok + test_create_sec_grp_no_rules (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSecurityGroupTests) ... + 2017-03-21 13:36:52,253 - neutron_utils - INFO - Creating security group with name - NeutronUtilsSecurityGroupTests-57c60864-f46c-4391-ba99-6acc4dd123ddname + 2017-03-21 13:36:52,634 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-57c60864-f46c-4391-ba99-6acc4dd123ddname + 2017-03-21 13:36:52,718 - neutron_utils - INFO - Deleting security group with name - NeutronUtilsSecurityGroupTests-57c60864-f46c-4391-ba99-6acc4dd123ddname ok + test_create_sec_grp_one_rule (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSecurityGroupTests) ... + 2017-03-21 13:36:53,082 - neutron_utils - INFO - Creating security group with name - NeutronUtilsSecurityGroupTests-a3ac62bb-a7e8-4fc2-ba4c-e656f1f3c9a1name + 2017-03-21 13:36:53,483 - neutron_utils - INFO - Retrieving security group rules associate with the security group - NeutronUtilsSecurityGroupTests-a3ac62bb-a7e8-4fc2-ba4c-e656f1f3c9a1name + 2017-03-21 13:36:53,548 - neutron_utils - INFO - Creating security group to security group - NeutronUtilsSecurityGroupTests-a3ac62bb-a7e8-4fc2-ba4c-e656f1f3c9a1name + 2017-03-21 13:36:53,548 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-a3ac62bb-a7e8-4fc2-ba4c-e656f1f3c9a1name + 2017-03-21 13:36:53,871 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-a3ac62bb-a7e8-4fc2-ba4c-e656f1f3c9a1name + 2017-03-21 13:36:53,944 - neutron_utils - INFO - Retrieving security group rules associate with the security group - NeutronUtilsSecurityGroupTests-a3ac62bb-a7e8-4fc2-ba4c-e656f1f3c9a1name + 2017-03-21 13:36:53,991 - neutron_utils - INFO - Retrieving security group with name - NeutronUtilsSecurityGroupTests-a3ac62bb-a7e8-4fc2-ba4c-e656f1f3c9a1name + 2017-03-21 13:36:54,069 - neutron_utils - INFO - Deleting security group rule with ID - 7f76046c-d043-46e0-9d12-4b983525810b + 2017-03-21 13:36:54,185 - neutron_utils - INFO - Deleting security group rule with ID - f18a9ed1-466f-4373-a6b2-82bd317bc838 + 2017-03-21 13:36:54,338 - neutron_utils - INFO - Deleting security group rule with ID - fe34a3d0-948e-47c1-abad-c3ec8d33b2fb + 2017-03-21 13:36:54,444 - neutron_utils - INFO - Deleting security group with name - NeutronUtilsSecurityGroupTests-a3ac62bb-a7e8-4fc2-ba4c-e656f1f3c9a1name ok + test_create_delete_keypair (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsKeypairTests) ... + 2017-03-21 13:36:54,637 - nova_utils - INFO - Creating keypair with name - NovaUtilsKeypairTests-5ce69b6f-d8d0-4b66-bd25-30a22cf3bda0 ok + test_create_key_from_file (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsKeypairTests) ... + 2017-03-21 13:36:58,989 - nova_utils - INFO - Saved public key to - tmp/NovaUtilsKeypairTests-df3e848d-a467-4cc4-99d5-022eb67eee94.pub + 2017-03-21 13:36:58,990 - nova_utils - INFO - Saved private key to - tmp/NovaUtilsKeypairTests-df3e848d-a467-4cc4-99d5-022eb67eee94 + 2017-03-21 13:36:58,990 - nova_utils - INFO - Saving keypair to - tmp/NovaUtilsKeypairTests-df3e848d-a467-4cc4-99d5-022eb67eee94.pub + 2017-03-21 13:36:58,990 - nova_utils - INFO - Creating keypair with name - NovaUtilsKeypairTests-df3e848d-a467-4cc4-99d5-022eb67eee94 ok + test_create_keypair (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsKeypairTests) ... + 2017-03-21 13:36:59,807 - nova_utils - INFO - Creating keypair with name - NovaUtilsKeypairTests-fc7f7ffd-80f6-43df-bd41-a3c014ba8c3d ok + test_floating_ips (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsKeypairTests) ... + 2017-03-21 13:37:02,765 - nova_utils - INFO - Creating floating ip to external network - admin_floating_net ok + test_create_delete_flavor (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsFlavorTests) ... ok + test_create_flavor (snaps.openstack.utils.tests.nova_utils_tests.NovaUtilsFlavorTests) ... ok + test_create_clean_flavor (snaps.openstack.tests.create_flavor_tests.CreateFlavorTests) ... ok + test_create_delete_flavor (snaps.openstack.tests.create_flavor_tests.CreateFlavorTests) ... ok + test_create_flavor (snaps.openstack.tests.create_flavor_tests.CreateFlavorTests) ... ok + test_create_flavor_existing (snaps.openstack.tests.create_flavor_tests.CreateFlavorTests) ... + 2017-03-21 13:37:18,545 - create_image - INFO - Found flavor with name - CreateFlavorTests-3befc152-4319-4f9c-82d4-75f8941d9533name ok + + ---------------------------------------------------------------------- + Ran 48 tests in 171.000s + + OK + 2017-03-21 13:37:18,620 - functest.core.testcase_base - INFO - api_check OK + 2017-03-21 13:37:18,977 - functest.core.testcase_base - INFO - The results were successfully pushed to DB + 2017-03-21 13:37:18,977 - run_tests - INFO - Test execution time: 02:52 + 2017-03-21 13:37:18,981 - run_tests - INFO - + + 2017-03-21 13:37:18,981 - run_tests - INFO - ============================================ + 2017-03-21 13:37:18,981 - run_tests - INFO - Running test case 'snaps_health_check'... + 2017-03-21 13:37:18,981 - run_tests - INFO - ============================================ + 2017-03-21 13:37:19,098 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds + 2017-03-21 13:37:19,099 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.7:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None + 2017-03-21 13:37:19,434 - file_utils - INFO - Attempting to read OS environment file - /home/opnfv/functest/conf/openstack.creds + 2017-03-21 13:37:19,435 - openstack_tests - INFO - OS Credentials = OSCreds - username=admin, password=admin, auth_url=http://192.168.10.7:5000/v3, project_name=admin, identity_api_version=3, image_api_version=1, network_api_version=2, compute_api_version=2, user_domain_id=default, proxy_settings=None + test_check_vm_ip_dhcp (snaps.openstack.tests.create_instance_tests.SimpleHealthCheck) ... + 2017-03-21 13:37:26,082 - create_image - INFO - Creating image + 2017-03-21 13:37:28,793 - create_image - INFO - Image is active with name - SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-image + 2017-03-21 13:37:28,793 - create_image - INFO - Image is now active with name - SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-image + 2017-03-21 13:37:28,794 - OpenStackNetwork - INFO - Creating neutron network SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-priv-net... + 2017-03-21 13:37:29,308 - neutron_utils - INFO - Creating network with name SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-priv-net + 2017-03-21 13:37:30,771 - neutron_utils - INFO - Creating subnet with name SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-priv-subnet + 2017-03-21 13:37:36,974 - neutron_utils - INFO - Creating port for network with name - SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-priv-net + 2017-03-21 13:37:38,188 - create_instance - INFO - Creating VM with name - SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-inst + 2017-03-21 13:37:41,538 - create_instance - INFO - Created instance with name - SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-inst + 2017-03-21 13:37:59,577 - create_instance - INFO - VM is - ACTIVE + 2017-03-21 13:37:59,577 - create_instance_tests - INFO - Looking for expression Lease of.*obtained in the console log + 2017-03-21 13:37:59,830 - create_instance_tests - INFO - DHCP lease obtained logged in console + 2017-03-21 13:37:59,830 - create_instance_tests - INFO - With correct IP address + 2017-03-21 13:37:59,830 - create_instance - INFO - Deleting Port - SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5dport-1 + 2017-03-21 13:37:59,830 - neutron_utils - INFO - Deleting port with name SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5dport-1 + 2017-03-21 13:38:00,705 - create_instance - INFO - Deleting VM instance - SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-inst + 2017-03-21 13:38:01,412 - create_instance - INFO - Checking deletion status + 2017-03-21 13:38:04,938 - create_instance - INFO - VM has been properly deleted VM with name - SimpleHealthCheck-23244728-5a5a-4545-9b16-50257a595e5d-inst + ok + + ---------------------------------------------------------------------- + Ran 1 test in 46.982s + + OK + 2017-03-21 13:38:06,417 - functest.core.testcase_base - INFO - snaps_health_check OK + 2017-03-21 13:38:06,778 - functest.core.testcase_base - INFO - The results were successfully pushed to DB + 2017-03-21 13:38:06,779 - run_tests - INFO - Test execution time: 00:47 + 2017-03-21 13:38:06,779 - run_tests - INFO - and root@22e436918db0:~/repos/functest/ci# functest testcase run vping_ssh @@ -206,12 +412,12 @@ command is used with 'functest tier':: root@22e436918db0:~/repos/functest/ci# functest tier get-tests healthcheck Test cases in tier 'healthcheck': - ['healthcheck'] + ['connection_check', 'api_check', 'snaps_health_check'] Please note that for some scenarios some test cases might not be launched. For example, the last example displayed only the 'odl' testcase for the given -environment. In this particular system the deployment does not support the 'onos' SDN +environment. In this particular system the deployment does not support the 'ocl' SDN Controller Test Case; for example. **Important** If you use the command 'functest tier run <tier_name>', then the @@ -229,32 +435,35 @@ two possibilities:: Functest includes a cleaning mechanism in order to remove all the OpenStack resources except those present before running any test. The script -*$REPOS_DIR/functest/functest/utils/generate_defaults.py* is called once when setting up +*$REPOS_DIR/functest/functest/utils/openstack_snapshot.py* is called once when setting up the Functest environment (i.e. CLI command 'functest env prepare') to snapshot all the OpenStack resources (images, networks, volumes, security groups, tenants, users) so that an eventual cleanup does not remove any of these defaults. -The script **clean_openstack.py** which is located in +It is also called before running a test except if it is disabled by configuration +in the testcases.yaml file (clean_flag=false). This flag has been added as some +upstream tests already include their own cleaning mechanism (e.g. Rally). + +The script **openstack_clean.py** which is located in *$REPOS_DIR/functest/functest/utils/* is normally called after a test execution. It is in charge of cleaning the OpenStack resources that are not specified in the defaults file generated previously which is stored in -*/home/opnfv/functest/conf/os_defaults.yaml* in the Functest docker container. +*/home/opnfv/functest/conf/openstack_snapshot.yaml* in the Functest docker container. It is important to mention that if there are new OpenStack resources created -manually after preparing the Functest environment, they will be removed, unless -you use the special method of invoking the test case with specific suppression -of clean up. (See the `Troubleshooting`_ section). +manually after the snapshot done before running the tests, they will be removed, +unless you use the special method of invoking the test case with specific +suppression of clean up. (See the `Troubleshooting`_ section). The reason to include this cleanup meachanism in Functest is because some -test suites such as Tempest or Rally create a lot of resources (users, -tenants, networks, volumes etc.) that are not always properly cleaned, so this -function has been set to keep the system as clean as it was before a -full Functest execution. +test suites create a lot of resources (users, tenants, networks, volumes etc.) +that are not always properly cleaned, so this function has been set to keep the +system as clean as it was before a full Functest execution. Although the Functest CLI provides an easy way to run any test, it is possible to do a direct call to the desired test script. For example: - python $REPOS_DIR/functest/functest/opnfv_tests/OpenStack/vPing/vPing_ssh.py -d + python $REPOS_DIR/functest/functest/opnfv_tests/openstack/vping/vping_ssh.py Automated testing @@ -282,38 +491,18 @@ The mechanism remains however as part of the CI evolution. CI provides some useful information passed to the container as environment variables: - * Installer (apex|compass|daisy|fuel|joid), stored in INSTALLER_TYPE + * Installer (apex|compass|fuel|joid), stored in INSTALLER_TYPE * Installer IP of the engine or VM running the actual deployment, stored in INSTALLER_IP * The scenario [controller]-[feature]-[mode], stored in DEPLOY_SCENARIO with - * controller = (odl|onos|ocl|nosdn) - * feature = (ovs(dpdk)|kvm|sfc|bgpvpn|multisites) + * controller = (odl|ocl|nosdn|onos) + * feature = (ovs(dpdk)|kvm|sfc|bgpvpn|multisites|netready|ovs_dpdk_bar) * mode = (ha|noha) The constraints per test case are defined in the Functest configuration file */home/opnfv/repos/functest/functest/ci/testcases.yaml*:: tiers: - - - name: healthcheck - order: 0 - ci_loop: '(daily)|(weekly)' - description : >- - First tier to be executed to verify the basic - operations in the VIM. - testcases: - - - name: healthcheck - criteria: 'status == "PASS"' - blocking: true - description: >- - This test case verifies the basic OpenStack services like - Keystone, Glance, Cinder, Neutron and Nova. - - dependencies: - installer: '' - scenario: '' - - name: smoke order: 1 @@ -343,13 +532,14 @@ We may distinguish 2 levels in the test case description: At the tier level, we define the following parameters: - * ci_loop: indicate if in automated mode, the test case must be run in daily and/or weekly jobs + * ci_loop: indicate if in automated mode, the test case must be run in dail and/or weekly jobs * description: a high level view of the test case For a given test case we defined: * the name of the test case * the criteria (experimental): a criteria used to declare the test case as PASS or FAIL * blocking: if set to true, if the test is failed, the execution of the following tests is canceled + * clean_flag: shall the functect internal mechanism be invoked after the test * the description of the test case * the dependencies: a combination of 2 regex on the scenario and the installer name * run: In Danube we introduced the notion of abstract class in order to harmonize the way to run internal, feature or vnf tests diff --git a/docs/testing/user/userguide/troubleshooting.rst b/docs/testing/user/userguide/troubleshooting.rst index 845501916..bcd9e1c84 100644 --- a/docs/testing/user/userguide/troubleshooting.rst +++ b/docs/testing/user/userguide/troubleshooting.rst @@ -44,8 +44,8 @@ These test cases can be run inside the container, using new Functest CLI as foll The Functest CLI is designed to route a call to the corresponding internal python scripts, located in paths: -*$REPOS_DIR/functest/functest/opnfv_tests/vPing/CI/libraries/vPing_ssh.py* and -*$REPOS_DIR/functest/functest/opnfv_tests/vPing/CI/libraries/vPing_userdata.py* +*$REPOS_DIR/functest/functest/opnfv_tests/openstack/vping/vping_ssh.py* and +*$REPOS_DIR/functest/functest/opnfv_tests/openstack/vping/vping_userdata.py* Notes: @@ -122,7 +122,7 @@ This test case creates a floating IP on the external network and assigns it to the second instance **opnfv-vping-2**. The purpose of this is to establish a SSH connection to that instance and SCP a script that will ping the first instance. This script is located in the repository under -*$REPOS_DIR/functest/functest/opnfv_tests/OpenStack/vPing/ping.sh* and takes an IP as +*$REPOS_DIR/functest/functest/opnfv_tests/openstack/vping/ping.sh* and takes an IP as a parameter. When the SCP is completed, the test will do an SSH call to that script inside the second instance. Some problems can happen here:: @@ -238,7 +238,8 @@ of the following | execution test cases are | Rally in Functest, does not contain all the needed | | missing | parameters or some parameters are not set properly. | | | The tempest.conf file is located in directory | -| | '/home/opnfv/.rally/tempest/for-deployment-<UUID>' | +| | '/home/opnfv/.rally/verification/verifier-<UUID> | +| | /for-deployment-<UUID>' | | | in the Functest Docker container. Use the "rally | | | deployment list" command in order to check the UUID | | | the UUID of the current deployment. | @@ -257,30 +258,6 @@ Rally The same error causes which were mentioned above for Tempest test cases, may also lead to errors in Rally as well. -It is possible to run only one Rally scenario, instead of the whole suite. -To do that, call the alternative python script as follows:: - - python $REPOS_DIR/functest/functest/opnfv_tests/OpenStack/rally/run_rally-cert.py -h - usage: run_rally-cert.py [-h] [-d] [-r] [-s] [-v] [-n] test_name - - positional arguments: - test_name Module name to be tested. Possible values are : [ - authenticate | glance | cinder | heat | keystone | neutron | - nova | quotas | requests | vm | all ] The 'all' value - performs all possible test scenarios - - optional arguments: - -h, --help show this help message and exit - -d, --debug Debug mode - -r, --report Create json result file - -s, --smoke Smoke test mode - -v, --verbose Print verbose info about the progress - -n, --noclean Don't clean the created resources for this test. - -For example, to run the Glance scenario with debug information:: - - python $REPOS_DIR/functest/functest/opnfv_tests/OpenStack/rally/run_rally-cert.py -d glance - Possible scenarios are: * authenticate * glance @@ -294,14 +271,10 @@ Possible scenarios are: * vm To know more about what those scenarios are doing, they are defined in directory: -*$REPOS_DIR/functest/functest/opnfv_tests/OpenStack/rally/scenario* +*$REPOS_DIR/functest/functest/opnfv_tests/openstack/rally/scenario* For more info about Rally scenario definition please refer to the Rally official documentation. `[3]`_ -If the flag *all* is specified, it will run all the scenarios one by one. Please -note that this might take some time (~1,5hr), taking around 1 hour alone to -complete the Nova scenario. - To check any possible problems with Rally, the logs are stored under */home/opnfv/functest/results/rally/* in the Functest Docker container. @@ -322,10 +295,8 @@ If any of the other test cases fails, check that Neutron and ODL have been correctly configured to work together. Check Neutron configuration files, accounts, IP addresses etc.). - ONOS ^^^^ - Please refer to the ONOS documentation. `ONOSFW User Guide`_ . @@ -335,14 +306,8 @@ Features Please refer to the dedicated feature user guides for details. -security_scan -^^^^^^^^^^^^^ - -See OpenSCAP web site: https://www.open-scap.org/ - - -NFV +VNF --- cloudify_ims @@ -378,10 +343,5 @@ described in the following table: +-----------------------------------+------------------------------------+ -parser -^^^^^^ - -For now log info is the only way to do trouble shooting - .. _`OPNFV Functest Developer Guide`: http://artifacts.opnfv.org/functest/docs/devguide/# |