diff options
Diffstat (limited to 'docs/testing/user')
| -rw-r--r-- | docs/testing/user/configguide/configguide.rst | 153 |
1 files changed, 89 insertions, 64 deletions
diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index c04689191..07f0a1681 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -16,19 +16,11 @@ Docker images are available on the dockerhub: * opnfv/functest-core * opnfv/functest-healthcheck * opnfv/functest-smoke + * opnfv/functest-benchmarking * opnfv/functest-features * opnfv/functest-components * opnfv/functest-vnf -The tag "opnfv-6.0.0" is the official release image in Fraser, but you can -also pull "fraser" tag as it is being maintained by Functest team and might -include bugfixes. - -The Functest docker container environment can -in principle- be also -used with non-OPNFV official installers (e.g. 'devstack'), with the -**disclaimer** that support for such environments is outside of the -scope and responsibility of the OPNFV project. - Preparing your environment ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -62,10 +54,9 @@ Create a directory for the different images (attached as a Docker volume):: images/CentOS-7-x86_64-GenericCloud.qcow2 images/cirros-0.4.0-x86_64-disk.img images/cirros-0.4.0-x86_64-lxc.tar.gz - images/cirros-d161201-aarch64-disk.img - images/cirros-d161201-aarch64-initramfs - images/cirros-d161201-aarch64-kernel images/cloudify-manager-premium-4.0.1.qcow2 + images/shaker-image-arm64.qcow2 + images/shaker-image.qcow images/ubuntu-14.04-server-cloudimg-amd64-disk1.img images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img images/ubuntu-16.04-server-cloudimg-amd64-disk1.img @@ -86,11 +77,22 @@ Results shall be displayed as follows:: +----------------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +----------------------------+------------------+---------------------+------------------+----------------+ - | connection_check | functest | healthcheck | 00:07 | PASS | - | api_check | functest | healthcheck | 07:46 | PASS | - | snaps_health_check | functest | healthcheck | 00:36 | PASS | + | connection_check | functest | healthcheck | 00:09 | PASS | + | tenantnetwork1 | functest | healthcheck | 00:14 | PASS | + | tenantnetwork2 | functest | healthcheck | 00:11 | PASS | + | vmready1 | functest | healthcheck | 00:19 | PASS | + | vmready2 | functest | healthcheck | 00:16 | PASS | + | singlevm1 | functest | healthcheck | 00:41 | PASS | + | singlevm2 | functest | healthcheck | 00:36 | PASS | + | vping_ssh | functest | healthcheck | 00:46 | PASS | + | vping_userdata | functest | healthcheck | 00:41 | PASS | + | cinder_test | functest | healthcheck | 01:18 | PASS | + | api_check | functest | healthcheck | 10:33 | PASS | + | snaps_health_check | functest | healthcheck | 00:44 | PASS | + | odl | functest | healthcheck | 00:00 | SKIP | +----------------------------+------------------+---------------------+------------------+----------------+ - NOTE: the duration is a reference and it might vary depending on your SUT. + +NOTE: the duration is a reference and it might vary depending on your SUT. Testing smoke suite ^^^^^^^^^^^^^^^^^^^ @@ -104,21 +106,45 @@ Run smoke suite:: Results shall be displayed as follows:: - +------------------------------+------------------+---------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +------------------------------+------------------+---------------+------------------+----------------+ - | vping_ssh | functest | smoke | 00:57 | PASS | - | vping_userdata | functest | smoke | 00:33 | PASS | - | tempest_smoke_serial | functest | smoke | 13:22 | PASS | - | rally_sanity | functest | smoke | 24:07 | PASS | - | refstack_defcore | functest | smoke | 05:21 | PASS | - | patrole | functest | smoke | 04:29 | PASS | - | snaps_smoke | functest | smoke | 46:54 | PASS | - | odl | functest | smoke | 00:00 | SKIP | - | neutron_trunk | functest | smoke | 00:00 | SKIP | - +------------------------------+------------------+---------------+------------------+----------------+ - Note: if the scenario does not support some tests, they are indicated as SKIP. - See User guide for details. + +------------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +------------------------------------+------------------+---------------+------------------+----------------+ + | tempest_smoke | functest | smoke | 04:24 | PASS | + | neutron-tempest-plugin-api | functest | smoke | 10:43 | PASS | + | rally_sanity | functest | smoke | 23:16 | PASS | + | refstack_defcore | functest | smoke | 06:30 | PASS | + | patrole | functest | smoke | 02:08 | PASS | + | snaps_smoke | functest | smoke | 00:00 | SKIP | + | neutron_trunk | functest | smoke | 02:09 | PASS | + | networking-bgpvpn | functest | smoke | 00:00 | SKIP | + | networking-sfc | functest | smoke | 00:00 | SKIP | + | barbican | functest | smoke | 00:00 | SKIP | + +------------------------------------+------------------+---------------+------------------+----------------+ + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. + +Testing benchmarking suite +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run benchmarking suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-benchmarking + +Results shall be displayed as follows:: + + +-------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------+------------------+----------------------+------------------+----------------+ + | vmtp | functest | benchmarking | 18:43 | PASS | + | shaker | functest | benchmarking | 29:45 | PASS | + +-------------------+------------------+----------------------+------------------+----------------+ + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. Testing features suite ^^^^^^^^^^^^^^^^^^^^^^ @@ -140,9 +166,12 @@ Results shall be displayed as follows:: | functest-odl-sfc | sfc | features | 00:00 | SKIP | | barometercollectd | barometer | features | 00:00 | SKIP | | fds | fastdatastacks | features | 00:00 | SKIP | + | vgpu | functest | features | 00:00 | SKIP | + | stor4nfv_os | stor4nfv | features | 00:00 | SKIP | +-----------------------------+------------------------+------------------+------------------+----------------+ - Note: if the scenario does not support some tests, they are indicated as SKIP. - See User guide for details. + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. Testing components suite ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -156,12 +185,13 @@ Run components suite:: Results shall be displayed as follows:: - +-------------------------------+------------------+--------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-------------------------------+------------------+--------------------+------------------+----------------+ - | tempest_full_parallel | functest | components | 48:28 | PASS | - | rally_full | functest | components | 126:02 | PASS | - +-------------------------------+------------------+--------------------+------------------+----------------+ + +--------------------------+------------------+--------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------+------------------+--------------------+------------------+----------------+ + | tempest_full | functest | components | 53:40 | FAIL | + | tempest_scenario | functest | components | 18:50 | PASS | + | rally_full | functest | components | 167:13 | PASS | + +--------------------------+------------------+--------------------+------------------+----------------+ Testing vnf suite ^^^^^^^^^^^^^^^^^ @@ -178,12 +208,13 @@ Results shall be displayed as follows:: +----------------------+------------------+--------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +----------------------+------------------+--------------+------------------+----------------+ - | cloudify_ims | functest | vnf | 28:15 | PASS | - | vyos_vrouter | functest | vnf | 17:59 | PASS | - | juju_epc | functest | vnf | 46:44 | PASS | + | cloudify | functest | vnf | 04:05 | PASS | + | cloudify_ims | functest | vnf | 24:07 | PASS | + | heat_ims | functest | vnf | 18:15 | PASS | + | vyos_vrouter | functest | vnf | 15:48 | PASS | + | juju_epc | functest | vnf | 29:38 | PASS | +----------------------+------------------+--------------+------------------+----------------+ - Functest Dockers for Kubernetes deployment ------------------------------------------ Docker images are available on the dockerhub: @@ -198,14 +229,7 @@ Preparing your environment cat env:: - INSTALLER_TYPE=XXX - DEPLOY_SCENARIO=XXX - -cat k8s.creds:: - - export KUBERNETES_PROVIDER=local - export KUBE_MASTER_URL=XXX - export KUBE_MASTER_IP=XXX + DEPLOY_SCENARIO=k8s-XXX Testing healthcheck suite ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -213,7 +237,6 @@ Testing healthcheck suite Run healthcheck suite:: sudo docker run -it --env-file env \ - -v $(pwd)/k8s.creds:/home/opnfv/functest/conf/env_file \ -v $(pwd)/config:/root/.kube/config \ opnfv/functest-kubernetes-healthcheck @@ -222,11 +245,11 @@ volume mapped to ~/.kube/config inside kubernetes container. Results shall be displayed as follows:: - +-------------------------+------------------+-----------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-------------------------+------------------+-----------------+------------------+----------------+ - | k8s_smoke | functest | healthcheck | 01:54 | PASS | - +-------------------------+------------------+-----------------+------------------+----------------+ + +-------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------+------------------+---------------------+------------------+----------------+ + | k8s_smoke | functest | healthcheck | 02:27 | PASS | + +-------------------+------------------+---------------------+------------------+----------------+ Testing smoke suite ^^^^^^^^^^^^^^^^^^^ @@ -234,7 +257,6 @@ Testing smoke suite Run smoke suite:: sudo docker run -it --env-file env \ - -v $(pwd)/k8s.creds:/home/opnfv/functest/conf/env_file \ -v $(pwd)/config:/root/.kube/config \ opnfv/functest-kubernetes-smoke @@ -243,7 +265,7 @@ Results shall be displayed as follows:: +-------------------------+------------------+---------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +-------------------------+------------------+---------------+------------------+----------------+ - | k8s_conformance | functest | smoke | 57:47 | PASS | + | k8s_conformance | functest | smoke | 57:14 | PASS | +-------------------------+------------------+---------------+------------------+----------------+ Testing features suite @@ -252,7 +274,6 @@ Testing features suite Run features suite:: sudo docker run -it --env-file env \ - -v $(pwd)/k8s.creds:/home/opnfv/functest/conf/env_file \ -v $(pwd)/config:/root/.kube/config \ opnfv/functest-kubernetes-features @@ -269,10 +290,13 @@ Environment variables ===================== Several environement variables may be specified: - * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid) + * INSTALLER_IP=<Specific IP Address> * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode> - + * NAMESERVER=XXX # if not 8.8.8.8 + * VOLUME_DEVICE_NAME=XXX # if not vdb + * EXTERNAL_NETWORK=XXX # if not first network with router:external=True + * NEW_USER_ROLE=XXX # if not member INSTALLER_IP is required by Barometer in order to access the installer node and the deployment. @@ -305,8 +329,9 @@ to the Functest Docker Container, using the -e only relevant to Jenkins CI invoked testing and **should not be used** when performing manual test scenarios: - * NODE_NAME = <Test POD Name> - * BUILD_TAG = <Jenkins Build Tag> + * INSTALLER_TYPE=(apex|compass|daisy|fuel) + * NODE_NAME=<Test POD Name> + * BUILD_TAG=<Jenkins Build Tag> where: |