diff options
Diffstat (limited to 'docs/testing/user/configguide')
| -rw-r--r-- | docs/testing/user/configguide/ci.rst | 13 | ||||
| -rw-r--r-- | docs/testing/user/configguide/configguide.rst | 621 | ||||
| -rw-r--r-- | docs/testing/user/configguide/index.rst | 122 | ||||
| -rw-r--r-- | docs/testing/user/configguide/intro.rst | 86 | ||||
| -rw-r--r-- | docs/testing/user/configguide/prerequisites.rst | 21 |
5 files changed, 389 insertions, 474 deletions
diff --git a/docs/testing/user/configguide/ci.rst b/docs/testing/user/configguide/ci.rst deleted file mode 100644 index f3901d867..000000000 --- a/docs/testing/user/configguide/ci.rst +++ /dev/null @@ -1,13 +0,0 @@ -Integration in CI -================= -In CI we use the Docker images and execute the appropriate commands within the -container from Jenkins. - -4 steps have been defined:: - * functest-cleanup: clean existing functest dockers on the jumphost - * functest-daily: run dockers opnfv/functest-* (healthcheck, smoke, features, vnf) - * functest-store-results: push logs to artifacts - -See `[3]`_ for details. - -.. _`[3]`: https://git.opnfv.org/releng/tree/jjb/functest/functest-daily-jobs.yml diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index d63286475..47c25adb1 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -1,4 +1,3 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. .. SPDX-License-Identifier: CC-BY-4.0 Installation and configuration @@ -10,201 +9,348 @@ the refactoring on dependency management should allow the creation of light and fully customized docker images. -Functest Dockers ----------------- +Functest Dockers for OpenStack deployment +----------------------------------------- Docker images are available on the dockerhub: - * opnfv/functest-core * opnfv/functest-healthcheck * opnfv/functest-smoke - * opnfv/functest-features - * opnfv/functest-components + * opnfv/functest-smoke-cntt + * opnfv/functest-benchmarking + * opnfv/functest-benchmarking-cntt * opnfv/functest-vnf - * opnfv/functest-parser - * opnfv/functest-restapi - -The tag "opnfv-5.0.0" is the official release image in Euphrates, but you can also pull "euphrates" -tag as it is being maintained by Functest team and might include bugfixes. - -The Functest docker container environment can -in principle- be also -used with non-OPNFV official installers (e.g. 'devstack'), with the -**disclaimer** that support for such environments is outside of the -scope and responsibility of the OPNFV project. Preparing your environment --------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^ cat env:: - INSTALLER_TYPE=XXX - INSTALLER_IP=XXX - EXTERNAL_NETWORK=XXX - DEPLOY_SCENARIO=XXX + DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario + NAMESERVER=XXX # if not 8.8.8.8 + EXTERNAL_NETWORK=XXX # if not first network with router:external=True + DASHBOARD_URL=XXX # else tempest_horizon will be skipped + NEW_USER_ROLE=XXX # if not member + SDN_CONTROLLER_IP=XXX # if odl scenario + VOLUME_DEVICE_NAME=XXX # if not vdb + FLAVOR_EXTRA_SPECS=hw:mem_page_size:large # if fdio scenarios See section on environment variables for details. -cat openstack.creds:: +cat env_file:: export OS_AUTH_URL=XXX export OS_USER_DOMAIN_NAME=XXX export OS_PROJECT_DOMAIN_NAME=XXX export OS_USERNAME=XXX - export OS_TENANT_NAME=XXX export OS_PROJECT_NAME=XXX export OS_PASSWORD=XXX - export OS_VOLUME_API_VERSION=XXX - export OS_IDENTITY_API_VERSION=XXX - export OS_IMAGE_API_VERSION=XXX + export OS_IDENTITY_API_VERSION=3 + export OS_REGION_NAME=XXX See section on OpenStack credentials for details. Create a directory for the different images (attached as a Docker volume):: - mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/euphrates | bash -s -- images && ls -1 images/* - - images/CentOS-7-aarch64-GenericCloud.qcow2 - images/CentOS-7-aarch64-GenericCloud.qcow2.xz - images/CentOS-7-x86_64-GenericCloud.qcow2 - images/cirros-0.4.0-x86_64-disk.img - images/cirros-0.4.0-x86_64-lxc.tar.gz - images/cirros-d161201-aarch64-disk.img - images/cirros-d161201-aarch64-initramfs - images/cirros-d161201-aarch64-kernel - images/cloudify-manager-premium-4.0.1.qcow2 - images/img - images/trusty-server-cloudimg-amd64-disk1.img + mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | sh -s -- images && ls -1 images/* + + images/cirros-0.5.1-aarch64-disk.img + images/cirros-0.5.1-x86_64-disk.img + images/cloudify-docker-manager-community-19.01.24.tar + images/Fedora-Cloud-Base-30-1.2.x86_64.qcow2 + images/shaker-image-1.3.0+stretch.qcow2 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img images/ubuntu-16.04-server-cloudimg-amd64-disk1.img - images/vyos-1.1.7.img - + images/vyos-1.1.8-amd64.qcow2 Testing healthcheck suite --------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^ Run healthcheck suite:: sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ - -v $(pwd)/images:/home/opnfv/functest/images \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ opnfv/functest-healthcheck Results shall be displayed as follows:: - +----------------------------+------------------+---------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +----------------------------+------------------+---------------------+------------------+----------------+ - | connection_check | functest | healthcheck | 00:02 | PASS | - | api_check | functest | healthcheck | 04:57 | PASS | - | snaps_health_check | functest | healthcheck | 00:51 | PASS | - +----------------------------+------------------+---------------------+------------------+----------------+ - NOTE: the duration is a reference and it might vary depending on your SUT. + +--------------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------+------------------+---------------------+------------------+----------------+ + | connection_check | functest | healthcheck | 00:03 | PASS | + | tenantnetwork1 | functest | healthcheck | 00:05 | PASS | + | tenantnetwork2 | functest | healthcheck | 00:06 | PASS | + | vmready1 | functest | healthcheck | 00:06 | PASS | + | vmready2 | functest | healthcheck | 00:08 | PASS | + | singlevm1 | functest | healthcheck | 00:32 | PASS | + | singlevm2 | functest | healthcheck | 00:37 | PASS | + | vping_ssh | functest | healthcheck | 00:46 | PASS | + | vping_userdata | functest | healthcheck | 00:39 | PASS | + | cinder_test | functest | healthcheck | 01:05 | PASS | + | tempest_smoke | functest | healthcheck | 05:39 | PASS | + | tempest_horizon | functest | healthcheck | 01:05 | PASS | + | odl | functest | healthcheck | 00:00 | SKIP | + +--------------------------+------------------+---------------------+------------------+----------------+ + +NOTE: the duration is a reference and it might vary depending on your SUT. Testing smoke suite -------------------- +^^^^^^^^^^^^^^^^^^^ Run smoke suite:: sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ - -v $(pwd)/images:/home/opnfv/functest/images \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ opnfv/functest-smoke Results shall be displayed as follows:: - +------------------------------+------------------+---------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +------------------------------+------------------+---------------+------------------+----------------+ - | vping_ssh | functest | smoke | 01:19 | PASS | - | vping_userdata | functest | smoke | 01:56 | PASS | - | tempest_smoke_serial | functest | smoke | 26:30 | PASS | - | rally_sanity | functest | smoke | 19:42 | PASS | - | refstack_defcore | functest | smoke | 22:00 | PASS | - | snaps_smoke | functest | smoke | 41:14 | PASS | - | odl | functest | smoke | 00:16 | PASS | - | odl_netvirt | functest | smoke | 00:00 | SKIP | - | fds | functest | smoke | 00:00 | SKIP | - +------------------------------+------------------+---------------+------------------+----------------+ - Note: if the scenario does not support some tests, they are indicated as SKIP. - See User guide for details. - -Testing features suite ----------------------- - -Run features suite:: + +---------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+---------------+------------------+----------------+ + | tempest_neutron | functest | smoke | 15:30 | PASS | + | tempest_cinder | functest | smoke | 02:01 | PASS | + | tempest_keystone | functest | smoke | 01:17 | PASS | + | tempest_heat | functest | smoke | 22:14 | PASS | + | tempest_telemetry | functest | smoke | 00:00 | SKIP | + | rally_sanity | functest | smoke | 17:24 | PASS | + | refstack_compute | functest | smoke | 07:03 | PASS | + | refstack_object | functest | smoke | 02:09 | PASS | + | refstack_platform | functest | smoke | 07:31 | PASS | + | tempest_full | functest | smoke | 41:52 | PASS | + | tempest_scenario | functest | smoke | 08:42 | PASS | + | tempest_slow | functest | smoke | 43:42 | PASS | + | patrole_admin | functest | smoke | 21:06 | PASS | + | patrole_member | functest | smoke | 21:23 | PASS | + | patrole_reader | functest | smoke | 21:56 | PASS | + | tempest_barbican | functest | smoke | 02:30 | PASS | + | tempest_octavia | functest | smoke | 00:00 | SKIP | + | tempest_cyborg | functest | smoke | 00:00 | SKIP | + +---------------------------+------------------+---------------+------------------+----------------+ + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. + +Testing smoke CNTT suite +^^^^^^^^^^^^^^^^^^^^^^^^ + +Run smoke-cntt suite:: sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ - -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-features + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-smoke-cntt Results shall be displayed as follows:: - +---------------------------+--------------------------+------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +---------------------------+--------------------------+------------------+------------------+----------------+ - | promise | promise | features | 00:00 | SKIP | - | bgpvpn | sdnvpn | features | 00:00 | SKIP | - | security_scan | securityscanning | features | 00:00 | SKIP | - | functest-odl-sfc | sfc | features | 00:00 | SKIP | - | domino-multinode | domino | features | 00:00 | SKIP | - | barometercollectd | barometer | features | 00:00 | SKIP | - +---------------------------+--------------------------+------------------+------------------+----------------+ - Note: if the scenario does not support some tests, they are indicated as SKIP. - See User guide for details. + +-------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------------+------------------+---------------+------------------+----------------+ + | tempest_neutron_cntt | functest | smoke | 11:35 | PASS | + | tempest_cinder_cntt | functest | smoke | 01:58 | PASS | + | tempest_keystone_cntt | functest | smoke | 01:13 | PASS | + | tempest_heat_cntt | functest | smoke | 22:32 | PASS | + | rally_sanity_cntt | functest | smoke | 17:16 | PASS | + | tempest_full_cntt | functest | smoke | 41:13 | PASS | + | tempest_scenario_cntt | functest | smoke | 08:57 | PASS | + | tempest_slow_cntt | functest | smoke | 35:58 | PASS | + +-------------------------------+------------------+---------------+------------------+----------------+ -Testing components suite ------------------------- +Testing benchmarking suite +^^^^^^^^^^^^^^^^^^^^^^^^^^ -Run components suite:: +Run benchmarking suite:: sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ - -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-components + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-benchmarking Results shall be displayed as follows:: - +-------------------------------+------------------+--------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-------------------------------+------------------+--------------------+------------------+----------------+ - | tempest_full_parallel | functest | components | 102:48 | PASS | - | rally_full | functest | components | 160:58 | PASS | - +-------------------------------+------------------+--------------------+------------------+----------------+ + +--------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------+------------------+----------------------+------------------+----------------+ + | rally_full | functest | benchmarking | 93:03 | PASS | + | rally_jobs | functest | benchmarking | 27:05 | PASS | + | vmtp | functest | benchmarking | 17:56 | PASS | + | shaker | functest | benchmarking | 24:02 | PASS | + +--------------------+------------------+----------------------+------------------+----------------+ + +Note: if the scenario does not support some tests, they are indicated as SKIP. +See User guide for details. + +Testing benchmarking CNTT suite +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run benchmarking-cntt suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-benchmarking-cntt + +Results shall be displayed as follows:: + + +-------------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+----------------------+------------------+----------------+ + | rally_full_cntt | functest | benchmarking | 89:52 | PASS | + | rally_jobs_cntt | functest | benchmarking | 19:39 | PASS | + | vmtp | functest | benchmarking | 16:59 | PASS | + | shaker | functest | benchmarking | 23:43 | PASS | + +-------------------------+------------------+----------------------+------------------+----------------+ Testing vnf suite ------------------ +^^^^^^^^^^^^^^^^^ Run vnf suite:: -sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ - -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-vnf + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-vnf + +Results shall be displayed as follows:: + + +----------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +----------------------+------------------+--------------+------------------+----------------+ + | cloudify | functest | vnf | 05:08 | PASS | + | cloudify_ims | functest | vnf | 24:46 | PASS | + | heat_ims | functest | vnf | 33:12 | PASS | + | vyos_vrouter | functest | vnf | 15:53 | PASS | + | juju_epc | functest | vnf | 27:52 | PASS | + +----------------------+------------------+--------------+------------------+----------------+ + +Functest Dockers for Kubernetes deployment +------------------------------------------ +Docker images are available on the dockerhub: + + * opnfv/functest-kubernetes-healthcheck + * opnfv/functest-kubernetes-smoke + * opnfv/functest-kubernetes-security + * opnfv/functest-kubernetes-benchmarking + * opnfv/functest-kubernetes-cnf + +Preparing your environment +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +cat env:: + + DEPLOY_SCENARIO=k8s-XXX + +Testing healthcheck suite +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run healthcheck suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-healthcheck + +A config file in the current dir 'config' is also required, which should be +volume mapped to ~/.kube/config inside kubernetes container. + +Results shall be displayed as follows:: + + +-------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------+------------------+---------------------+------------------+----------------+ + | k8s_quick | functest | healthcheck | 00:18 | PASS | + | k8s_smoke | functest | healthcheck | 01:14 | PASS | + +-------------------+------------------+---------------------+------------------+----------------+ + +Testing smoke suite +^^^^^^^^^^^^^^^^^^^ + +Run smoke suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-smoke + +Results shall be displayed as follows:: + + +---------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+---------------+------------------+----------------+ + | k8s_conformance | functest | smoke | 94:26 | PASS | + | xrally_kubernetes | functest | smoke | 13:05 | PASS | + +---------------------------+------------------+---------------+------------------+----------------+ + +Testing security suite +^^^^^^^^^^^^^^^^^^^^^^ + +Run smoke suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-security + +Results shall be displayed as follows:: + + +---------------------------+------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+------------------+------------------+----------------+ + | kube_hunter | functest | security | 00:19 | PASS | + | kube_bench_master | functest | security | 00:02 | PASS | + | kube_bench_node | functest | security | 00:01 | PASS | + +---------------------------+------------------+------------------+------------------+----------------+ + +Testing benchmarking suite +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run benchmarking suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-benchmarking Results shall be displayed as follows:: - +---------------------------------+------------------+--------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +---------------------------------+------------------+--------------+------------------+----------------+ - | cloudify_ims | functest | vnf | 21:25 | PASS | - | orchestra_openims | functest | vnf | 11:02 | FAIL | - | orchestra_clearwaterims | functest | vnf | 09:13 | FAIL | - +---------------------------------+------------------+--------------+------------------+----------------+ + +--------------------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------------+------------------+----------------------+------------------+----------------+ + | xrally_kubernetes_full | functest | benchmarking | 34:16 | PASS | + +--------------------------------+------------------+----------------------+------------------+----------------+ + +Testing cnf suite +^^^^^^^^^^^^^^^^^ +Run cnf suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-cnf + +Results shall be displayed as follows:: + + +-------------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+--------------+------------------+----------------+ + | k8s_vims | functest | cnf | 09:06 | PASS | + | helm_vims | functest | cnf | 08:54 | PASS | + | cnf_conformance | functest | cnf | 02:00 | PASS | + +-------------------------+------------------+--------------+------------------+----------------+ Environment variables ===================== -Several environement variables may be specified: - * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid) +Several environment variables may be specified: + * INSTALLER_IP=<Specific IP Address> * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode> + * NAMESERVER=XXX # if not 8.8.8.8 + * VOLUME_DEVICE_NAME=XXX # if not vdb + * EXTERNAL_NETWORK=XXX # if not first network with router:external=True + * NEW_USER_ROLE=XXX # if not member - -INSTALLER IP may be required by some test cases like SFC or Barometer in order -to access the installer node and the deployment. +INSTALLER_IP is required by Barometer in order to access the installer node and +the deployment. The format for the DEPLOY_SCENARIO env variable can be described as follows: * vim: (os|k8s) = OpenStack or Kubernetes @@ -221,8 +367,8 @@ release note. **NOTE:** The scenario name is mainly used to automatically detect if a test suite is runnable or not (e.g. it will prevent ODL test suite to be -run on 'nosdn' scenarios). If not set, Functest will try to run the default test -cases that might not include SDN controller or a specific feature +run on 'nosdn' scenarios). If not set, Functest will try to run the default +test cases that might not include SDN controller or a specific feature. **NOTE:** An HA scenario means that 3 OpenStack controller nodes are deployed. It does not necessarily mean that the whole system is HA. See @@ -234,9 +380,9 @@ to the Functest Docker Container, using the -e only relevant to Jenkins CI invoked testing and **should not be used** when performing manual test scenarios: - * NODE_NAME = <Test POD Name> - * BUILD_TAG = <Jenkins Build Tag> - * CI_DEBUG = <DebugTraceValue> + * INSTALLER_TYPE=(apex|compass|daisy|fuel) + * NODE_NAME=<Test POD Name> + * BUILD_TAG=<Jenkins Build Tag> where: @@ -256,11 +402,6 @@ where: which are independently pushed to the results database from different Jenkins jobs. DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. - * <DebugTraceValue> = "true" or "false" - Default = "false", if not specified - If "true" is specified, then additional debug trace - text can be sent to the test results file / log files - and also to the standard console output. Openstack credentials @@ -268,7 +409,7 @@ Openstack credentials OpenStack credentials are mandatory and must be provided to Functest. When running the command "functest env prepare", the framework will automatically look for the Openstack credentials file -"/home/opnfv/functest/conf/openstack.creds" and will exit with +"/home/opnfv/functest/conf/env_file" and will exit with error if it is not present or is empty. There are 2 ways to provide that file: @@ -276,7 +417,7 @@ There are 2 ways to provide that file: * by using a Docker volume with -v option when creating the Docker container. This is referred to in docker documentation as "Bind Mounting". See the usage of this parameter in the following chapter. - * or creating manually the file '/home/opnfv/functest/conf/openstack.creds' + * or creating manually the file '/home/opnfv/functest/conf/env_file' inside the running container and pasting the credentials in it. Consult your installer guide for further details. This is however not instructed in this document. @@ -317,194 +458,6 @@ be:: export OS_INSECURE=true -Functest docker container directory structure -============================================= -Inside the Functest docker container, the following directory structure -should now be in place:: - - `-- - |- home - | |-- opnfv - | | `- functest - | | |-- conf - | | `-- results - | `-- repos - | `-- vnfs - |- src - | |-- tempest - | |-- vims-test - | |-- odl_test - | `-- fds - `- usr - `- lib - `- python2.7 - `- site-packages - `- functest - |-- ... - -Underneath the '/home/opnfv/functest' directory, the Functest docker container -includes two main directories: - - * The **conf** directory stores configuration files (e.g. the - OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'), - * the **results** directory stores some temporary result log files - -src and repos directories are used to host third party code used for the tests. - -The structure of functest repo can be described as follows:: - - |-- INFO - |-- LICENSE - |-- api - | `-- apidoc - |-- build.sh - |-- commons - |-- docker - | |-- components - | |-- core - | |-- features - | |-- healthcheck - | |-- smoke - | |-- vnf - | |-- parser - | |-- restapi - | |-- thirdparty-requirements.txt - |-- docs - | |-- com - | |-- images - | |-- release - | | `-- release-notes - | |-- results - | | testing - | | |-- developer - | | `-- user - | | |-- configguide - | | `-- userguide - `-- functest - |-- api - | |-- base.py - | |-- server.py - | |-- urls.py - | |-- common - | | |-- api_utils.py - | | |-- thread.py - | `-- resources - | `-- v1 - | |-- creds.py - | |-- envs.py - | |-- testcases.py - | |-- tiers.py - | |-- tasks.py - | `-- database - | |-- db.py - | `-- v1 - | |-- handlers.py - | |-- models.py - | `-- swagger - |-- ci - │ |-- check_deployment.py - │ |-- config_aarch64_patch.yaml - │ |-- config_functest.yaml - │ |-- config_patch.yaml - │ |-- download_images.sh - │ |-- logging.ini - │ |-- rally_aarch64_patch.conf - │ |-- run_tests.py - │ |-- testcases.yaml - │ |-- tier_builder.py - │ |-- tier_handler.py - |-- cli - │ |-- cli_base.py - │ |-- commands - │ │ |-- cli_env.py - │ │ |-- cli_os.py - │ │ |-- cli_testcase.py - │ │ |-- cli_tier.py - │ |-- functest-complete.sh - |-- core - │ |-- feature.py - │ |-- robotframework.py - │ |-- testcase.py - │ |-- unit.py - │ |-- vnf.py - |-- energy - │ |-- energy.py - |-- opnfv_tests - │ `-- openstack - │ |-- rally - │ |-- refstack_client - │ |-- snaps - │ |-- tempest - │ |-- vping - │ `-- sdn - │ │ `-- odl - │ `-- vnf - │ |-- ims - │ `-- router - |-- tests - │ `-- unit - │ |-- ci - │ |-- cli - │ |-- core - │ |-- energy - │ |-- features - │ |-- odl - │ |-- openstack - │ |-- opnfv_tests - │ |-- test_utils.py - │ |-- utils - │ `-- vnf - |-- utils - | |-- config.py - | |-- constants.py - | |-- decorators.py - | |-- env.py - | |-- functest_utils.py - | |-- functest_vacation.py - | |-- openstack_clean.py - | |-- openstack_tacker.py - | `-- openstack_utils.py - |-- requirements.txt - |-- setup.cfg - |-- setup.py - |-- test-requirements.txt - |-- tox.ini - |-- upper-constraints.txt - - (Note: All *.pyc files removed from above list for brevity...) - -We may distinguish several directories, the first level has 5 directories: - -* **api**: This directory is dedicated to the API (framework) documentations. -* **commons**: This directory is dedicated for storage of traffic profile or - any other test inputs that could be reused by any test project. -* **docker**: This directory includes the needed files and tools to - build the Functest Docker images. -* **docs**: This directory includes documentation: Release Notes, - User Guide, Configuration Guide and Developer Guide. -* **functest**: This directory contains all the code needed to run - functest internal cases and OPNFV onboarded feature or VNF test cases. - -Functest directory has 7 sub-directories, which is located under -/usr/lib/python2.7/site-packages/functest: - * **api**: This directory is dedicated for the internal Functest API. - * **ci**: This directory contains test structure definition files - (e.g <filename>.yaml) and bash shell/python scripts used to - configure and execute Functional tests. The test execution script - can be executed under the control of Jenkins CI jobs. - * **cli**: This directory holds the python based Functest CLI utility - source code, which is based on the Python 'click' framework. - * **core**: This directory holds the python based Functest core - source code. Three abstraction classes have been created to ease - the integration of internal, feature or vnf cases. - * **opnfv_tests**: This directory includes the scripts required by - Functest internal test cases and other feature projects test cases. - * **tests**: This directory includes the functest unit tests. - * **utils**: this directory holds Python source code for some general - purpose helper utilities, which testers can also re-use in their - own test code. See for an example the Openstack helper utility: - 'openstack_utils.py'. - Logs ==== @@ -512,7 +465,7 @@ By default all the logs are put un /home/opnfv/functest/results/functest.log. If you want to have more logs in console, you may edit the logging.ini file manually. Connect on the docker then edit the file located in -/usr/lib/python2.7/site-packages/functest/ci/logging.ini +/usr/lib/python3.8/site-packages/xtesting/ci/logging.ini Change wconsole to console in the desired module to get more traces. @@ -522,7 +475,8 @@ Configuration You may also directly modify the python code or the configuration file (e.g. testcases.yaml used to declare test constraints) under -/usr/lib/python2.7/site-packages/functest +/usr/lib/python3.8/site-packages/xtesting and +/usr/lib/python3.8/site-packages/functest Tips @@ -580,7 +534,7 @@ and credentials are working as expected. Once the credentials are there inside the container, they should be sourced before running any Openstack commands:: - source /home/opnfv/functest/conf/openstack.creds + source /home/opnfv/functest/conf/env_file After this, try to run any OpenStack command to see if you get any output, for instance:: @@ -614,6 +568,8 @@ If the OpenStack command still does not show anything or complains about connectivity issues, it could be due to an incorrect url given to the OS_AUTH_URL environment variable. Check the deployment settings. +.. _`Proxy support`: + Proxy support ------------- If your Jumphost node is operating behind a http proxy, then there are @@ -623,7 +579,7 @@ succeed: #. Initial installation of docker engine First, try following the official Docker documentation for Proxy settings. Some issues were experienced on CentOS 7 based Jumphost. Some tips are documented - in section: `Docker Installation on CentOS behind http proxy`_ + in section: :ref:`Docker Installation on CentOS behind http proxy` below. If that is the case, make sure the resolv.conf and the needed @@ -631,14 +587,14 @@ http_proxy and https_proxy environment variables, as well as the 'no_proxy' environment variable are set correctly:: # Make double sure that the 'no_proxy=...' line in the - # 'openstack.creds' file is commented out first. Otherwise, the + # 'env_file' file is commented out first. Otherwise, the # values set into the 'no_proxy' environment variable below will # be ovewrwritten, each time the command - # 'source ~/functest/conf/openstack.creds' is issued. + # 'source ~/functest/conf/env_file' is issued. cd ~/functest/conf/ - sed -i 's/export no_proxy/#export no_proxy/' openstack.creds - source ./openstack.creds + sed -i 's/export no_proxy/#export no_proxy/' env_file + source ./env_file # Next calculate some IP addresses for which http_proxy # usage should be excluded: @@ -669,7 +625,7 @@ docker container:: Note: In a Jumphost node based on the CentOS family OS, the **nc** commands might not work. You can use the **curl** command instead. - curl http://www.opnfv.org:80 + curl https://www.opnfv.org/ <HTML><HEAD><meta http-equiv="content-type" . @@ -686,6 +642,8 @@ commands might not work. You can use the **curl** command instead. (Ignore the content. If command returns a valid HTML page, it proves the connection.) +.. _`Docker Installation on CentOS behind http proxy`: + Docker Installation on CentOS behind http proxy ----------------------------------------------- This section is applicable for CentOS family OS on Jumphost which @@ -725,11 +683,10 @@ and install the **docker-engine**. The instructions conclude with a work with the above pre-requisite actions. -.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html .. _`dockerdocs`: https://docs.docker.com/ .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/ .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html -.. _`images/CentOS-7-x86_64-GenericCloud.qcow2` http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img -.. _`images/cirros-0.4.0-x86_64-disk.img` https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img -.. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img` https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 +.. _`images/CentOS-7-x86_64-GenericCloud.qcow2`: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 +.. _`images/cirros-0.5.1-x86_64-disk.img`: http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img +.. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img diff --git a/docs/testing/user/configguide/index.rst b/docs/testing/user/configguide/index.rst index fd997344b..e5963435e 100644 --- a/docs/testing/user/configguide/index.rst +++ b/docs/testing/user/configguide/index.rst @@ -1,130 +1,22 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 +.. SPDX-License-Identifier: CC-BY-4.0 +.. _functest-install-guide: *************************** Functest Installation Guide *************************** .. toctree:: - :numbered: :maxdepth: 2 - -Introduction -============ -This document describes how to install and configure Functest in OPNFV. - -High level architecture ------------------------ - -The high level architecture of Functest within OPNFV can be described as -follows:: - - CIMC/Lights+out management Admin Mgmt/API Public Storage Private - PXE - + + + + + + - | | | | | | - | +----------------------------+ | | | | | - | | | | | | | | - +-----+ Jumphost | | | | | | - | | | | | | | | - | | +--------------------+ | | | | | | - | | | | | | | | | | - | | | Tools | +----------------+ | | | - | | | - Rally | | | | | | | - | | | - Robot | | | | | | | - | | | - RefStack | | | | | | | - | | | | |-------------------------+ | | - | | | Testcases | | | | | | | - | | | - VIM | | | | | | | - | | | | | | | | | | - | | | - SDN Controller | | | | | | | - | | | | | | | | | | - | | | - Features | | | | | | | - | | | | | | | | | | - | | | - VNF | | | | | | | - | | | | | | | | | | - | | +--------------------+ | | | | | | - | | Functest Docker + | | | | | - | | | | | | | | - | | | | | | | | - | | | | | | | | - | +----------------------------+ | | | | | - | | | | | | - | +----------------+ | | | | | - | | 1 | | | | | | - +----+ +--------------+-+ | | | | | - | | | 2 | | | | | | - | | | +--------------+-+ | | | | | - | | | | 3 | | | | | | - | | | | +--------------+-+ | | | | | - | | | | | 4 | | | | | | - | +-+ | | +--------------+-+ | | | | | - | | | | | 5 +-------------+ | | | | - | +-+ | | nodes for | | | | | | - | | | | deploying +---------------------+ | | | - | +-+ | OPNFV | | | | | | - | | | +------------------------------+ | | - | +-+ SUT | | | | | | - | | +--------------------------------------+ | - | | | | | | | | - | | +----------------------------------------------+ - | +----------------+ | | | | | - | | | | | | - + + + + + + - SUT = System Under Test - -Note connectivity to management network is not needed for most of the testcases. -But it may be needed for some specific snaps tests. - -All the libraries and dependencies needed by all of the Functest tools are -pre-installed into the Docker images. This allows running Functest on any -platform. - -The automated mechanisms inside the Functest Docker containers will: - - * Prepare the environment according to the System Under Test (SUT) - * Perform the appropriate functional tests - * Push the test results into the OPNFV test result database (optional) - -The OpenStack credentials file must be provided to the container. - -These Docker images can be integrated into CI or deployed independently. - -Please note that the Functest Docker images have been designed for OPNFV, -however, it would be possible to adapt them to any OpenStack based VIM + -controller environment, since most of the test cases are integrated from -upstream communities. - -The functional test cases are described in the Functest User Guide `[2]`_ - -.. include:: ./prerequisites.rst - -.. include:: ./configguide.rst - -.. include:: ./ci.rst - + intro.rst + prerequisites.rst + configguide.rst References ========== -`[1]`_ : Keystone and public end point constraint - -`[2]`_ : Functest User guide - -`[3]`_ : Functest Jenkins jobs - -`[4]`_ : Functest Configuration guide - -`[5]`_ : OPNFV main site - -`[6]`_ : Functest wiki page +`[1]`_ : Functest Jenkins jobs IRC support channel: #opnfv-functest -.. _`[1]`: https://ask.openstack.org/en/question/68144/keystone-unable-to-use-the-public-endpoint/ -.. _`[2]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html -.. _`[3]`: https://git.opnfv.org/releng/tree/jjb/functest/functest-daily-jobs.yml -.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html -.. _`[5]`: http://www.opnfv.org -.. _`[6]`: https://wiki.opnfv.org/functest +.. _`[1]`: https://github.com/opnfv/releng/blob/master/jjb/functest/functest.yaml diff --git a/docs/testing/user/configguide/intro.rst b/docs/testing/user/configguide/intro.rst new file mode 100644 index 000000000..02329995c --- /dev/null +++ b/docs/testing/user/configguide/intro.rst @@ -0,0 +1,86 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Introduction +============ +This document describes how to install and configure Functest in OPNFV. + +High level architecture +----------------------- + +The high level architecture of Functest within OPNFV can be described as +follows:: + + CIMC/Lights+out management Admin Mgmt/API Public Storage Private + PXE + + + + + + + + | | | | | | + | +----------------------------+ | | | | | + | | | | | | | | + +-----+ Jumphost | | | | | | + | | | | | | | | + | | +--------------------+ | | | | | | + | | | | | | | | | | + | | | Tools | +----------------+ | | | + | | | - Rally | | | | | | | + | | | - Robot | | | | | | | + | | | - RefStack | | | | | | | + | | | | |-------------------------+ | | + | | | Testcases | | | | | | | + | | | - VIM | | | | | | | + | | | | | | | | | | + | | | - SDN Controller | | | | | | | + | | | | | | | | | | + | | | - Features | | | | | | | + | | | | | | | | | | + | | | - VNF | | | | | | | + | | | | | | | | | | + | | +--------------------+ | | | | | | + | | Functest Docker + | | | | | + | | | | | | | | + | | | | | | | | + | | | | | | | | + | +----------------------------+ | | | | | + | | | | | | + | +----------------+ | | | | | + | | 1 | | | | | | + +----+ +--------------+-+ | | | | | + | | | 2 | | | | | | + | | | +--------------+-+ | | | | | + | | | | 3 | | | | | | + | | | | +--------------+-+ | | | | | + | | | | | 4 | | | | | | + | +-+ | | +--------------+-+ | | | | | + | | | | | 5 +-------------+ | | | | + | +-+ | | nodes for | | | | | | + | | | | deploying +---------------------+ | | | + | +-+ | OPNFV | | | | | | + | | | +------------------------------+ | | + | +-+ SUT | | | | | | + | | +--------------------------------------+ | + | | | | | | | | + | | +----------------------------------------------+ + | +----------------+ | | | | | + | | | | | | + + + + + + + + SUT = System Under Test + +All the libraries and dependencies needed by all of the Functest tools are +pre-installed into the Docker images. This allows running Functest on any +platform. + +The automated mechanisms inside the Functest Docker containers will: + + * Prepare the environment according to the System Under Test (SUT) + * Perform the appropriate functional tests + * Push the test results into the OPNFV test result database (optional) + +The OpenStack credentials file must be provided to the container. + +These Docker images can be integrated into CI or deployed independently. + +Please note that the Functest Docker images have been designed for OPNFV, +however, it would be possible to adapt them to any OpenStack based VIM + +controller environment, since most of the test cases are integrated from +upstream communities. + +The functional test cases are described in the :ref:`functest-userguide` diff --git a/docs/testing/user/configguide/prerequisites.rst b/docs/testing/user/configguide/prerequisites.rst index 8289803bb..dab4a9e6b 100644 --- a/docs/testing/user/configguide/prerequisites.rst +++ b/docs/testing/user/configguide/prerequisites.rst @@ -1,7 +1,9 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + Prerequisites ============= The OPNFV deployment is out of the scope of this document but it can be -found in http://docs.opnfv.org. +found in https://docs.opnfv.org/en/stable-hunter/. The OPNFV platform is considered as the SUT in this document. Several prerequisites are needed for Functest: @@ -12,12 +14,6 @@ Several prerequisites are needed for Functest: #. An admin/management network created on the SUT #. Connectivity from the Jumphost to the SUT public/external network -Some specific SNAPS tests may require a connectivity from the Jumphost to the -SUT admin/management network but most of the test cases do not. This requirement -can be changed by overriding the 'interface' attribute (OS_INTERFACE) value -to 'public' in the credentials file. Another means to circumvent this issue -would be to change the 'snaps.use_keystone' value from True to False. - WARNING: Connectivity from Jumphost is essential and it is of paramount importance to make sure it is working before even considering to install and run Functest. Make also sure you understand how your networking is @@ -29,7 +25,7 @@ deployment has been triggered previously, but it could be any server with proper connectivity to the SUT. NOTE: If your Jumphost is operating behind a company http proxy and/or -firewall, please consult first the section `Proxy Support`_, towards +firewall, please consult first the section :ref:`Proxy support`, towards the end of this document. The section details some tips/tricks which *may* be of help in a proxified environment. @@ -44,8 +40,8 @@ commands below are offered as a short reference. *Tip:* For running docker containers behind the proxy, you need first some extra configuration which is described in section -`Docker Installation on CentOS behind http proxy`_. You should follow -that section before installing the docker engine. +:ref:`Docker Installation on CentOS behind http proxy`. You should follow that +section before installing the docker engine. Docker installation needs to be done as root user. You may use other userid's to create and run the actual containers later if so desired. @@ -96,7 +92,4 @@ should thus be known. Ensure you can reach each node in the SUT, from the Jumphost using the 'ping' command using the respective IP address on the public/external network for each node in the SUT. The details of how to determine the needed IP addresses for each node in the SUT may vary according -to the used installer and are therefore ommitted here. - -.. _`[1]`: https://ask.openstack.org/en/question/68144/keystone-unable-to-use-the-public-endpoint/ -.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html +to the used installer and are therefore omitted here. |