diff options
Diffstat (limited to 'docs/testing/user/configguide')
| -rw-r--r-- | docs/testing/user/configguide/ci.rst | 50 | ||||
| -rw-r--r-- | docs/testing/user/configguide/configguide.rst | 844 | ||||
| -rw-r--r-- | docs/testing/user/configguide/index.rst | 36 | ||||
| -rw-r--r-- | docs/testing/user/configguide/prerequisites.rst | 28 |
4 files changed, 383 insertions, 575 deletions
diff --git a/docs/testing/user/configguide/ci.rst b/docs/testing/user/configguide/ci.rst index 384bc34e5..ada803c58 100644 --- a/docs/testing/user/configguide/ci.rst +++ b/docs/testing/user/configguide/ci.rst @@ -1,50 +1,14 @@ Integration in CI ================= -In CI we use the Docker image and execute the appropriate commands within the +In CI we use the Docker images and execute the appropriate commands within the container from Jenkins. -Docker creation in set-functest-env builder `[3]`_:: - - envs="-e INSTALLER_TYPE=${INSTALLER_TYPE} -e INSTALLER_IP=${INSTALLER_IP} -e NODE_NAME=${NODE_NAME}" - [...] - docker pull opnfv/functest:$DOCKER_TAG >/dev/null - cmd="sudo docker run -id ${envs} ${volumes} ${custom_params} ${TESTCASE_OPTIONS} opnfv/functest:${DOCKER_TAG} /bin/bash" - echo "Functest: Running docker run command: ${cmd}" - ${cmd} >${redirect} - sleep 5 - container_id=$(docker ps | grep "opnfv/functest:${DOCKER_TAG}" | awk '{print $1}' | head -1) - echo "Container ID=${container_id}" - if [ -z ${container_id} ]; then - echo "Cannot find opnfv/functest container ID ${container_id}. Please check if it is existing." - docker ps -a - exit 1 - fi - echo "Starting the container: docker start ${container_id}" - docker start ${container_id} - sleep 5 - docker ps >${redirect} - if [ $(docker ps | grep "opnfv/functest:${DOCKER_TAG}" | wc -l) == 0 ]; then - echo "The container opnfv/functest with ID=${container_id} has not been properly started. Exiting..." - exit 1 - fi - - cmd="python ${FUNCTEST_REPO_DIR}/functest/ci/prepare_env.py start" - echo "Executing command inside the docker: ${cmd}" - docker exec ${container_id} ${cmd} - - -Test execution in functest-all builder `[3]`_:: - - branch=${GIT_BRANCH##*/} - echo "Functest: run $FUNCTEST_SUITE_NAME on branch ${branch}" - cmd="functest testcase run $FUNCTEST_SUITE_NAME" - fi - container_id=$(docker ps -a | grep opnfv/functest | awk '{print $1}' | head -1) - docker exec $container_id $cmd - ret_value=$? - exit $ret_value - -Docker clean in functest-cleanup builder `[3]`_ calling docker rm and docker rmi +4 steps have been defined:: + * functest-cleanup: clean existing functest dockers on the jumphost + * set-functest-env-alpine: prepare en environement files + * functest-daily: run dockers opnfv/functest-* (healthcheck, smoke, features, vnf) + * functest-store-results: push logs to artifacts +See `[3]`_ for details. .. _`[3]`: https://git.opnfv.org/releng/tree/jjb/functest/functest-daily-jobs.yml diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index d19939afe..0a260ad0a 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -1,346 +1,366 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. SPDX-License-Identifier: CC-BY-4.0 -Installation and configuration (Ubuntu) -======================================= +Installation and configuration +============================== -The historical docker file is based on Ubuntu. It has been maintained for -Euphrates. +Alpine containers have been introduced in Euphrates. +Alpine allows Functest testing in several very light containers and thanks to +the refactoring on dependency management should allow the creation of light and +fully customized docker files. -Pulling the Docker image ------------------------- -Pull the Functest Docker image ('opnfv/functest') from the public -dockerhub registry under the OPNFV account: [̀`dockerhub`_], with the -following docker command:: - - docker pull opnfv/functest:<TagIdentifier> - -where <TagIdentifier> identifies a release of the Functest docker -container image in the public Dockerhub registry. There are many tags -created automatically by the CI mechanisms, and you must ensure you -pull an image with the **correct tag** to match the OPNFV software -release installed in your environment. All available tagged images can -be seen from location [FunctestDockerTags_]. For example, when running -on the first official release of the OPNFV Danube system platform, -tag "danube.1.0" is needed. For the second and third releases, the tag -"danube.2.0" and "danube.3.0" can be used respectively. -Pulling other tags might cause some problems while running the tests. -Docker images pulled without a tag specifier bear the implicitly -assigned label "latest". If you need to specifically pull the latest -Functest docker image, then omit the tag argument:: - - docker pull opnfv/functest - -After pulling the Docker image, check that it is available with the -following docker command:: - - [functester@jumphost ~]$ docker images - REPOSITORY TAG IMAGE ID CREATED SIZE - opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.321 GB - opnfv/functest danube.2.0 d2c174a91911 7 minutes ago 1.471 GB - opnfv/functest danube.1.0 13fa54a1b238 4 weeks ago 1.29 GB +It is still possible to use the monolithic functest opnfv/functest especially +for tests on Aarch64 architecture. + +Functest Dockers +---------------- +Docker files are available on the dockerhub: + + * opnfv/functest-core + * opnfv/functest-healthcheck + * opnfv/functest-smoke + * opnfv/functest-features + * opnfv/functest-components + * opnfv/functest-vnf + * opnfv/functest-parser + * opnfv/functest-restapi + +By default, we use the docker tag latest, but you may pull a tagged docker +image. The Functest docker container environment can -in principle- be also used with non-OPNFV official installers (e.g. 'devstack'), with the **disclaimer** that support for such environments is outside of the scope and responsibility of the OPNFV project. -Please note that alpine dockers have been introduced in Euphrates. See alpine -section for details. -Accessing the Openstack credentials ------------------------------------ -OpenStack credentials are mandatory and must be provided to Functest. -When running the command "functest env prepare", the framework will -automatically look for the Openstack credentials file -"/home/opnfv/functest/conf/openstack.creds" and will exit with -error if it is not present or it is empty. +Preparing your environment +-------------------------- -There are 2 ways to provide that file: +cat env:: - * by using a Docker volume with -v option when creating the Docker container. - This is referred to in docker documentation as "Bind Mounting". - See the usage of this parameter in the following chapter. - * or creating manually the file '/home/opnfv/functest/conf/openstack.creds' - inside the running container and pasting the credentials in it. Consult - your installer guide for further details. This is however not - instructed in this document. + INSTALLER_TYPE=XXX + INSTALLER_IP=XXX + EXTERNAL_NETWORK=XXX + DEPLOY_SCENARIO=XXX -There is a default environment variable in the Functest container **$creds** -that points to the credentials absolute path to help the user with this task. +See section on environment variables for details. -In proxified environment you may need to change the credentials file. -There are some tips in chapter: `Proxy support`_ +cat openstack.creds:: + + export OS_AUTH_URL=XXX + export OS_USER_DOMAIN_NAME=XXX + export OS_PROJECT_DOMAIN_NAME=XXX + export OS_USERNAME=XXX + export OS_TENANT_NAME=XXX + export OS_PROJECT_NAME=XXX + export OS_PASSWORD=XXX + export OS_VOLUME_API_VERSION=XXX + export OS_IDENTITY_API_VERSION=XXX + export OS_IMAGE_API_VERSION=XXX + +See section on OpenStack credentials for details. -Functest Docker parameters +Create a directory for the different images (included as volume):: + + mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* + + images/CentOS-7-aarch64-GenericCloud.qcow2 + images/CentOS-7-aarch64-GenericCloud.qcow2.xz + images/CentOS-7-x86_64-GenericCloud.qcow2 + images/cirros-0.3.5-x86_64-disk.img + images/cirros-0.3.5-x86_64-lxc.tar.gz + images/cirros-d161201-aarch64-disk.img + images/cirros-d161201-aarch64-initramfs + images/cirros-d161201-aarch64-kernel + images/cloudify-manager-premium-4.0.1.qcow2 + images/img + images/trusty-server-cloudimg-amd64-disk1.img + images/ubuntu-14.04-server-cloudimg-amd64-disk1.img + images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img + images/ubuntu-16.04-server-cloudimg-amd64-disk1.img + images/vyos-1.1.7.img + + +Testing healthcheck suite -------------------------- -This chapter explains how to run a container for executing functest -test suites. Numbered list below explains some details of the -recommended parameters for invoking docker container - #. It is a good practice to assign a precise container name through - the **--name** option. +Run healthcheck suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-healthcheck + +Results shall be displayed as follows:: + + +----------------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +----------------------------+------------------+---------------------+------------------+----------------+ + | connection_check | functest | healthcheck | 00:02 | PASS | + | api_check | functest | healthcheck | 04:57 | PASS | + | snaps_health_check | functest | healthcheck | 00:51 | PASS | + +----------------------------+------------------+---------------------+------------------+----------------+ + +Testing smoke suite +------------------- + +Run smoke suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-smoke + +Results shall be displayed as follows:: + + +------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +------------------------------+------------------+---------------+------------------+----------------+ + | vping_ssh | functest | smoke | 01:19 | PASS | + | vping_userdata | functest | smoke | 01:56 | PASS | + | tempest_smoke_serial | functest | smoke | 26:30 | PASS | + | rally_sanity | functest | smoke | 19:42 | PASS | + | refstack_defcore | functest | smoke | 22:00 | PASS | + | snaps_smoke | functest | smoke | 41:14 | PASS | + | odl | functest | smoke | 00:16 | PASS | + | odl_netvirt | functest | smoke | 00:00 | SKIP | + | fds | functest | smoke | 00:00 | SKIP | + +------------------------------+------------------+---------------+------------------+----------------+ + Note: if the scenario does not support some tests, they are indicated as SKIP. + See User guide for details. + +Testing features suite +---------------------- + +Run features suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-features + +Results shall be displayed as follows:: + + +---------------------------+--------------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+--------------------------+------------------+------------------+----------------+ + | promise | promise | features | 00:00 | SKIP | + | bgpvpn | sdnvpn | features | 00:00 | SKIP | + | security_scan | securityscanning | features | 00:00 | SKIP | + | functest-odl-sfc | sfc | features | 00:00 | SKIP | + | domino-multinode | domino | features | 00:00 | SKIP | + | barometercollectd | barometer | features | 00:00 | SKIP | + +---------------------------+--------------------------+------------------+------------------+----------------+ + Note: if the scenario does not support some tests, they are indicated as SKIP. + See User guide for details. + +Testing components suite +------------------------ - #. Assign parameter for installer type:: +Run components suite:: - -e "INSTALLER_TYPE=<type>" - # Use one of following apex, compass, fuel or joid + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-components - #. Functest needs to know the IP of some installers:: +Results shall be displayed as follows:: - -e "INSTALLER_IP=<Specific IP Address>" + +-------------------------------+------------------+--------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------------+------------------+--------------------+------------------+----------------+ + | tempest_full_parallel | functest | components | 102:48 | PASS | + | rally_full | functest | components | 160:58 | PASS | + | tempest_custom | functest | components | 00:00 | SKIP | + +-------------------------------+------------------+--------------------+------------------+----------------+ - These two env variables are useful extract some information - from the deployment. However, for some test cases like - SFC or Barometer they are mandatory since the tests - need to access the installer node and the deployment. +Testing vnf suite +----------------- - #. Credentials for accessing the Openstack. - Most convenient way of passing them to container is by having a - local copy of the credentials file in Jumphost and then using the - **-v** option. In the example we have local file by the name of - "overcloudrc" and we are using that as an argument:: +Run vnf suite:: - -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds +sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-vnf - The credentials file needs to exist in the Docker container - under the path: '/home/opnfv/functest/conf/openstack.creds'. +Results shall be displayed as follows:: - **WARNING:** If you are using the Joid installer, you must pass the - credentials using the **-v** option: - -v /var/lib/jenkins/admin-openrc:/home/opnfv/functest/conf/openstack.creds. - See the section `Accessing the Openstack credentials`_ above. + +---------------------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------------+------------------+--------------+------------------+----------------+ + | cloudify_ims | functest | vnf | 21:25 | PASS | + | orchestra_openims | functest | vnf | 11:02 | FAIL | + | orchestra_clearwaterims | functest | vnf | 09:13 | FAIL | + | vyos_vrouter | functest | vnf | 00:00 | SKIP | + +---------------------------------+------------------+--------------+------------------+----------------+ - #. Passing deployment scenario - When running Functest against any of the supported OPNFV scenarios, - it is recommended to include also the environment variable - **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable - is passed with the format:: - -e "DEPLOY_SCENARIO=os-<controller>-<nfv_feature>-<ha_mode>" - where: - os = OpenStack (No other VIM choices currently available) - controller is one of ( nosdn | odl_l2 | odl_l3 ) - nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature ) - If several features are pertinent then use the underscore - character '_' to separate each feature (e.g. ovs_kvm) - 'nofeature' indicates no NFV feature is deployed - ha_mode (high availability) is one of ( ha | noha ) +Environment variables +===================== - **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are - supported. The name passed in to the Functest Docker container - must match the scenario used when the actual OPNFV platform was - deployed. See release note to see the list of supported scenarios. +Several environement variables may be specified: + * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid|osa) + * INSTALLER_IP=<Specific IP Address> + * DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode> - **NOTE:** The scenario name is mainly used to automatically detect - if a test suite is runnable or not (e.g. it will prevent ONOS test suite - to be run on ODL scenarios). If not set, Functest will try to run the - default test cases that might not include SDN controller or a specific - feature - **NOTE:** A HA scenario means that 3 OpenStack controller nodes are - deployed. It does not necessarily mean that the whole system is HA. See - installer release notes for details. +INSTALLER IP may be required by some test cases like SFC or Barometer in order +to access the installer node and the deployment. +The format for the DEPLOY_SCENARIO env variable can be described as follows: + * vim: (os|k8s) = OpenStack or Kubernetes + * controller is one of ( nosdn | odl ) + * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature ) + * ha_mode (high availability) is one of ( ha | noha ) -Putting all above together, when using installer 'fuel' and an invented -INSTALLER_IP of '10.20.0.2', the recommended command to create the -Functest Docker container is as follows:: +If several features are pertinent then use the underscore character '_' to +separate each feature (e.g. ovs_kvm) 'nofeature' indicates no OPNFV feature is +deployed - docker run --name "FunctestContainer" -it \ - -e "INSTALLER_IP=10.20.0.2" \ - -e "INSTALLER_TYPE=fuel" \ - -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \ - -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \ - opnfv/functest /bin/bash +The list of supported scenarios per release/installer is indicated in the +release note. -After the *run* command, a new prompt appears which means that we are inside -the container and ready to move to the next step. +**NOTE:** The scenario name is mainly used to automatically detect +if a test suite is runnable or not (e.g. it will prevent ONOS test suite to be +run on ODL scenarios). If not set, Functest will try to run the default test +cases that might not include SDN controller or a specific +feature -For tips on how to set up container with installer Apex, see chapter -`Apex Installer Tips`_. +**NOTE:** A HA scenario means that 3 OpenStack controller nodes are +deployed. It does not necessarily mean that the whole system is HA. See +installer release notes for details. Finally, three additional environment variables can also be passed in to the Functest Docker Container, using the -e -"<EnvironmentVariable>=<Value>" mechanism. The first two of these are +"<EnvironmentVariable>=<Value>" mechanism. The first two parameters are only relevant to Jenkins CI invoked testing and **should not be used** -when performing manual test scenarios:: - - -e "NODE_NAME=<Test POD Name>" \ - -e "BUILD_TAG=<Jenkins Build Tag>" \ - -e "CI_DEBUG=<DebugTraceValue>" - where: - <Test POD Name> = Symbolic name of the POD where the tests are run. - Visible in test results files, which are stored - to the database. This option is only used when - tests are activated under Jenkins CI control. - It indicates the POD/hardware where the test has - been run. If not specified, then the POD name is - defined as "Unknown" by default. - DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. - <Jenkins Build tag> = Symbolic name of the Jenkins Build Job. - Visible in test results files, which are stored - to the database. This option is only set when - tests are activated under Jenkins CI control. - It enables the correlation of test results, - which - are independently pushed to the results database - from different Jenkins jobs. - DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. - <DebugTraceValue> = "true" or "false" - Default = "false", if not specified - If "true" is specified, then additional debug trace - text can be sent to the test results file / log files - and also to the standard console output. - -Installer Tips --------------- - -Apex Installer Tips -^^^^^^^^^^^^^^^^^^^ -Some specific tips are useful for the Apex Installer case. If not using -Apex Installer; ignore this section. - -In case of Triple-O based installer (like Apex) the docker container -needs to connect to the installer VM, so it is then required that some -known SSH keys are present in docker container. Since the Jumphost root -SSH keys are already known, easiest way is to use those using the -'Bind mount' method. See below for sample parameter:: - - -v /root/.ssh/id_rsa:/root/.ssh/id_rsa - - NOTE: You need the "sudo" when creating the container to access root - users ssh credentials even the docker command itself might not - require that. - -HINT! In case of Triple-O installers you can find value for the -INSTALLER_IP parameter by executing command and note the returned IP -address:: - - inst=$(sudo virsh list | grep -iEo "undercloud|instack") - sudo virsh domifaddr ${inst} - - NOTE: In releases prior to Colorado, the name 'instack' was - used. Currently the name 'undercloud' is used. - -You can copy the credentials file from the "stack" users home directory -in installer VM to Jumphost. Please check the correct IP from the -command above. In the example below we are using invented IP address -"192.168.122.89":: - - scp stack@192.168.122.89:overcloudrc . - -Here is an example of the full docker command invocation for an Apex -installed system, using latest Functest docker container, for -illustration purposes:: - - sudo docker run -it --name "ApexFuncTestODL" \ - -e "INSTALLER_IP=192.168.122.89" \ - -e "INSTALLER_TYPE=apex" \ - -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \ - -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \ - -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \ - opnfv/functest /bin/bash - -Compass installer local development env usage Tips -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In the compass-functest local test case check and development environment, -in order to get openstack service inside the functest container, some -parameters should be configured during container creation, which are -hard to guess for freshman. This section will provide the guideline, the -parameters values are defaults here, which should be adjusted according -to the settings, the complete steps are given here so as not to appear -too abruptly. - -1, Pull Functest docker image from public dockerhub:: - - docker pull opnfv/functest:<Tag> - -<Tag> here can be "brahmaputra.1.0", "colorado.1.0", etc. -Tag omitted means the latest docker image:: - - docker pull opnfv/functest - -2, Functest Docker container creation - -To make a file used for the environment, such as 'functest-docker-env':: - - CINDER_ENDPOINT_TYPE=publicURL - NOVA_ENDPOINT_TYPE=publicURL - OS_ENDPOINT_TYPE=publicURL - OS_INTERFACE=publicURL - OS_USERNAME=admin - OS_PASSWORD='990232e0885da343ac805523322d' - OS_PROJECT_NAME=admin - OS_TENANT_NAME=admin - OS_AUTH_URL=https://192.16.1.222:5000/v3 - OS_NO_CACHE=1 - OS_USER_DOMAIN_NAME=Default - OS_PROJECT_DOMAIN_NAME=Default - OS_REGION_NAME=RegionOne - OS_IDENTITY_API_VERSION=3 - OS_AUTH_VERSION=3 +when performing manual test scenarios: + + * NODE_NAME = <Test POD Name> + * BUILD_TAG = <Jenkins Build Tag> + * CI_DEBUG = <DebugTraceValue> + +where: + + * <Test POD Name> = Symbolic name of the POD where the tests are run. + Visible in test results files, which are stored + to the database. This option is only used when + tests are activated under Jenkins CI control. + It indicates the POD/hardware where the test has + been run. If not specified, then the POD name is + defined as "Unknown" by default. + DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. + * <Jenkins Build tag> = Symbolic name of the Jenkins Build Job. + Visible in test results files, which are stored + to the database. This option is only set when + tests are activated under Jenkins CI control. + It enables the correlation of test results, + which + are independently pushed to the results database + from different Jenkins jobs. + DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. + * <DebugTraceValue> = "true" or "false" + Default = "false", if not specified + If "true" is specified, then additional debug trace + text can be sent to the test results file / log files + and also to the standard console output. + + +Openstack credentials +===================== +OpenStack credentials are mandatory and must be provided to Functest. +When running the command "functest env prepare", the framework will +automatically look for the Openstack credentials file +"/home/opnfv/functest/conf/openstack.creds" and will exit with +error if it is not present or is empty. +There are 2 ways to provide that file: -Note: please adjust the content according to the environment, such as -'TENANT_ID' maybe used for some special cases. + * by using a Docker volume with -v option when creating the Docker container. + This is referred to in docker documentation as "Bind Mounting". + See the usage of this parameter in the following chapter. + * or creating manually the file '/home/opnfv/functest/conf/openstack.creds' + inside the running container and pasting the credentials in it. Consult + your installer guide for further details. This is however not + instructed in this document. -Then to create the Functest docker:: +There is a default environment variable in the Functest container **$creds** +that points to the credentials absolute path to help the user with this task. - docker run --privileged=true --rm -t \ - --env-file functest-docker-env \ - --name <Functest_Container_Name> \ - opnfv/functest:<Tag> /bin/bash +In proxified environment you may need to change the credentials file. +There are some tips in chapter: `Proxy support`_ -3, To attach Functest container +SSL Support +----------- +If you need to connect to a server that is TLS-enabled (the auth URL +begins with "https") and it uses a certificate from a private CA or a +self-signed certificate, then you will need to specify the path to an +appropriate CA certificate to use, to validate the server certificate +with the environment variable OS_CACERT:: + + echo $OS_CACERT + /etc/ssl/certs/ca.crt + +However, this certificate does not exist in the container by default. +It has to be copied manually from the OpenStack deployment. This can be +done in 2 ways: -Before trying to attach the Functest container, the status can be checked by:: + #. Create manually that file and copy the contents from the OpenStack + controller. + #. (Recommended) Add the file using a Docker volume when starting the + container:: - docker ps -a + -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert -to attach the 'Up' status Functest container and start bash mode:: +You might need to export OS_CACERT environment variable inside the +container:: - docker exec -it <Functest_Container_Name> bash + export OS_CACERT=/etc/ssl/certs/ca.crt -4, Functest environment preparation and check +Certificate verification can be turned off using OS_INSECURE=true. For +example, Fuel uses self-signed cacerts by default, so an pre step would +be:: -To see the Section below `Preparing the Functest environment`_. + export OS_INSECURE=true Functest docker container directory structure ---------------------------------------------- +============================================= Inside the Functest docker container, the following directory structure should now be in place:: - `-- home - | `-- opnfv - | |-- functest - | | |-- conf - | | |-- data - | | |-- images - | | `-- results - | `-- repos - | |-- doctor + `-- + |- home + | |-- opnfv + | | `- functest + | | |-- conf + | | `-- results + | `-- repos | `-- vnfs - -- src - |-- tempest - |-- vims-test - |-- odl_test - `-- fds - -Underneath the '/home/opnfv/' directory, the Functest docker container + |- src + | |-- tempest + | |-- vims-test + | |-- odl_test + | `-- fds + `- usr + `- lib + `- python2.7 + `- site-packages + `- functest + |-- ... + +Underneath the '/home/opnfv/functest' directory, the Functest docker container includes two main directories: - * The **functest** directory stores configuration files (e.g. the + * The **conf** directory stores configuration files (e.g. the OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'), - the **data** directory stores a 'cirros' test image used in some - functional tests and the **results** directory stores some temporary - result log files - * The **repos** directory holds various repositories. The directories - are used for the installation of the needed tooling (e.g. rally) or - for the retrieval of feature projects scenarios (e.g. promise) + * the **results** directory stores some temporary result log files + +src and repos directories are used to host third party code used for the tests. -The structure under the **functest** repository can be described as -follows:: +The functest code is under /usr/lib/python2.7/site-packages/functest +The structure can be described as follows:: |-- INFO |-- LICENSE @@ -465,8 +485,7 @@ follows:: We may distinguish several directories, the first level has 5 directories: -* **api**: This directory is dedicated for the internal Functest API and the API - (framework) documentations. +* **api**: This directory is dedicated to the API (framework) documentations. * **commons**: This directory is dedicated for storage of traffic profile or any other test inputs that could be reused by any test project. @@ -477,7 +496,9 @@ We may distinguish several directories, the first level has 5 directories: * **functest**: This directory contains all the code needed to run functest internal cases and OPNFV onboarded feature or VNF test cases. -Functest directory has 6 directories: +Functest directory has 7 sub-directories: + * **api**: This directory is dedicated for the internal Functest API and the + API (framework) documentations. * **ci**: This directory contains test structure definition files (e.g <filename>.yaml) and bash shell/python scripts used to configure and execute Functional tests. The test execution script @@ -495,8 +516,31 @@ Functest directory has 6 directories: own test code. See for an example the Openstack helper utility: 'openstack_utils.py'. -Useful Docker commands ----------------------- + +Logs +==== +By default all the logs are put un /home/opnfv/functest/results/functest.log. +If you want to have more logs in console, you may edit the logging.ini file +manually. +Connect on the docker then edit the file located in +/usr/lib/python2.7/site-packages/functest/ci/logging.ini + +Change wconsole to console in the desired module to get more traces. + + +Configuration +============= + +You may also directly modify the python code or the configuration file (e.g. +testcases.yaml used to declare test constraints) under +/usr/lib/python2.7/site-packages/functest + + +Tips +==== + +Docker +------ When typing **exit** in the container prompt, this will cause exiting the container and probably stopping it. When stopping a running Docker container all the changes will be lost, there is a keyboard shortcut @@ -538,116 +582,6 @@ destroy it:: Check the Docker documentation [`dockerdocs`_] for more information. -Preparing the Functest environment ----------------------------------- -Once the Functest docker container is up and running, the required -Functest environment needs to be prepared. A custom built **functest** -CLI utility is available to perform the needed environment preparation -action. Once the environment is prepared, the **functest** CLI utility -can be used to run different functional tests. The usage of the -**functest** CLI utility to run tests is described further in the -`Functest User Guide`_ - -Prior to commencing the Functest environment preparation, we can check -the initial status of the environment. Issue the **functest env status** -command at the prompt:: - - functest env status - Functest environment is not installed. - - Note: When the Functest environment is prepared, the command will - return the status: "Functest environment ready to run tests." - -To prepare the Functest docker container for test case execution, issue -the **functest env prepare** command at the prompt:: - - functest env prepare - -This script will make sure that the requirements to run the tests are -met and will install the needed libraries and tools by all Functest -test cases. It should be run only once every time the Functest docker -container is started from scratch. If you try to run this command, on -an already prepared environment, you will be prompted whether you really -want to continue or not:: - - functest env prepare - It seems that the environment has been already prepared. - Do you want to do it again? [y|n] - - (Type 'n' to abort the request, or 'y' to repeat the - environment preparation) - - -To list some basic information about an already prepared Functest -docker container environment, issue the **functest env show** at the -prompt:: - - functest env show - +======================================================+ - | Functest Environment info | - +======================================================+ - | INSTALLER: apex, 192.168.122.89 | - | SCENARIO: os-odl_l2-nofeature-ha | - | POD: localhost | - | GIT BRANCH: master | - | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 | - | DEBUG FLAG: false | - +------------------------------------------------------+ - | STATUS: ready | - +------------------------------------------------------+ - - Where: - - INSTALLER: Displays the INSTALLER_TYPE value - - here = "apex" - and the INSTALLER_IP value - - here = "192.168.122.89" - SCENARIO: Displays the DEPLOY_SCENARIO value - - here = "os-odl_l2-nofeature-ha" - POD: Displays the value passed in NODE_NAME - - here = "localhost" - GIT BRANCH: Displays the git branch of the OPNFV Functest - project repository included in the Functest - Docker Container. - - here = "master" - (In first official colorado release - would be "colorado.1.0") - GIT HASH: Displays the git hash of the OPNFV Functest - project repository included in the Functest - Docker Container. - - here = "5bf1647dec6860464eeb082b2875798f0759aa91" - DEBUG FLAG: Displays the CI_DEBUG value - - here = "false" - - NOTE: In Jenkins CI runs, an additional item "BUILD TAG" - would also be listed. The value is set by Jenkins CI. - -Finally, the **functest** CLI has a **--help** options: - -Some examples:: - - functest --help Usage: functest [OPTIONS] COMMAND [ARGS]... - - Options: - --version Show the version and exit. - -h, --help Show this message and exit. - - Commands: - env - openstack - testcase - tier - - functest env --help - Usage: functest env [OPTIONS] COMMAND [ARGS]... - - Options: - -h, --help Show this message and exit. - - Commands: - prepare Prepares the Functest environment. - show Shows information about the current... - status Checks if the Functest environment is ready... Checking Openstack and credentials ---------------------------------- @@ -673,53 +607,24 @@ This command must show a set of environment variables starting with *OS_*, for example:: OS_REGION_NAME=RegionOne - OS_DEFAULT_DOMAIN=default + OS_USER_DOMAIN_NAME=Default OS_PROJECT_NAME=admin - OS_PASSWORD=admin - OS_AUTH_STRATEGY=keystone - OS_AUTH_URL=http://172.30.10.3:5000/v2.0 + OS_AUTH_VERSION=3 + OS_IDENTITY_API_VERSION=3 + OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc + OS_AUTH_URL=http://10.1.0.9:5000/v3 OS_USERNAME=admin OS_TENANT_NAME=admin OS_ENDPOINT_TYPE=internalURL - OS_NO_CACHE=true + OS_INTERFACE=internalURL + OS_NO_CACHE=1 + OS_PROJECT_DOMAIN_NAME=Default + If the OpenStack command still does not show anything or complains about connectivity issues, it could be due to an incorrect url given to the OS_AUTH_URL environment variable. Check the deployment settings. -SSL Support ------------ -If you need to connect to a server that is TLS-enabled (the auth URL -begins with "https") and it uses a certificate from a private CA or a -self-signed certificate, then you will need to specify the path to an -appropriate CA certificate to use, to validate the server certificate -with the environment variable OS_CACERT:: - - echo $OS_CACERT - /etc/ssl/certs/ca.crt - -However, this certificate does not exist in the container by default. -It has to be copied manually from the OpenStack deployment. This can be -done in 2 ways: - - #. Create manually that file and copy the contents from the OpenStack - controller. - #. (Recommended) Add the file using a Docker volume when starting the - container:: - - -v <path_to_your_cert_file>:/etc/ssl/certs/ca.cert - -You might need to export OS_CACERT environment variable inside the -container:: - - export OS_CACERT=/etc/ssl/certs/ca.crt - -Certificate verification can be turned off using OS_INSECURE=true. For -example, Fuel uses self-signed cacerts by default, so an pre step would -be:: - - export OS_INSECURE=true - Proxy support ------------- If your Jumphost node is operating behind a http proxy, then there are @@ -732,12 +637,6 @@ succeed: in section: `Docker Installation on CentOS behind http proxy`_ below. - #. Execution of the Functest environment preparation inside the - created docker container Functest needs internet access to - download some resources for some test cases. This might not - work properly if the Jumphost is connecting to internet - through a http Proxy. - If that is the case, make sure the resolv.conf and the needed http_proxy and https_proxy environment variables, as well as the 'no_proxy' environment variable are set correctly:: @@ -769,10 +668,6 @@ http_proxy and https_proxy environment variables, as well as the git config --global http.sslVerify True git config --global http.proxy <Your http proxy settings> -Validation check: Before running **'functest env prepare'** CLI command, -make sure you can reach http and https sites from inside the Functest -docker container. - For example, try to use the **nc** command from inside the functest docker container:: @@ -786,12 +681,14 @@ Note: In a Jumphost node based on the CentOS family OS, the **nc** commands might not work. You can use the **curl** command instead. curl http://www.opnfv.org:80 + <HTML><HEAD><meta http-equiv="content-type" . . </BODY></HTML> curl https://www.opnfv.org:443 + <HTML><HEAD><meta http-equiv="content-type" . . @@ -833,68 +730,19 @@ should be followed **before** installing the docker engine:: EnvironmentFile=/etc/sysconfig/docker (ignore_errors=yes) DropInPaths=/etc/systemd/system/docker.service.d/env.conf -Now follow the instructions in [`InstallDockerCentOS`_] to download +Now follow the instructions in [`Install Docker on CentOS`_] to download and install the **docker-engine**. The instructions conclude with a "test pull" of a sample "Hello World" docker container. This should now work with the above pre-requisite actions. -Installation and Configuration (Alpine) -======================================= - -Introduction to Alpine ----------------------- -Alpine container have been introduced in Euphrates and released as experimental. -Alpine allows Functest testing in several very light container and thanks to -the refactoring on dependency management shoudl allow the creation of light and -fully customized docker files - -Functest Alpine ---------------- -Docker files are available on the the dockerhub: - - * opnfv/functest-core - * opnfv/functest-healthcheck - * opnfv/functest-smoke - * opnfv/functest-features - * opnfv/functest-components - * opnfv/functest-vnf - - -Preparing your environment --------------------------- - -cat env:: - - INSTALLER_TYPE=XXX - INSTALLER_IP=XXX - EXTERNAL_NETWORK=XXX - -cat openstack.creds:: - - export OS_AUTH_URL=XXX - export OS_USER_DOMAIN_NAME=XXX - export OS_PROJECT_DOMAIN_NAME=XXX - export OS_USERNAME=XXX - export OS_TENANT_NAME=XXX - export OS_PROJECT_NAME=XXX - export OS_PASSWORD=XXX - export OS_VOLUME_API_VERSION=XXX - export OS_IDENTITY_API_VERSION=XXX - export OS_IMAGE_API_VERSION=XXX - -md5sum images/*md5sum images/*:: - - c03e55c22b6fb2127e7de391b488d8d6 `images/CentOS-7-x86_64-GenericCloud.qcow2`_ - f8ab98ff5e73ebab884d80c9dc9c7290 `images/cirros-0.3.5-x86_64-disk.img`_ - 845c9b0221469f9e0f4d7ea0039ab5f2 `images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`_ - +.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html .. _`dockerdocs`: https://docs.docker.com/ .. _`dockerhub`: https://hub.docker.com/r/opnfv/functest/ .. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy .. _`FunctestDockerTags`: https://hub.docker.com/r/opnfv/functest/tags/ -.. _`InstallDockerCentOS`: https://docs.docker.com/engine/installation/linux/centos/ +.. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/ .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html -:: _`images/CentOS-7-x86_64-GenericCloud.qcow2` http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img -:: _`images/cirros-0.3.5-x86_64-disk.img` https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img -:: _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img` https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 +.. _`images/CentOS-7-x86_64-GenericCloud.qcow2` http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img +.. _`images/cirros-0.3.5-x86_64-disk.img` https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img +.. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img` https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 diff --git a/docs/testing/user/configguide/index.rst b/docs/testing/user/configguide/index.rst index 69c41660a..98d798e25 100644 --- a/docs/testing/user/configguide/index.rst +++ b/docs/testing/user/configguide/index.rst @@ -1,9 +1,9 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 -================================== -OPNFV FUNCTEST Configuration Guide -================================== +=========================== +Functest Installation Guide +=========================== .. toctree:: :numbered: @@ -23,13 +23,12 @@ Version history | | | | * update testcase list | | | | | * update docker command | +------------+----------+------------------+----------------------------------+ +| 2017-08-16 | 1.0.2 | Morgan Richomme | Adaptations for Euphrates | ++------------+----------+------------------+----------------------------------+ Introduction ============ This document describes how to install and configure Functest in OPNFV. -The Functest CLI is used during the Functest environment preparation -phase. The given example commands should work in both virtual and bare -metal cases alike. High level architecture ----------------------- @@ -44,14 +43,12 @@ follows:: | +----------------------------+ | | | | | | | | | | | | | +-----+ Jumphost | | | | | | - | | +--------+ | | | | | | | | | | | | | | +--------------------+ | | | | | | | | | | | | | | | | | | | Tools | +----------------+ | | | | | | - Rally | | | | | | | | | | - Robot | | | | | | | - | | | - TestON | | | | | | | | | | - RefStack | | | | | | | | | | | |-------------------------+ | | | | | Testcases | | | | | | | @@ -93,9 +90,12 @@ follows:: + + + + + + SUT = System Under Test -All the libraries and dependencies needed by all of the Functest tools -are pre-installed into the Docker image. This allows running Functest -on any platform on any Operating System. +Note connectivity to management network is not needed for most of the testcases. +But it may be needed for some specific snaps tests. + +All the libraries and dependencies needed by all of the Functest tools are +pre-installed into the Docker images. This allows running Functest on any +platform. The automated mechanisms inside the Functest Docker container will: @@ -131,15 +131,15 @@ References `[4]`_ : Functest Configuration guide -`OPNFV main site`_ +`[5]`_ : OPNFV main site -`Functest page`_ +`[6]`_ : Functest wiki page IRC support channel: #opnfv-functest .. _`[1]`: https://ask.openstack.org/en/question/68144/keystone-unable-to-use-the-public-endpoint/ -.. _`[2]`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html -.. _`[3]`: https://git.opnfv.org/cgit/releng/tree/jjb/functest/functest-ci-jobs.yml -.. _`[4]`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/configguide/index.html -.. _`OPNFV main site`: http://www.opnfv.org -.. _`Functest page`: https://wiki.opnfv.org/functest +.. _`[2]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html +.. _`[3]`: https://git.opnfv.org/releng/tree/jjb/functest/functest-daily-jobs.yml +.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html +.. _`[5]`: http://www.opnfv.org +.. _`[6]`: https://wiki.opnfv.org/functest diff --git a/docs/testing/user/configguide/prerequisites.rst b/docs/testing/user/configguide/prerequisites.rst index 94f676008..418cd1a73 100644 --- a/docs/testing/user/configguide/prerequisites.rst +++ b/docs/testing/user/configguide/prerequisites.rst @@ -11,7 +11,12 @@ Several prerequisites are needed for Functest: #. A public/external network created on the SUT #. An admin/management network created on the SUT #. Connectivity from the Jumphost to the SUT public/external network - #. Connectivity from the Jumphost to the SUT admin/management network + +Some specific SNAPS tests may require a connectivity from the Jumphost to the +SUT admin/management network but most of the test cases do not. This requirement +can be overridden by overriding the 'interface' attribute (OS_INTERFACE) value +to 'public'. Another means to circumvent this issue would be to change the +'snaps.use_keystone' value from True to False. WARNING: Connectivity from Jumphost is essential and it is of paramount importance to make sure it is working before even considering to install @@ -85,20 +90,11 @@ the public/external subnet is reachable from the Jumphost. *Hint:* For the given OPNFV Installer in use, the IP sub-net address used for the public/external network is usually a planning item and -should thus be known. Consult the OPNFV Configuration guide `[4]`_, and -ensure you can reach each node in the SUT, from the Jumphost using the -'ping' command using the respective IP address on the public/external -network for each node in the SUT. The details of how to determine the -needed IP addresses for each node in the SUT may vary according to the -used installer and are therefore ommitted here. - -Connectivity to Admin/Management network on SUT ------------------------------------------------ -Some of the Functest tools need to have access to the OpenStack -admin/management network of the controllers `[1]`_. - -For this reason, check the connectivity from the Jumphost to all the -controllers in cluster in the OpenStack admin/management network range. +should thus be known. Ensure you can reach each node in the SUT, from the +Jumphost using the 'ping' command using the respective IP address on the +public/external network for each node in the SUT. The details of how to +determine the needed IP addresses for each node in the SUT may vary according +to the used installer and are therefore ommitted here. .. _`[1]`: https://ask.openstack.org/en/question/68144/keystone-unable-to-use-the-public-endpoint/ -.. _`[4]`: http://artifacts.opnfv.org/functest/danube/docs/configguide/index.html +.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html |