From 5504724dc69096b36948de9bd07b82f5058242f0 Mon Sep 17 00:00:00 2001 From: Morgan Richomme Date: Tue, 12 Sep 2017 11:39:11 +0200 Subject: Update Functest documentation for Euphrates - Better description of Alpine docker (default artifact) - Reorganization of config and user guide - Upgrade of the developer guide Change-Id: Ie8e10f027bfcdb01c992cd161a1af2d6d6e29536 Signed-off-by: Morgan Richomme --- docs/testing/user/configguide/configguide.rst | 844 +++++++++++--------------- 1 file changed, 346 insertions(+), 498 deletions(-) (limited to 'docs/testing/user/configguide/configguide.rst') diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index d19939afe..0a260ad0a 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -1,346 +1,366 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. SPDX-License-Identifier: CC-BY-4.0 -Installation and configuration (Ubuntu) -======================================= +Installation and configuration +============================== -The historical docker file is based on Ubuntu. It has been maintained for -Euphrates. +Alpine containers have been introduced in Euphrates. +Alpine allows Functest testing in several very light containers and thanks to +the refactoring on dependency management should allow the creation of light and +fully customized docker files. -Pulling the Docker image ------------------------- -Pull the Functest Docker image ('opnfv/functest') from the public -dockerhub registry under the OPNFV account: [̀`dockerhub`_], with the -following docker command:: - - docker pull opnfv/functest: - -where identifies a release of the Functest docker -container image in the public Dockerhub registry. There are many tags -created automatically by the CI mechanisms, and you must ensure you -pull an image with the **correct tag** to match the OPNFV software -release installed in your environment. All available tagged images can -be seen from location [FunctestDockerTags_]. For example, when running -on the first official release of the OPNFV Danube system platform, -tag "danube.1.0" is needed. For the second and third releases, the tag -"danube.2.0" and "danube.3.0" can be used respectively. -Pulling other tags might cause some problems while running the tests. -Docker images pulled without a tag specifier bear the implicitly -assigned label "latest". If you need to specifically pull the latest -Functest docker image, then omit the tag argument:: - - docker pull opnfv/functest - -After pulling the Docker image, check that it is available with the -following docker command:: - - [functester@jumphost ~]$ docker images - REPOSITORY TAG IMAGE ID CREATED SIZE - opnfv/functest latest 8cd6683c32ae 2 weeks ago 1.321 GB - opnfv/functest danube.2.0 d2c174a91911 7 minutes ago 1.471 GB - opnfv/functest danube.1.0 13fa54a1b238 4 weeks ago 1.29 GB +It is still possible to use the monolithic functest opnfv/functest especially +for tests on Aarch64 architecture. + +Functest Dockers +---------------- +Docker files are available on the dockerhub: + + * opnfv/functest-core + * opnfv/functest-healthcheck + * opnfv/functest-smoke + * opnfv/functest-features + * opnfv/functest-components + * opnfv/functest-vnf + * opnfv/functest-parser + * opnfv/functest-restapi + +By default, we use the docker tag latest, but you may pull a tagged docker +image. The Functest docker container environment can -in principle- be also used with non-OPNFV official installers (e.g. 'devstack'), with the **disclaimer** that support for such environments is outside of the scope and responsibility of the OPNFV project. -Please note that alpine dockers have been introduced in Euphrates. See alpine -section for details. -Accessing the Openstack credentials ------------------------------------ -OpenStack credentials are mandatory and must be provided to Functest. -When running the command "functest env prepare", the framework will -automatically look for the Openstack credentials file -"/home/opnfv/functest/conf/openstack.creds" and will exit with -error if it is not present or it is empty. +Preparing your environment +-------------------------- -There are 2 ways to provide that file: +cat env:: - * by using a Docker volume with -v option when creating the Docker container. - This is referred to in docker documentation as "Bind Mounting". - See the usage of this parameter in the following chapter. - * or creating manually the file '/home/opnfv/functest/conf/openstack.creds' - inside the running container and pasting the credentials in it. Consult - your installer guide for further details. This is however not - instructed in this document. + INSTALLER_TYPE=XXX + INSTALLER_IP=XXX + EXTERNAL_NETWORK=XXX + DEPLOY_SCENARIO=XXX -There is a default environment variable in the Functest container **$creds** -that points to the credentials absolute path to help the user with this task. +See section on environment variables for details. -In proxified environment you may need to change the credentials file. -There are some tips in chapter: `Proxy support`_ +cat openstack.creds:: + + export OS_AUTH_URL=XXX + export OS_USER_DOMAIN_NAME=XXX + export OS_PROJECT_DOMAIN_NAME=XXX + export OS_USERNAME=XXX + export OS_TENANT_NAME=XXX + export OS_PROJECT_NAME=XXX + export OS_PASSWORD=XXX + export OS_VOLUME_API_VERSION=XXX + export OS_IDENTITY_API_VERSION=XXX + export OS_IMAGE_API_VERSION=XXX + +See section on OpenStack credentials for details. -Functest Docker parameters +Create a directory for the different images (included as volume):: + + mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* + + images/CentOS-7-aarch64-GenericCloud.qcow2 + images/CentOS-7-aarch64-GenericCloud.qcow2.xz + images/CentOS-7-x86_64-GenericCloud.qcow2 + images/cirros-0.3.5-x86_64-disk.img + images/cirros-0.3.5-x86_64-lxc.tar.gz + images/cirros-d161201-aarch64-disk.img + images/cirros-d161201-aarch64-initramfs + images/cirros-d161201-aarch64-kernel + images/cloudify-manager-premium-4.0.1.qcow2 + images/img + images/trusty-server-cloudimg-amd64-disk1.img + images/ubuntu-14.04-server-cloudimg-amd64-disk1.img + images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img + images/ubuntu-16.04-server-cloudimg-amd64-disk1.img + images/vyos-1.1.7.img + + +Testing healthcheck suite -------------------------- -This chapter explains how to run a container for executing functest -test suites. Numbered list below explains some details of the -recommended parameters for invoking docker container - #. It is a good practice to assign a precise container name through - the **--name** option. +Run healthcheck suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-healthcheck + +Results shall be displayed as follows:: + + +----------------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +----------------------------+------------------+---------------------+------------------+----------------+ + | connection_check | functest | healthcheck | 00:02 | PASS | + | api_check | functest | healthcheck | 04:57 | PASS | + | snaps_health_check | functest | healthcheck | 00:51 | PASS | + +----------------------------+------------------+---------------------+------------------+----------------+ + +Testing smoke suite +------------------- + +Run smoke suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-smoke + +Results shall be displayed as follows:: + + +------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +------------------------------+------------------+---------------+------------------+----------------+ + | vping_ssh | functest | smoke | 01:19 | PASS | + | vping_userdata | functest | smoke | 01:56 | PASS | + | tempest_smoke_serial | functest | smoke | 26:30 | PASS | + | rally_sanity | functest | smoke | 19:42 | PASS | + | refstack_defcore | functest | smoke | 22:00 | PASS | + | snaps_smoke | functest | smoke | 41:14 | PASS | + | odl | functest | smoke | 00:16 | PASS | + | odl_netvirt | functest | smoke | 00:00 | SKIP | + | fds | functest | smoke | 00:00 | SKIP | + +------------------------------+------------------+---------------+------------------+----------------+ + Note: if the scenario does not support some tests, they are indicated as SKIP. + See User guide for details. + +Testing features suite +---------------------- + +Run features suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-features + +Results shall be displayed as follows:: + + +---------------------------+--------------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+--------------------------+------------------+------------------+----------------+ + | promise | promise | features | 00:00 | SKIP | + | bgpvpn | sdnvpn | features | 00:00 | SKIP | + | security_scan | securityscanning | features | 00:00 | SKIP | + | functest-odl-sfc | sfc | features | 00:00 | SKIP | + | domino-multinode | domino | features | 00:00 | SKIP | + | barometercollectd | barometer | features | 00:00 | SKIP | + +---------------------------+--------------------------+------------------+------------------+----------------+ + Note: if the scenario does not support some tests, they are indicated as SKIP. + See User guide for details. + +Testing components suite +------------------------ - #. Assign parameter for installer type:: +Run components suite:: - -e "INSTALLER_TYPE=" - # Use one of following apex, compass, fuel or joid + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-components - #. Functest needs to know the IP of some installers:: +Results shall be displayed as follows:: - -e "INSTALLER_IP=" + +-------------------------------+------------------+--------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------------+------------------+--------------------+------------------+----------------+ + | tempest_full_parallel | functest | components | 102:48 | PASS | + | rally_full | functest | components | 160:58 | PASS | + | tempest_custom | functest | components | 00:00 | SKIP | + +-------------------------------+------------------+--------------------+------------------+----------------+ - These two env variables are useful extract some information - from the deployment. However, for some test cases like - SFC or Barometer they are mandatory since the tests - need to access the installer node and the deployment. +Testing vnf suite +----------------- - #. Credentials for accessing the Openstack. - Most convenient way of passing them to container is by having a - local copy of the credentials file in Jumphost and then using the - **-v** option. In the example we have local file by the name of - "overcloudrc" and we are using that as an argument:: +Run vnf suite:: - -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds +sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-vnf - The credentials file needs to exist in the Docker container - under the path: '/home/opnfv/functest/conf/openstack.creds'. +Results shall be displayed as follows:: - **WARNING:** If you are using the Joid installer, you must pass the - credentials using the **-v** option: - -v /var/lib/jenkins/admin-openrc:/home/opnfv/functest/conf/openstack.creds. - See the section `Accessing the Openstack credentials`_ above. + +---------------------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------------+------------------+--------------+------------------+----------------+ + | cloudify_ims | functest | vnf | 21:25 | PASS | + | orchestra_openims | functest | vnf | 11:02 | FAIL | + | orchestra_clearwaterims | functest | vnf | 09:13 | FAIL | + | vyos_vrouter | functest | vnf | 00:00 | SKIP | + +---------------------------------+------------------+--------------+------------------+----------------+ - #. Passing deployment scenario - When running Functest against any of the supported OPNFV scenarios, - it is recommended to include also the environment variable - **DEPLOY_SCENARIO**. The **DEPLOY_SCENARIO** environment variable - is passed with the format:: - -e "DEPLOY_SCENARIO=os---" - where: - os = OpenStack (No other VIM choices currently available) - controller is one of ( nosdn | odl_l2 | odl_l3 ) - nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature ) - If several features are pertinent then use the underscore - character '_' to separate each feature (e.g. ovs_kvm) - 'nofeature' indicates no NFV feature is deployed - ha_mode (high availability) is one of ( ha | noha ) +Environment variables +===================== - **NOTE:** Not all possible combinations of "DEPLOY_SCENARIO" are - supported. The name passed in to the Functest Docker container - must match the scenario used when the actual OPNFV platform was - deployed. See release note to see the list of supported scenarios. +Several environement variables may be specified: + * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid|osa) + * INSTALLER_IP= + * DEPLOY_SCENARIO=--- - **NOTE:** The scenario name is mainly used to automatically detect - if a test suite is runnable or not (e.g. it will prevent ONOS test suite - to be run on ODL scenarios). If not set, Functest will try to run the - default test cases that might not include SDN controller or a specific - feature - **NOTE:** A HA scenario means that 3 OpenStack controller nodes are - deployed. It does not necessarily mean that the whole system is HA. See - installer release notes for details. +INSTALLER IP may be required by some test cases like SFC or Barometer in order +to access the installer node and the deployment. +The format for the DEPLOY_SCENARIO env variable can be described as follows: + * vim: (os|k8s) = OpenStack or Kubernetes + * controller is one of ( nosdn | odl ) + * nfv_feature is one or more of ( ovs | kvm | sfc | bgpvpn | nofeature ) + * ha_mode (high availability) is one of ( ha | noha ) -Putting all above together, when using installer 'fuel' and an invented -INSTALLER_IP of '10.20.0.2', the recommended command to create the -Functest Docker container is as follows:: +If several features are pertinent then use the underscore character '_' to +separate each feature (e.g. ovs_kvm) 'nofeature' indicates no OPNFV feature is +deployed - docker run --name "FunctestContainer" -it \ - -e "INSTALLER_IP=10.20.0.2" \ - -e "INSTALLER_TYPE=fuel" \ - -e "DEPLOY_SCENARIO=os-odl_l2-ovs_kvm-ha" \ - -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \ - opnfv/functest /bin/bash +The list of supported scenarios per release/installer is indicated in the +release note. -After the *run* command, a new prompt appears which means that we are inside -the container and ready to move to the next step. +**NOTE:** The scenario name is mainly used to automatically detect +if a test suite is runnable or not (e.g. it will prevent ONOS test suite to be +run on ODL scenarios). If not set, Functest will try to run the default test +cases that might not include SDN controller or a specific +feature -For tips on how to set up container with installer Apex, see chapter -`Apex Installer Tips`_. +**NOTE:** A HA scenario means that 3 OpenStack controller nodes are +deployed. It does not necessarily mean that the whole system is HA. See +installer release notes for details. Finally, three additional environment variables can also be passed in to the Functest Docker Container, using the -e -"=" mechanism. The first two of these are +"=" mechanism. The first two parameters are only relevant to Jenkins CI invoked testing and **should not be used** -when performing manual test scenarios:: - - -e "NODE_NAME=" \ - -e "BUILD_TAG=" \ - -e "CI_DEBUG=" - where: - = Symbolic name of the POD where the tests are run. - Visible in test results files, which are stored - to the database. This option is only used when - tests are activated under Jenkins CI control. - It indicates the POD/hardware where the test has - been run. If not specified, then the POD name is - defined as "Unknown" by default. - DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. - = Symbolic name of the Jenkins Build Job. - Visible in test results files, which are stored - to the database. This option is only set when - tests are activated under Jenkins CI control. - It enables the correlation of test results, - which - are independently pushed to the results database - from different Jenkins jobs. - DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. - = "true" or "false" - Default = "false", if not specified - If "true" is specified, then additional debug trace - text can be sent to the test results file / log files - and also to the standard console output. - -Installer Tips --------------- - -Apex Installer Tips -^^^^^^^^^^^^^^^^^^^ -Some specific tips are useful for the Apex Installer case. If not using -Apex Installer; ignore this section. - -In case of Triple-O based installer (like Apex) the docker container -needs to connect to the installer VM, so it is then required that some -known SSH keys are present in docker container. Since the Jumphost root -SSH keys are already known, easiest way is to use those using the -'Bind mount' method. See below for sample parameter:: - - -v /root/.ssh/id_rsa:/root/.ssh/id_rsa - - NOTE: You need the "sudo" when creating the container to access root - users ssh credentials even the docker command itself might not - require that. - -HINT! In case of Triple-O installers you can find value for the -INSTALLER_IP parameter by executing command and note the returned IP -address:: - - inst=$(sudo virsh list | grep -iEo "undercloud|instack") - sudo virsh domifaddr ${inst} - - NOTE: In releases prior to Colorado, the name 'instack' was - used. Currently the name 'undercloud' is used. - -You can copy the credentials file from the "stack" users home directory -in installer VM to Jumphost. Please check the correct IP from the -command above. In the example below we are using invented IP address -"192.168.122.89":: - - scp stack@192.168.122.89:overcloudrc . - -Here is an example of the full docker command invocation for an Apex -installed system, using latest Functest docker container, for -illustration purposes:: - - sudo docker run -it --name "ApexFuncTestODL" \ - -e "INSTALLER_IP=192.168.122.89" \ - -e "INSTALLER_TYPE=apex" \ - -e "DEPLOY_SCENARIO=os-odl_l2-nofeature-ha" \ - -v /root/.ssh/id_rsa:/root/.ssh/id_rsa \ - -v ~/overcloudrc:/home/opnfv/functest/conf/openstack.creds \ - opnfv/functest /bin/bash - -Compass installer local development env usage Tips -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In the compass-functest local test case check and development environment, -in order to get openstack service inside the functest container, some -parameters should be configured during container creation, which are -hard to guess for freshman. This section will provide the guideline, the -parameters values are defaults here, which should be adjusted according -to the settings, the complete steps are given here so as not to appear -too abruptly. - -1, Pull Functest docker image from public dockerhub:: - - docker pull opnfv/functest: - - here can be "brahmaputra.1.0", "colorado.1.0", etc. -Tag omitted means the latest docker image:: - - docker pull opnfv/functest - -2, Functest Docker container creation - -To make a file used for the environment, such as 'functest-docker-env':: - - CINDER_ENDPOINT_TYPE=publicURL - NOVA_ENDPOINT_TYPE=publicURL - OS_ENDPOINT_TYPE=publicURL - OS_INTERFACE=publicURL - OS_USERNAME=admin - OS_PASSWORD='990232e0885da343ac805523322d' - OS_PROJECT_NAME=admin - OS_TENANT_NAME=admin - OS_AUTH_URL=https://192.16.1.222:5000/v3 - OS_NO_CACHE=1 - OS_USER_DOMAIN_NAME=Default - OS_PROJECT_DOMAIN_NAME=Default - OS_REGION_NAME=RegionOne - OS_IDENTITY_API_VERSION=3 - OS_AUTH_VERSION=3 +when performing manual test scenarios: + + * NODE_NAME = + * BUILD_TAG = + * CI_DEBUG = + +where: + + * = Symbolic name of the POD where the tests are run. + Visible in test results files, which are stored + to the database. This option is only used when + tests are activated under Jenkins CI control. + It indicates the POD/hardware where the test has + been run. If not specified, then the POD name is + defined as "Unknown" by default. + DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. + * = Symbolic name of the Jenkins Build Job. + Visible in test results files, which are stored + to the database. This option is only set when + tests are activated under Jenkins CI control. + It enables the correlation of test results, + which + are independently pushed to the results database + from different Jenkins jobs. + DO NOT USE THIS OPTION IN MANUAL TEST SCENARIOS. + * = "true" or "false" + Default = "false", if not specified + If "true" is specified, then additional debug trace + text can be sent to the test results file / log files + and also to the standard console output. + + +Openstack credentials +===================== +OpenStack credentials are mandatory and must be provided to Functest. +When running the command "functest env prepare", the framework will +automatically look for the Openstack credentials file +"/home/opnfv/functest/conf/openstack.creds" and will exit with +error if it is not present or is empty. +There are 2 ways to provide that file: -Note: please adjust the content according to the environment, such as -'TENANT_ID' maybe used for some special cases. + * by using a Docker volume with -v option when creating the Docker container. + This is referred to in docker documentation as "Bind Mounting". + See the usage of this parameter in the following chapter. + * or creating manually the file '/home/opnfv/functest/conf/openstack.creds' + inside the running container and pasting the credentials in it. Consult + your installer guide for further details. This is however not + instructed in this document. -Then to create the Functest docker:: +There is a default environment variable in the Functest container **$creds** +that points to the credentials absolute path to help the user with this task. - docker run --privileged=true --rm -t \ - --env-file functest-docker-env \ - --name \ - opnfv/functest: /bin/bash +In proxified environment you may need to change the credentials file. +There are some tips in chapter: `Proxy support`_ -3, To attach Functest container +SSL Support +----------- +If you need to connect to a server that is TLS-enabled (the auth URL +begins with "https") and it uses a certificate from a private CA or a +self-signed certificate, then you will need to specify the path to an +appropriate CA certificate to use, to validate the server certificate +with the environment variable OS_CACERT:: + + echo $OS_CACERT + /etc/ssl/certs/ca.crt + +However, this certificate does not exist in the container by default. +It has to be copied manually from the OpenStack deployment. This can be +done in 2 ways: -Before trying to attach the Functest container, the status can be checked by:: + #. Create manually that file and copy the contents from the OpenStack + controller. + #. (Recommended) Add the file using a Docker volume when starting the + container:: - docker ps -a + -v :/etc/ssl/certs/ca.cert -to attach the 'Up' status Functest container and start bash mode:: +You might need to export OS_CACERT environment variable inside the +container:: - docker exec -it bash + export OS_CACERT=/etc/ssl/certs/ca.crt -4, Functest environment preparation and check +Certificate verification can be turned off using OS_INSECURE=true. For +example, Fuel uses self-signed cacerts by default, so an pre step would +be:: -To see the Section below `Preparing the Functest environment`_. + export OS_INSECURE=true Functest docker container directory structure ---------------------------------------------- +============================================= Inside the Functest docker container, the following directory structure should now be in place:: - `-- home - | `-- opnfv - | |-- functest - | | |-- conf - | | |-- data - | | |-- images - | | `-- results - | `-- repos - | |-- doctor + `-- + |- home + | |-- opnfv + | | `- functest + | | |-- conf + | | `-- results + | `-- repos | `-- vnfs - -- src - |-- tempest - |-- vims-test - |-- odl_test - `-- fds - -Underneath the '/home/opnfv/' directory, the Functest docker container + |- src + | |-- tempest + | |-- vims-test + | |-- odl_test + | `-- fds + `- usr + `- lib + `- python2.7 + `- site-packages + `- functest + |-- ... + +Underneath the '/home/opnfv/functest' directory, the Functest docker container includes two main directories: - * The **functest** directory stores configuration files (e.g. the + * The **conf** directory stores configuration files (e.g. the OpenStack creds are stored in path '/home/opnfv/functest/conf/openstack.creds'), - the **data** directory stores a 'cirros' test image used in some - functional tests and the **results** directory stores some temporary - result log files - * The **repos** directory holds various repositories. The directories - are used for the installation of the needed tooling (e.g. rally) or - for the retrieval of feature projects scenarios (e.g. promise) + * the **results** directory stores some temporary result log files + +src and repos directories are used to host third party code used for the tests. -The structure under the **functest** repository can be described as -follows:: +The functest code is under /usr/lib/python2.7/site-packages/functest +The structure can be described as follows:: |-- INFO |-- LICENSE @@ -465,8 +485,7 @@ follows:: We may distinguish several directories, the first level has 5 directories: -* **api**: This directory is dedicated for the internal Functest API and the API - (framework) documentations. +* **api**: This directory is dedicated to the API (framework) documentations. * **commons**: This directory is dedicated for storage of traffic profile or any other test inputs that could be reused by any test project. @@ -477,7 +496,9 @@ We may distinguish several directories, the first level has 5 directories: * **functest**: This directory contains all the code needed to run functest internal cases and OPNFV onboarded feature or VNF test cases. -Functest directory has 6 directories: +Functest directory has 7 sub-directories: + * **api**: This directory is dedicated for the internal Functest API and the + API (framework) documentations. * **ci**: This directory contains test structure definition files (e.g .yaml) and bash shell/python scripts used to configure and execute Functional tests. The test execution script @@ -495,8 +516,31 @@ Functest directory has 6 directories: own test code. See for an example the Openstack helper utility: 'openstack_utils.py'. -Useful Docker commands ----------------------- + +Logs +==== +By default all the logs are put un /home/opnfv/functest/results/functest.log. +If you want to have more logs in console, you may edit the logging.ini file +manually. +Connect on the docker then edit the file located in +/usr/lib/python2.7/site-packages/functest/ci/logging.ini + +Change wconsole to console in the desired module to get more traces. + + +Configuration +============= + +You may also directly modify the python code or the configuration file (e.g. +testcases.yaml used to declare test constraints) under +/usr/lib/python2.7/site-packages/functest + + +Tips +==== + +Docker +------ When typing **exit** in the container prompt, this will cause exiting the container and probably stopping it. When stopping a running Docker container all the changes will be lost, there is a keyboard shortcut @@ -538,116 +582,6 @@ destroy it:: Check the Docker documentation [`dockerdocs`_] for more information. -Preparing the Functest environment ----------------------------------- -Once the Functest docker container is up and running, the required -Functest environment needs to be prepared. A custom built **functest** -CLI utility is available to perform the needed environment preparation -action. Once the environment is prepared, the **functest** CLI utility -can be used to run different functional tests. The usage of the -**functest** CLI utility to run tests is described further in the -`Functest User Guide`_ - -Prior to commencing the Functest environment preparation, we can check -the initial status of the environment. Issue the **functest env status** -command at the prompt:: - - functest env status - Functest environment is not installed. - - Note: When the Functest environment is prepared, the command will - return the status: "Functest environment ready to run tests." - -To prepare the Functest docker container for test case execution, issue -the **functest env prepare** command at the prompt:: - - functest env prepare - -This script will make sure that the requirements to run the tests are -met and will install the needed libraries and tools by all Functest -test cases. It should be run only once every time the Functest docker -container is started from scratch. If you try to run this command, on -an already prepared environment, you will be prompted whether you really -want to continue or not:: - - functest env prepare - It seems that the environment has been already prepared. - Do you want to do it again? [y|n] - - (Type 'n' to abort the request, or 'y' to repeat the - environment preparation) - - -To list some basic information about an already prepared Functest -docker container environment, issue the **functest env show** at the -prompt:: - - functest env show - +======================================================+ - | Functest Environment info | - +======================================================+ - | INSTALLER: apex, 192.168.122.89 | - | SCENARIO: os-odl_l2-nofeature-ha | - | POD: localhost | - | GIT BRANCH: master | - | GIT HASH: 5bf1647dec6860464eeb082b2875798f0759aa91 | - | DEBUG FLAG: false | - +------------------------------------------------------+ - | STATUS: ready | - +------------------------------------------------------+ - - Where: - - INSTALLER: Displays the INSTALLER_TYPE value - - here = "apex" - and the INSTALLER_IP value - - here = "192.168.122.89" - SCENARIO: Displays the DEPLOY_SCENARIO value - - here = "os-odl_l2-nofeature-ha" - POD: Displays the value passed in NODE_NAME - - here = "localhost" - GIT BRANCH: Displays the git branch of the OPNFV Functest - project repository included in the Functest - Docker Container. - - here = "master" - (In first official colorado release - would be "colorado.1.0") - GIT HASH: Displays the git hash of the OPNFV Functest - project repository included in the Functest - Docker Container. - - here = "5bf1647dec6860464eeb082b2875798f0759aa91" - DEBUG FLAG: Displays the CI_DEBUG value - - here = "false" - - NOTE: In Jenkins CI runs, an additional item "BUILD TAG" - would also be listed. The value is set by Jenkins CI. - -Finally, the **functest** CLI has a **--help** options: - -Some examples:: - - functest --help Usage: functest [OPTIONS] COMMAND [ARGS]... - - Options: - --version Show the version and exit. - -h, --help Show this message and exit. - - Commands: - env - openstack - testcase - tier - - functest env --help - Usage: functest env [OPTIONS] COMMAND [ARGS]... - - Options: - -h, --help Show this message and exit. - - Commands: - prepare Prepares the Functest environment. - show Shows information about the current... - status Checks if the Functest environment is ready... Checking Openstack and credentials ---------------------------------- @@ -673,53 +607,24 @@ This command must show a set of environment variables starting with *OS_*, for example:: OS_REGION_NAME=RegionOne - OS_DEFAULT_DOMAIN=default + OS_USER_DOMAIN_NAME=Default OS_PROJECT_NAME=admin - OS_PASSWORD=admin - OS_AUTH_STRATEGY=keystone - OS_AUTH_URL=http://172.30.10.3:5000/v2.0 + OS_AUTH_VERSION=3 + OS_IDENTITY_API_VERSION=3 + OS_PASSWORD=da54c27ae0d10dfae5297e6f0d6be54ebdb9f58d0f9dfc + OS_AUTH_URL=http://10.1.0.9:5000/v3 OS_USERNAME=admin OS_TENANT_NAME=admin OS_ENDPOINT_TYPE=internalURL - OS_NO_CACHE=true + OS_INTERFACE=internalURL + OS_NO_CACHE=1 + OS_PROJECT_DOMAIN_NAME=Default + If the OpenStack command still does not show anything or complains about connectivity issues, it could be due to an incorrect url given to the OS_AUTH_URL environment variable. Check the deployment settings. -SSL Support ------------ -If you need to connect to a server that is TLS-enabled (the auth URL -begins with "https") and it uses a certificate from a private CA or a -self-signed certificate, then you will need to specify the path to an -appropriate CA certificate to use, to validate the server certificate -with the environment variable OS_CACERT:: - - echo $OS_CACERT - /etc/ssl/certs/ca.crt - -However, this certificate does not exist in the container by default. -It has to be copied manually from the OpenStack deployment. This can be -done in 2 ways: - - #. Create manually that file and copy the contents from the OpenStack - controller. - #. (Recommended) Add the file using a Docker volume when starting the - container:: - - -v :/etc/ssl/certs/ca.cert - -You might need to export OS_CACERT environment variable inside the -container:: - - export OS_CACERT=/etc/ssl/certs/ca.crt - -Certificate verification can be turned off using OS_INSECURE=true. For -example, Fuel uses self-signed cacerts by default, so an pre step would -be:: - - export OS_INSECURE=true - Proxy support ------------- If your Jumphost node is operating behind a http proxy, then there are @@ -732,12 +637,6 @@ succeed: in section: `Docker Installation on CentOS behind http proxy`_ below. - #. Execution of the Functest environment preparation inside the - created docker container Functest needs internet access to - download some resources for some test cases. This might not - work properly if the Jumphost is connecting to internet - through a http Proxy. - If that is the case, make sure the resolv.conf and the needed http_proxy and https_proxy environment variables, as well as the 'no_proxy' environment variable are set correctly:: @@ -769,10 +668,6 @@ http_proxy and https_proxy environment variables, as well as the git config --global http.sslVerify True git config --global http.proxy -Validation check: Before running **'functest env prepare'** CLI command, -make sure you can reach http and https sites from inside the Functest -docker container. - For example, try to use the **nc** command from inside the functest docker container:: @@ -786,12 +681,14 @@ Note: In a Jumphost node based on the CentOS family OS, the **nc** commands might not work. You can use the **curl** command instead. curl http://www.opnfv.org:80 + curl https://www.opnfv.org:443 +