aboutsummaryrefslogtreecommitdiffstats
BranchCommit messageAuthorAge
stable/iruyaadd yardstick iruya 9.0.0 release notesrexlee87765 years
masteradd yardstick iruya 9.0.0 release notesrexlee87765 years
stable/hunterYardstick Hunter 8.0.0 release notesrexlee87766 years
stable/gambiaYardsitck Gambia Release 7.2.0 release notes.rexlee87766 years
stable/frasertc046 kill keystone process condition incorrect - dovetailrexlee87766 years
stable/euphratesBugfix: failed to generete gui files in stable/euphrates in CIchenjiankun7 years
stable/danubeUpdate release note for Danube.3.2JingLu57 years
stable/coloradoMerge "arm64: Using wily kernel since the vivid is gone" into stable/coloradoJing Lu8 years
stable/brahmaputraRelease notes for 3.0Ana C9 years
 
TagDownloadAuthorAge
opnfv-8.0.0commit 9bd54d7914...jenkins-ci6 years
opnfv-7.2.0commit eb6d97898b...jenkins-ci6 years
opnfv-7.1.0commit a1e5634d35...jenkins-ci6 years
opnfv-7.0.0commit b9fa3eac6c...jenkins-ci6 years
ovp-2.0.0commit 6c10a2d9cc...rexlee87766 years
opnfv-6.2.1commit a99613376d...jenkins-ci6 years
opnfv-6.2.0commit b07610535f...jenkins-ci6 years
opnfv-6.1.0commit 7dc30d54a7...Aric Gardner7 years
opnfv-6.0.0commit a4c8f2a99f...Trevor Bramwell7 years
opnfv-5.1.0commit 22c3213246...Ross Brattain7 years
opnfv-5.0.0commit 269c070a51...JingLu57 years
opnfv-5.0.RC1commit 1f04604b78...JingLu57 years
danube.3.2commit e9338f29a7...JingLu57 years
danube.3.1commit 795438d076...Ross Brattain7 years
danube.3.0commit 5bdc049447...Ross Brattain8 years
danube.2.0commit 1b267df543...JingLu58 years
danube.2.RC1commit 1b267df543...JingLu58 years
danube.1.0commit f990f63a40...kubi8 years
colorado.3.0commit 5315d784b0...Aric Gardner8 years
colorado.2.0commit ec0e69dd20...Aric Gardner8 years
colorado.1.0commit 42b98f2da7...kubi8 years
brahmaputra.3.0commit 654e1479f4...Ana C9 years
brahmaputra.2.0commit 2e0401b13b...Ana C9 years
brahmaputra.1.0commit bc63c492a8...Ana C9 years
* The API / Test collection framework * The dashboards * The automatic reporting portals * The testcase catalog This document describes how, as a developer, you may interact with the Functest project. The first section details the main working areas of the project. The Second part is a list of "How to" to help you to join the Functest family whatever your field of interest is. ======================== Functest developer areas ======================== Functest High level architecture ================================ Functest is a project delivering test containers dedicated to OPNFV. It includes the tools, the scripts and the test scenarios. In Euphrates Alpine containers have been introduced in order to lighten the container and manage testing slicing. The new containers are created according to the different tiers: * functest-core: https://hub.docker.com/r/opnfv/functest-core/ * functest-healthcheck: https://hub.docker.com/r/opnfv/functest-healthcheck/ * functest-smoke: https://hub.docker.com/r/opnfv/functest-smoke/ * functest-features: https://hub.docker.com/r/opnfv/functest-features/ * functest-components: https://hub.docker.com/r/opnfv/functest-components/ * functest-vnf: https://hub.docker.com/r/opnfv/functest-vnf/ * functest-restapi: https://hub.docker.com/r/opnfv/functest-restapi/ Standalone functest dockers are maintained for Euphrates but Alpine containers are recommended. Functest can be described as follow:: +----------------------+ | | | +--------------+ | +-------------------+ | | | | Public | | | | Tools | +------------------+ OPNFV | | | Scripts | | | System Under Test | | | Scenarios | | | | | | | | | | | +--------------+ | +-------------------+ | | | Functest Docker | | | +----------------------+ Functest internal test cases ============================ The internal test cases in Euphrates are: * connection_check * vping_ssh * vping_userdata * odl * rally_full * rally_sanity * tempest_smoke * tempest_full * cloudify_ims By internal, we mean that this particular test cases have been developed and/or integrated by functest contributors and the associated code is hosted in the Functest repository. An internal case can be fully developed or a simple integration of upstream suites (e.g. Tempest/Rally developed in OpenStack, or odl suites are just integrated in Functest). The structure of this repository is detailed in `[1]`_. The main internal test cases are in the opnfv_tests subfolder of the repository, the internal test cases can be grouped by domain: * sdn: odl, odl_fds * openstack: connection_check, vping_ssh, vping_userdata, tempest_*, rally_* * vnf: cloudify_ims If you want to create a new test case you will have to create a new folder under the testcases directory (See next section for details). Functest external test cases ============================ The external test cases are inherited from other OPNFV projects, especially the feature projects. The external test cases are: * barometer * bgpvpn * doctor * domino * fds * promise * refstack_defcore * functest-odl-sfc * orchestra_clearwaterims * orchestra_openims * vyos_vrouter * juju_vepc External test cases integrated in previous versions but not released in Euphrates: * copper * moon * netready * security_scan The code to run these test cases is hosted in the repository of the project. Please note that orchestra test cases are hosted in Functest repository and not in orchestra repository. Vyos_vrouter and juju_vepc code is also hosted in functest as there are no dedicated projects. Functest framework ================== Functest is a framework. Historically Functest is released as a docker file, including tools, scripts and a CLI to prepare the environment and run tests. It simplifies the integration of external test suites in CI pipeline and provide commodity tools to collect and display results. Since Colorado, test categories also known as **tiers** have been created to group similar tests, provide consistent sub-lists and at the end optimize test duration for CI (see How To section). The definition of the tiers has been agreed by the testing working group. The tiers are: * healthcheck * smoke * features * components * vnf Functest abstraction classes ============================ In order to harmonize test integration, abstraction classes have been introduced: * testcase: base for any test case * unit: run unit tests as test case * feature: abstraction for feature project * vnf: abstraction for vnf onboarding The goal is to unify the way to run tests in Functest. Feature, unit and vnf_base inherit from testcase:: +----------------------------------------------------------------+ | | | TestCase | | | | - init() | | - run() | | - push_to_db() | | - is_successful() | | | +----------------------------------------------------------------+ | | | | V V V V +--------------------+ +---------+ +------------------------+ +-----------------+ | | | | | | | | | feature | | unit | | vnf | | robotframework | | | | | | | | | | | | | |- prepare() | | | | - execute() | | | |- deploy_orchestrator() | | | | BashFeature class | | | |- deploy_vnf() | | | | | | | |- test_vnf() | | | | | | | |- clean() | | | +--------------------+ +---------+ +------------------------+ +-----------------+ Functest util classes ===================== In order to simplify the creation of test cases, Functest develops also some functions that are used by internal test cases. Several features are supported such as logger, configuration management and Openstack capabilities (tacker,..). These functions can be found under <repo>/functest/utils and can be described as follows:: functest/utils/ |-- config.py |-- constants.py |-- decorators.py |-- env.py |-- functest_utils.py |-- openstack_tacker.py `-- openstack_utils.py TestAPI ======= Functest is using the Test collection framework and the TestAPI developed by the OPNFV community. See `[4]`_ for details. Reporting ========= A web page is automatically generated every day to display the status based on jinja2 templates `[3]`_. Dashboard ========= Additional dashboarding is managed at the testing group level, see `[5]`_ for details. ======= How TOs ======= See How to section on Functest wiki `[6]`_ ========== References ========== _`[1]`: http://artifacts.opnfv.org/functest/docs/configguide/index.html Functest configuration guide _`[2]`: http://artifacts.opnfv.org/functest/docs/userguide/index.html functest user guide _`[3]`: https://git.opnfv.org/releng-testresults/tree/reporting _`[4]`: http://docs.opnfv.org/en/latest/testing/testing-dev.html _`[5]`: https://opnfv.biterg.io/goto/283dba93ca18e95964f852c63af1d1ba _`[6]`: https://wiki.opnfv.org/pages/viewpage.action?pageId=7768932 IRC support chan: #opnfv-functest