aboutsummaryrefslogtreecommitdiffstats
path: root/functest/tests
AgeCommit message (Collapse)AuthorFilesLines
2019-10-23Allow blacklisting tests if public endpoint onlyCédric Ollivier1-2/+4
It allows reusing the blacklisting system if only public endpoints are reachable (ONAP Openlab testing). It contains a first list of tempest-based tests which force the use of admin endpoints. Change-Id: Ic1ab07dec3ae1f2a0ca50b80e08cd0ac396f2e56 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-19Update Flake8Cédric Ollivier1-0/+1
It avoids the next incompatibility: ERROR: flake8 2.5.5 has requirement mccabe<0.5,>=0.2.1, but you'll have mccabe 0.6.1 which is incompatible. Change-Id: Ic3bb14b032adfee64109549abc0a09d8cc86968f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-11Leverage on Rally task tagsCédric Ollivier1-9/+13
It avoids parsing rally task outputs which may hang (subprocess communicate) as seen in rally_full and rally_jobs. It simply selects test names as tags. Change-Id: I88b54a8f155e557f8a606fdbd7d86c1f4d5dae3b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-08Count all hypervisors by defaultCédric Ollivier1-1/+1
If one hypervisor is down, few tempest test fails. User is free to set SKIP_DOWN_HYPERVISORS if it's down on purpose. Change-Id: I7b6a4d0d8f67755d8c1550fd1bc6fd707634f68b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-01Merge "Count all active hypervisors"Cedric Ollivier1-2/+2
2019-09-30Count all active hypervisorsCédric Ollivier1-2/+2
Else tempest_slow fails if one hypervisor is down (detected in ONAP Openlab). It also improves Shaker, Rally and Vmtp which count them. Change-Id: Iee7c20e0357d9237501c5c451580d8f6409ac86b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-29Detect OpenStack TrainCédric Ollivier1-0/+8
It takes into account latest max nova microversions [1] [1] https://docs.openstack.org/nova/latest/reference/api-microversion-history.html Change-Id: Icb9838b09ad22f6e87e9d8fb3c47525bdc1e3307 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-07-27Switch to Python 3.7 and Alpine 3.10Cédric Ollivier3-9/+9
It also allows building docs as doc8 is broken due to latest OpenStack's upper-constraints. pylint is updated to 2.3.1 (lastest py3.7 version) It disables perm as umask is currently false on lf-virtual1. Change-Id: If09d4796d48b7d0591e7926621d3bbf5ca1b6c24 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-06-02Publish rally logsCédric Ollivier1-3/+14
Both Rally and Tempest publish Rally debug logs. It also moves the logics from tempest to rally. Change-Id: I5c057d830202baddd73577ade1b09ce304e3c5a5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-05-25Set utf-8 in decode and encode callsCédric Ollivier2-7/+7
It eases backporting in iruya and hunter as a few logs can't be decoded as ascii [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-vnf-iruya-juju_epc-run/56/console Change-Id: I52a6f9a25130cbf62e8ce3b9972064ab73177665 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-05-24Decode Bytes in logging callsCédric Ollivier1-3/+6
It fixes a few incorrect types in log calls [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-latest-daily/163/ Change-Id: I01790f6454694f434d31366f9c1cf9d98ff5a9cf Signed-off-by: Cédric Ollivier <ollivier.cedric@gmail.com>
2019-05-21Avoid running local dnsmasq when testing IMSCédric Ollivier1-1/+2
It may be unsafe and it's useless for testing IMS if BONO_IP is set. Change-Id: I8b0f010956defbfa1fd22b5aa5cbea70bd2968a4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-05-13Update to Python3Cédric Ollivier3-59/+29
Functest containers leverage on Python3 instead of python2. https://mail.python.org/pipermail/python-dev/2018-March/152348.html It also updates robotframework librairies to latest release and Vmtp to master ([1] is needed) It patches cloudify rest client to support python3. Vmtp is currently disabled because it currently supports python2 only. [1] https://github.com/openstack/vmtp/commit/a5d062881d91bf4f547d92c6e289bea30feb5d6e#diff-b4ef698db8ca845e5845c4618278f29a Change-Id: I39964a212ec2d4dbf53c9ea7c63f02e0b6a05b48 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-04-20Add Stein in Nova API microversionsCédric Ollivier1-0/+8
Change-Id: I42e854122fde9495cb3472ccc78900eccb20670d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-03-19Do not allow testcase skipping in refstackJuha Kosonen1-0/+19
Set the result of refstack_defcore as failed if there were skipped testcases in a run. Change-Id: Id782c2dee511cefbafd5b4440f386fd66f934a47 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2019-03-12Remove Snaps-based testcasesCédric Ollivier5-575/+0
Snaps hasn't been synchronized for a while regarding requirements. We do remove it due to the inactivity. All test result tabs will be updated in a second change. Change-Id: I834afd902829ed3883b0e88e92aa806ec43d6fcf Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-03-07Merge tempest.py and conf_utils.pyJuha Kosonen2-162/+133
Change-Id: I597abdbb5bd3de023aa7b66fa27764ca2ec27f0d Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2019-02-15Enrich Rally results outputStamatis Katsaounis1-4/+21
This patch adds success/failure information of rally test cases to summary output. Change-Id: Ie4f9b4972a26e187bb7afdddfd5ce51b16f3db12 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-02-14Generate xunit reports (rally and tempest)Cédric Ollivier3-52/+60
It adds xunit reports for rally-based and tempest-based testcases. It completes the reports provided by snaps (thanks to Xtesting). All rally related operations are moved to rally. It allows removing the rally dependency to tempest which was false. Change-Id: Ia7d2476f58f4f68b7c88442e50cad844037a36e9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-02-13Specify which rally tests to runStamatis Katsaounis1-6/+2
This patch adds the ability to specify which rally test to run by setting the appropriate configuration value. Change-Id: I6cbc9d12aada4f8eab68d2219d94ae1391a79021 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-02-12Generate one single report for Rally testcasesCédric Ollivier1-0/+24
It eases sharing one single report including all task results. For the time being, rally_jobs only generates neutron.html. Then it only adds one report for rally_full and rally_sanity. We may consider generating only one json file but it asks for a refactoring (result parsing) which can be done in a second change. Change-Id: I8fa75ae80750336950f355c2aedc3dd4dc621cc5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-01-19Move logics out of TempestCommon.__init__()Cédric Ollivier1-0/+1
It conforms with best programming practices. It allows muting console and increasing log level (debug). Change-Id: I0e0cfef6abe3b354a43817fcb242c71da701e237 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-12-27Update Xtesting to 0.61Cédric Ollivier1-63/+100
It now generates html reports for unit tests (snaps). Change-Id: Ica8fab2b383723957c1068c5accbcab9568002ac Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-12-19Add task arguments to rally_jobsJuha Kosonen1-4/+4
Define floating network as a Rally task argument. Enable rally_jobs subtest. Change-Id: I37d7c4718cfb3de3ad1980878e0529d652013ccb Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-10-21Harden Rally generate reportCédric Ollivier1-4/+2
It converts html_file to str and raises all possible exceptions. It sometimes fails when locally generating reports. Change-Id: Idf6a12aaac4561800cd5e364f8bcc495aabbee1d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-10-15Merge "Ease configuring extra specs via env vars"Cedric Ollivier1-0/+3
2018-10-15Ease configuring extra specs via env varsDelia Popescu1-0/+3
Ease configuring flavor extra specs, as well as image extra specs. Set environment variables for image_properties and flavor extra_specs for ovs|fdio scenario from the start. Image extra_properties and flavor extra_specs will continue to get updates from functest config file but so will from IMAGE_PROPERTIES and FLAVOR_EXTRA_SPECS env variables. Change-Id: Ie1c69e66d2e2e158915d214402fe44d9991c56d3 Signed-off-by: Delia Popescu <delia.popescu@enea.com>
2018-10-15Do not create files in site-package directoryJuha Kosonen1-0/+3
Use /home/opnfv/functest/data/rally/task directory for storing temporary files when running rally tasks. Also removes unused vm scenario from task service list. Change-Id: Icfd4f99ebe6194360bda8bd88e497e018a9541a0 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-10-14Fix image properties parsingCédric Ollivier2-20/+41
It removes useless spaces and now conforms with oslo.conf dict. New helpers and unit tests are available. Change-Id: I20335c5e5c3840cc872e158c6072d97a3f2cb98c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-18Add support for Rally OpenStack CI test casesJuha Kosonen1-4/+6
Include Neutron rally-jobs. Change-Id: Ic1122962bd7d876d179729640a8a321bc36442d8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-09-14Update get_openstack_version()Cédric Ollivier1-2/+2
It takes Rocky maximum version into account. Change-Id: I55469de4a63f1f824222660130a20d7fd8fa42c3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-08Merge "Fix dict conversion in tempest.conf"Cedric Ollivier1-2/+2
2018-09-08Fix dict conversion in tempest.confCédric Ollivier1-2/+2
https://github.com/openstack/oslo.config/blob/master/oslo_config/types.py#L600 https://build.opnfv.org/ci/view/fuel/job/functest-fuel-virtual-daily-master/817/consoleFull Change-Id: Icde9938bb9f6c0c37d2504317f31b0af4ef9ee8e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-07Do not raise exceptions in conf_utilsJuha Kosonen1-29/+5
Return a reasonable value for caller to process. Change-Id: I5a83544aaebb4e53db37ddaa13033d77848722b8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-09-05Update scenario section in tempest.confCédric Ollivier1-0/+20
It's required at least by Barbican tempest plugin. Change-Id: I46a65ee5fee044917c5375adaa8ebb529efe86ec Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-03Merge "Refactor RallyBase"Cedric Ollivier1-27/+32
2018-09-03Refactor RallyBaseJuha Kosonen1-27/+32
This provides a more generic way to integrate test case lists which are not hosted in Functest. Also removes vm scenarios since they have never been a part of actual rally runs. Change-Id: Ib0a020fe72800915bbf2d10ecc690a248d33c246 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-09-02Update Xtesting to 0.60Cédric Ollivier1-1/+1
Xtesting leverages on stevedore [1] which will bring much more flexibility (discovery, module or class renaming, etc). OPNFV Features have to be updated as well: - https://gerrit.opnfv.org/gerrit/61713 - https://gerrit.opnfv.org/gerrit/61715 - https://gerrit.opnfv.org/gerrit/61717 - https://gerrit.opnfv.org/gerrit/61719 [1] https://docs.openstack.org/stevedore/latest/ Change-Id: I0ce45eb6c9af66546c2c020eb962fa457f6c7e1a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-29Create new project/user for tempest testsJuha Kosonen2-15/+12
JIRA: FUNCTEST-1009 Change-Id: I6b9bca3827ba6cc804c25689a896ca63eec6a78b Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-28Merge "Remove installer type from rally blacklist"Cedric Ollivier1-10/+0
2018-08-28Remove installer type from tempest blacklistJuha Kosonen1-3/+0
Enables utilization of blacklist also on non-OPNFV environments since there is no link to installer type. JIRA: FUNCTEST-1008 Change-Id: I3a012b8b7e7dbd049f8794dff3c9d56dcc1cadad Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-28Remove installer type from rally blacklistJuha Kosonen1-10/+0
Enables utilization of blacklist also on non-OPNFV environments since there is no link to installer type. JIRA: FUNCTEST-1008 Change-Id: I3eb4377af153480794c1a71c5d7ef3e78dacacb9 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-25Remove INSTALLER_TYPE logics in odlCédric Ollivier1-34/+1
JIRA: APEX-621 JIRA: COMPASS-606 Change-Id: I76dde66aefc67f80b40496f705f2864ea9355dcf Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-24Create new project/user for snaps testsJuha Kosonen1-12/+30
JIRA: FUNCTEST-1003 Change-Id: Ic55998977386f95f619a355d22bd285782fe81f0 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-14Create new project/user for rally testJuha Kosonen2-7/+8
FUNCTEST-1002 Change-Id: Id19b9aa70aeded6d99978084b3de4a63e31a81ad Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-10Remove explicit Rally task validationJuha Kosonen1-14/+1
Utilize the output of Rally task execution since it internally validates the task. Use Popen.communicate to get the output from the running Rally task. Change-Id: I9b1299bc26129ff4f89adccec611c78195a3ab02 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-07Update Xtesting to 0.55.0Cédric Ollivier1-3/+31
It allows calling one skipped test via run_tests (parallel testing). Change-Id: I04e5c69f2129e28f1451815938cd5d0f8bbe9e44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-04Make synchronous Shade callsCédric Ollivier1-1/+1
It makes difficult to find the issues (floating ip, boot, ssh connection, etc...) for a small benefit regarding the current testcase implementations. It increases the default floating ip timeout because it has been reached when running Functest multiple times in parallel. It may be the root cause when running singlevm vs XCI (in that case, neutron api and agents are hosted in a nested vm). Change-Id: Ia97e04902321644c207c59987283754318aee5e5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-03Protect VMTP vs cidr conflictsCédric Ollivier1-0/+1
It now runs vtmp via a new admin member. Else it fails due to cidr already used. It only creates a router and both tenant networks are still created by VMTP. Change-Id: I05ed41a3b7a01ee97c720176c404929bf8ea97a0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-31Print stderr in logs when calling cmd over sshCédric Ollivier1-1/+1
Change-Id: I152ac898661a0e99a6232df86689c70c468ceaa5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>