aboutsummaryrefslogtreecommitdiffstats
path: root/functest/tests
AgeCommit message (Collapse)AuthorFilesLines
2019-03-19Do not allow testcase skipping in refstackJuha Kosonen1-0/+19
Set the result of refstack_defcore as failed if there were skipped testcases in a run. Change-Id: Id782c2dee511cefbafd5b4440f386fd66f934a47 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com> (cherry picked from commit cfd79ec4c39e215a25fc3d99fc5368e7d874f7bf)
2019-02-18Enrich Rally results outputStamatis Katsaounis1-4/+21
This patch adds success/failure information of rally test cases to summary output. Change-Id: Ie4f9b4972a26e187bb7afdddfd5ce51b16f3db12 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com> (cherry picked from commit 13777d607dd219ace0157e560ff11c3cd6f1aad0)
2019-02-15Generate xunit reports (rally and tempest)Cédric Ollivier3-52/+60
It adds xunit reports for rally-based and tempest-based testcases. It completes the reports provided by snaps (thanks to Xtesting). All rally related operations are moved to rally. It allows removing the rally dependency to tempest which was false. Change-Id: Ia7d2476f58f4f68b7c88442e50cad844037a36e9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3393f2016483555c27d612c69ec11274cc8aa72a)
2019-02-14Specify which rally tests to runStamatis Katsaounis1-6/+2
This patch adds the ability to specify which rally test to run by setting the appropriate configuration value. Change-Id: I6cbc9d12aada4f8eab68d2219d94ae1391a79021 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com> (cherry picked from commit d0cb38d7ad5046700564f2700df9b8c92706b32c)
2019-02-13Generate one single report for Rally testcasesCédric Ollivier1-0/+24
It eases sharing one single report including all task results. For the time being, rally_jobs only generates neutron.html. Then it only adds one report for rally_full and rally_sanity. We may consider generating only one json file but it asks for a refactoring (result parsing) which can be done in a second change. Change-Id: I8fa75ae80750336950f355c2aedc3dd4dc621cc5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 2df7a0728d687de9a24d8432ef30bbd2ecd771c8)
2019-01-20Move logics out of TempestCommon.__init__()Cédric Ollivier1-0/+1
It conforms with best programming practices. It allows muting console and increasing log level (debug). Change-Id: I0e0cfef6abe3b354a43817fcb242c71da701e237 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b482b494ce695e83cca0a10e7750200fd1be42ea)
2018-12-27Update Xtesting to 0.61Cédric Ollivier1-63/+100
It now generates html reports for unit tests (snaps). Change-Id: Ica8fab2b383723957c1068c5accbcab9568002ac Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 98ee1837b939f32c2c3bc17787065d0bf0a22c3f)
2018-10-21Harden Rally generate reportCédric Ollivier1-4/+2
It converts html_file to str and raises all possible exceptions. It sometimes fails when locally generating reports. Change-Id: Idf6a12aaac4561800cd5e364f8bcc495aabbee1d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4d3627e56d333e313b67bbf1e7880b11883b5652)
2018-10-16Merge "Do not create files in site-package directory" into stable/gambiaCedric Ollivier1-0/+3
2018-10-16Ease configuring extra specs via env varsDelia Popescu1-0/+3
Ease configuring flavor extra specs, as well as image extra specs. Set environment variables for image_properties and flavor extra_specs for ovs|fdio scenario from the start. Image extra_properties and flavor extra_specs will continue to get updates from functest config file but so will from IMAGE_PROPERTIES and FLAVOR_EXTRA_SPECS env variables. Change-Id: Ie1c69e66d2e2e158915d214402fe44d9991c56d3 Signed-off-by: Delia Popescu <delia.popescu@enea.com> (cherry picked from commit 1a00b9de56ef65eef2a0b269a514cb25491ac54b)
2018-10-16Do not create files in site-package directoryJuha Kosonen1-0/+3
Use /home/opnfv/functest/data/rally/task directory for storing temporary files when running rally tasks. Also removes unused vm scenario from task service list. Change-Id: Icfd4f99ebe6194360bda8bd88e497e018a9541a0 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com> (cherry picked from commit 520331a06750dd02228f840c346402f52fb864e8)
2018-10-14Fix image properties parsingCédric Ollivier2-20/+41
It removes useless spaces and now conforms with oslo.conf dict. New helpers and unit tests are available. Change-Id: I20335c5e5c3840cc872e158c6072d97a3f2cb98c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 904377462f58464937d4cafeb8c432f024a85bd4)
2018-09-20Add support for Rally OpenStack CI test casesJuha Kosonen1-4/+6
Include Neutron rally-jobs. Change-Id: Ic1122962bd7d876d179729640a8a321bc36442d8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com> (cherry picked from commit 5d54a533d25a989dc64a2789788ff2f2fd6ad5c6)
2018-09-08Merge "Fix dict conversion in tempest.conf"Cedric Ollivier1-2/+2
2018-09-08Fix dict conversion in tempest.confCédric Ollivier1-2/+2
https://github.com/openstack/oslo.config/blob/master/oslo_config/types.py#L600 https://build.opnfv.org/ci/view/fuel/job/functest-fuel-virtual-daily-master/817/consoleFull Change-Id: Icde9938bb9f6c0c37d2504317f31b0af4ef9ee8e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-07Do not raise exceptions in conf_utilsJuha Kosonen1-29/+5
Return a reasonable value for caller to process. Change-Id: I5a83544aaebb4e53db37ddaa13033d77848722b8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-09-05Update scenario section in tempest.confCédric Ollivier1-0/+20
It's required at least by Barbican tempest plugin. Change-Id: I46a65ee5fee044917c5375adaa8ebb529efe86ec Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-03Merge "Refactor RallyBase"Cedric Ollivier1-27/+32
2018-09-03Refactor RallyBaseJuha Kosonen1-27/+32
This provides a more generic way to integrate test case lists which are not hosted in Functest. Also removes vm scenarios since they have never been a part of actual rally runs. Change-Id: Ib0a020fe72800915bbf2d10ecc690a248d33c246 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-09-02Update Xtesting to 0.60Cédric Ollivier1-1/+1
Xtesting leverages on stevedore [1] which will bring much more flexibility (discovery, module or class renaming, etc). OPNFV Features have to be updated as well: - https://gerrit.opnfv.org/gerrit/61713 - https://gerrit.opnfv.org/gerrit/61715 - https://gerrit.opnfv.org/gerrit/61717 - https://gerrit.opnfv.org/gerrit/61719 [1] https://docs.openstack.org/stevedore/latest/ Change-Id: I0ce45eb6c9af66546c2c020eb962fa457f6c7e1a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-29Create new project/user for tempest testsJuha Kosonen2-15/+12
JIRA: FUNCTEST-1009 Change-Id: I6b9bca3827ba6cc804c25689a896ca63eec6a78b Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-28Merge "Remove installer type from rally blacklist"Cedric Ollivier1-10/+0
2018-08-28Remove installer type from tempest blacklistJuha Kosonen1-3/+0
Enables utilization of blacklist also on non-OPNFV environments since there is no link to installer type. JIRA: FUNCTEST-1008 Change-Id: I3a012b8b7e7dbd049f8794dff3c9d56dcc1cadad Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-28Remove installer type from rally blacklistJuha Kosonen1-10/+0
Enables utilization of blacklist also on non-OPNFV environments since there is no link to installer type. JIRA: FUNCTEST-1008 Change-Id: I3eb4377af153480794c1a71c5d7ef3e78dacacb9 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-25Remove INSTALLER_TYPE logics in odlCédric Ollivier1-34/+1
JIRA: APEX-621 JIRA: COMPASS-606 Change-Id: I76dde66aefc67f80b40496f705f2864ea9355dcf Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-24Create new project/user for snaps testsJuha Kosonen1-12/+30
JIRA: FUNCTEST-1003 Change-Id: Ic55998977386f95f619a355d22bd285782fe81f0 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-14Create new project/user for rally testJuha Kosonen2-7/+8
FUNCTEST-1002 Change-Id: Id19b9aa70aeded6d99978084b3de4a63e31a81ad Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-10Remove explicit Rally task validationJuha Kosonen1-14/+1
Utilize the output of Rally task execution since it internally validates the task. Use Popen.communicate to get the output from the running Rally task. Change-Id: I9b1299bc26129ff4f89adccec611c78195a3ab02 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-07Update Xtesting to 0.55.0Cédric Ollivier1-3/+31
It allows calling one skipped test via run_tests (parallel testing). Change-Id: I04e5c69f2129e28f1451815938cd5d0f8bbe9e44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-04Make synchronous Shade callsCédric Ollivier1-1/+1
It makes difficult to find the issues (floating ip, boot, ssh connection, etc...) for a small benefit regarding the current testcase implementations. It increases the default floating ip timeout because it has been reached when running Functest multiple times in parallel. It may be the root cause when running singlevm vs XCI (in that case, neutron api and agents are hosted in a nested vm). Change-Id: Ia97e04902321644c207c59987283754318aee5e5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-03Protect VMTP vs cidr conflictsCédric Ollivier1-0/+1
It now runs vtmp via a new admin member. Else it fails due to cidr already used. It only creates a router and both tenant networks are still created by VMTP. Change-Id: I05ed41a3b7a01ee97c720176c404929bf8ea97a0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-31Print stderr in logs when calling cmd over sshCédric Ollivier1-1/+1
Change-Id: I152ac898661a0e99a6232df86689c70c468ceaa5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-30Remove VnfOnBoardingCédric Ollivier1-196/+0
All the vnfs are now inherating from scenarios. Change-Id: I63509102067676ca0676b773db542086caa02d84 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-28Leverage vyos_vrouter on CloudifyCédric Ollivier2-61/+2
It also stop rebooting the VMs. Change-Id: Ib0f9e1aac1dd8505378b0688930eb41a590a11f1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-27Stop redirecting output to json filesCédric Ollivier1-0/+1
Rally may print additional data to console which results in an invalid json files. The json files are generated via rally instead of dumping stdout. It's mandatory to backport the next bugfix [1] in stable/fraser. [1] https://gerrit.opnfv.org/gerrit/#/c/60265/ Change-Id: Ifbd90e685a7b52c70b3110a46e3c7991eddc05f4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-23New Heat IMS testcaseValentin Boucher1-3/+3
In order to validate OpenStack Master deployment we create this new testcase using OpenStack Heat as an Orchestrator for Clearwater VNF JIRA: FUNCTEST-995 Change-Id: I8b7b74a3753c2d4d4614e9a2798283bd3f99d5d2 Signed-off-by: Valentin Boucher <valentin.boucher@kontron.com>
2018-07-21Add helpers to detect OpenStack versionsCédric Ollivier1-1/+121
It leverages on Nova API Microversions [1]. [1] https://docs.openstack.org/nova/latest/reference/api-microversion-history.html Change-Id: I60abf9089036e56fcbcd6c2b734666ea20875ceb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-20Stop Fuel logics in odlCédric Ollivier1-12/+0
Fuel is setting the right env vars [1]. [1] https://gerrit.opnfv.org/gerrit/#/c/59929/ Change-Id: Ieb0ae1887859e0b7d3f77dfb5100a633b6479a37 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-20Cover cinder testcasesDelia Popescu2-0/+271
JIRA: FUNCTEST-908 Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com> Change-Id: Ie9662877cba89ea96de4403ee2b736c118a044d5 Signed-off-by: Delia Popescu <delia.popescu@enea.com>
2018-07-18Remove logics about Daisy in odlCédric Ollivier1-9/+2
Daisy is now setting the right env vars [1]. [1] https://gerrit.opnfv.org/gerrit/#/c/59839/1 Change-Id: If23c0ac0b5ab2f4f4b9515e27d845cb9c4d07ded Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-17Leverage cloudify_ims on CloudifyCédric Ollivier2-49/+2
It also defines 2 new methods to avoid duplicating code: - TenantNetwork1.get_public_auth_url() - VmReady1.publish_image_alt() Shaker and Tempest are updated as well. Change-Id: Ie9095c57c580b95f067d5a8275ad71818b4df335 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-12Protect vs former interface formatCédric Ollivier1-2/+11
Compass still set publicURL [1] [1] https://build.opnfv.org/ci/view/functest/job/functest-compass-baremetal-daily-fraser/lastFailedBuild/consoleFull Change-Id: I88c6ca336a7017f97f4c07719156e567d1afb9df Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-11Clean all possible resourcesCédric Ollivier1-3/+11
It checks all openstack resources before cleaning. It allows removing images even if the test fail quickly. Change-Id: I1763b8a70fa7ffeb03c435e8e775c7afd1a7adac Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-11Check the status of inherited stepsCédric Ollivier1-1/+4
Change-Id: If63c228f2844194279b560515f4ab24a4ee88be0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-11Update rally image name regexCédric Ollivier1-1/+2
The default value doesn't conform with our image names. Else it downloads the image from internet which may be not possible (offline testing, proxy or deployment l3 issues [1]) [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-virtual-suite-master/113/console Change-Id: I7d951967784d591e7155fe8582299aaa9b680b10 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-09Merge "Refactor tempest common"Cedric Ollivier1-18/+13
2018-07-09Add Vmtp as a new testcaseCédric Ollivier2-0/+88
VMTP is a small python application that will automatically perform ping connectivity, round trip time measurement (latency) and TCP/UDP throughput. JIRA: FUNCTEST-980 Change-Id: I8e94fc52a429f505ac14e14bbb9d0a3b20996ba4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-08Refactor tempest commonCédric Ollivier1-18/+13
It also renames two testcases and updates all docs because tests run in parallel in case of tempest smoke. To force one thread as before: run: module: 'functest.opnfv_tests.openstack.tempest.tempest' class: 'TempestCommon' args: mode: '^tempest\.(api|scenario).*\[.*\bsmoke\b.*\]$' option: - '--concurrency=1' JIRA: FUNCTEST-989 Change-Id: I1348f5908400a5046be8edf983db4afa12e546c3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-07Cover vping_sshCédric Ollivier1-0/+123
Change-Id: I37c52057a45d76e081a1a0145baee96041ca25a5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-30Inherit rally from VmReady1Cédric Ollivier2-216/+19
It also fixes clean(). Change-Id: I42d1305d5cdb5a717cfa7fce26798ac7102310f7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>