summaryrefslogtreecommitdiffstats
path: root/functest/opnfv_tests/openstack/tempest/tempest.py
AgeCommit message (Collapse)AuthorFilesLines
2018-09-23Protect vs former OS env varsCédric Ollivier1-0/+5
rally_jobs is failing vs APEX which still sets OS_TENANT [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-master/324/console Change-Id: I672bfa93b37b1b773ea7d20fdd91bea0ff14b79f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 83e0e6836d7f5e18eb4eb8c9d28b4e6980b790c2)
2018-09-23Set default_domain_id in tempest.confCédric Ollivier1-1/+2
It allows verifying via non-default domain. Change-Id: I95ae26fd3bfd1c10421c02d2b4f6e8a9b6aa49e6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 11d28aaf2e726859b59b7511d64dc4dd4b422e1e)
2018-09-22Set admin_domain_scope to True (tempest)Cédric Ollivier1-5/+5
It completes the previous commit [1]. It also sets cidr and fully forces Keystone v3. [1] https://gerrit.opnfv.org/gerrit/#/c/62777/ Change-Id: Ida948a4ff051dc1d1325d29a815c201366a1e33e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f4b545efe2339bdb5f6117d484e127adda5c8021)
2018-09-22Grant new tempest user as admin in domain tooCédric Ollivier1-0/+3
It seems mandatory for test_list_projects_returns_only_authorized_projects Change-Id: Id994e66b088d751c4f6d7f260f8e482c2ee83307 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 17e87d5af713bb07759c4b2af0c2b40077343a32)
2018-09-22Stop creating network resources in tempestCédric Ollivier1-2/+4
fixed_network_name is not used in neutron or nova gates [1]. It may bypass juju bug (subnet discovery). [1] http://logs.openstack.org/46/603046/1/check/tempest-full/a8468a1/controller/logs/tempest_conf.txt Change-Id: I820e8c723918ed683ed1f8016363eb70bad0d14a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit ac31a32320122ea1683fe9884e5bf1f969678b03)
2018-09-08Update scenario section in tempest.confCédric Ollivier1-0/+24
It's required at least by Barbican tempest plugin. Change-Id: I46a65ee5fee044917c5375adaa8ebb529efe86ec Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9ec639526ed32d4b05168627c5c016fb66b7d008)
2018-08-29Create new project/user for tempest testsJuha Kosonen1-1/+22
JIRA: FUNCTEST-1009 Change-Id: I6b9bca3827ba6cc804c25689a896ca63eec6a78b Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com> (cherry picked from commit 9d03fcdd488c385842b43fd914915cbe4ae5390a)
2018-08-29Support pre-Provisioned credentialsCédric Ollivier1-1/+2
If USE_DYNAMIC_CREDENTIALS = False, Functest is expected for /home/opnfv/functest/data/accounts.yaml which contains the set of pre-existing credentials to use for testing [1]. It also sets admin_domain_scope and default_credentials_domain_name. [1] https://docs.openstack.org/tempest/latest/configuration.html Change-Id: Ie7b81f0c56049260ca99b8e3a6d58cb8a2660659 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7e0f65a25f1c83c90051e13e4aa5333aaaa8c6d5)
2018-08-23Redirect stderr to stdout when running tempestCédric Ollivier1-8/+4
JIRA: FUNCTEST-999 Change-Id: I64bfa31b01aa2ce5444b6e0f0ac46f8cde8dbaa3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c4b3f91c6bb79c50adc1c5a4397666ebcd1f7375)
2018-08-23Create output directoryGeorg Kunz1-0/+2
The rally.conf gets copied before the target directory has been created. JIRA: FUNCTEST-1007 JIRA: DOVETAIL-730 Change-Id: Ie774fce994ada644c1163600e6c62e1231fde443 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com> (cherry picked from commit b55aa749c379ce7af918f43558f2195e2259d872)
2018-08-22Fix fixed_network_name in tempest.confCédric Ollivier1-1/+7
It also copies rally.conf and tempest.log in res_dir. Change-Id: Ib907a29ba3f47ca861d3af8afc0ee74446222e89 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0fd76322f95b755d347826f707c092105506311b)
2018-08-07Fix swift operator roleCédric Ollivier1-1/+1
[1] https://build.opnfv.org/ci/view/functest/job/functest-fuel-armband-baremetal-daily-master/35/console Change-Id: I7236bf5a74f36941f499da32c5c77dcd0b98b6f1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3d1986d1a8035e93b7ad7e4331cb1cbe126a857f)
2018-07-30Delete alt images before removing the accountCédric Ollivier1-1/+1
Else it could raise unauthorized exceptions [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-master/188/console Change-Id: Ia9b857069f775e2c8ecc5e07cf39b3a1fb7e3540 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 28a706149ede8c644f63c7b5a4f3b02a9806e548)
2018-07-27Set filename_alt in tempestCédric Ollivier1-0/+1
Change-Id: I9137ee30bd2107ec23ef6cfea92ccfc5f010410d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 29bd4108c51180a4d74443f676fc72866af6f1f6)
2018-07-27Leverage cloudify_ims on CloudifyCédric Ollivier1-2/+1
It also defines 2 new methods to avoid duplicating code: - TenantNetwork1.get_public_auth_url() - VmReady1.publish_image_alt() Shaker and Tempest are updated as well. Change-Id: Ie9095c57c580b95f067d5a8275ad71818b4df335 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8e96ddfb6a6548a9a7a9b4a686e01378425a1cd8)
2018-07-27Refactor tempest commonCédric Ollivier1-83/+17
It also renames two testcases and updates all docs because tests run in parallel in case of tempest smoke. To force one thread as before: run: module: 'functest.opnfv_tests.openstack.tempest.tempest' class: 'TempestCommon' args: mode: '^tempest\.(api|scenario).*\[.*\bsmoke\b.*\]$' option: - '--concurrency=1' JIRA: FUNCTEST-989 Change-Id: I1348f5908400a5046be8edf983db4afa12e546c3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit eb8f57466325f398c6b8b5f4d15005c71879a269)
2018-07-26Set swift_operator_role if requiredCédric Ollivier1-0/+14
Apex creates member as default role which conflicts with the default swift_operator_role [1]. It detects the default role in lowercase as well and write it in rally.conf if required. [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-virtual-suite-master/116/console Change-Id: Ic4dc98c793207fc32ff35852472742c92db6699c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit bafbd217c68c70ae66e052833c3e33298b35da0c)
2018-07-12Avoid duplicating tempest section in rally.confCédric Ollivier1-1/+2
Change-Id: I79598cce2c7dfe0a2e969ddd32735c3eb189fa7a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8b912cec489a96178fcac5667c03e20850149c3a)
2018-07-11Add tempest section in rally.confCédric Ollivier1-0/+1
It completes the previous patch [1][2]. [1] https://gerrit.opnfv.org/gerrit/#/c/59681/ [2] https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-master/555/console Change-Id: I93ad9db2c7a8a0691fed8bf58f5e21d8f3dc9c57 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit bf39607ee1128f6a9b46c37c0a8ac1e582d39ea5)
2018-07-11Clean all possible resourcesCédric Ollivier1-2/+4
It checks all openstack resources before cleaning. It allows removing images even if the test fail quickly. Change-Id: I1763b8a70fa7ffeb03c435e8e775c7afd1a7adac Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 6166caffe31888ef237874fdcaf088ac4ff8d003)
2018-07-11Check the status of inherited stepsCédric Ollivier1-1/+2
Change-Id: If63c228f2844194279b560515f4ab24a4ee88be0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4fb7e1330137fb77ed1b29bbc7c1491b6e12d9b6)
2018-07-11Force result = 0 if exceptionCédric Ollivier1-0/+1
self.result can be positive due to intermediary steps (scenarios). Else it can return false positives [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-virtual-suite-master/113/console Change-Id: Iecadee7f7380fda5273a0668aff1b441647c45aa Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f83c5fc1f0ecd3b93a39b3cf90ed9cd478cb37a6)
2018-07-11Update rally image name regexCédric Ollivier1-0/+11
The default value doesn't conform with our image names. Else it downloads the image from internet which may be not possible (offline testing, proxy or deployment l3 issues [1]) [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-virtual-suite-master/113/console Change-Id: I7d951967784d591e7155fe8582299aaa9b680b10 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8c5703e4e2382f7091aa7f8008d65ce68fa0bf56)
2018-07-03Inherit rally from VmReady1Cédric Ollivier1-4/+1
It also fixes clean(). Change-Id: I42d1305d5cdb5a717cfa7fce26798ac7102310f7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 43d6672e637a5b5b68ac4c382140996b4b9b17da)
2018-07-03Improve scenarios to reduce duplicate codesCédric Ollivier1-23/+4
Change-Id: I71f79b9ac8314ac979d4b1009f662baa7c93a4fd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit d2bf2e00eaedb5113f4baee3a707aa026dd1c18d)
2018-07-03Fix user/project create operationsCédric Ollivier1-3/+8
os_client_config creates a singleton which forbids the previous processing. It also cleans remaining resources and allows publishing public images (needed by Tempest) Change-Id: I3a0ad8664b263d3ee8599efe2973465006d5750b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c717c43b1257e126296a4b1d697c8f6a72728855)
2018-07-03Update tempest to inherit from VmReady1Cédric Ollivier1-138/+46
It also copies tempest.conf due to side effects raised by Barbican plugins. Change-Id: Ie87680f5d2e68527a5a61aac302838afc339dde8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9ef0bcbeb22ca70ff2ceb750c08fb24e46a9d0ea)
2018-07-03Publish tenantnetwork scenariosCédric Ollivier1-3/+5
They ease creating all tenant network ressources. These scenarios should be reused by other more complex ones. Change-Id: I4ce1872065bde92408d34f2ca8823ce98275d8d5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 979161874ba837f55a53aba391966c52433123e6)
2018-05-24Set image metadata in tempestCédric Ollivier1-1/+2
It applied the same operations as [1]. Only one config is used between rally, tempest and vping. [1] https://gerrit.opnfv.org/gerrit/#/c/57613/ Change-Id: I4d5fd2117d404b7eaa79326e21a35272232d0b92 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 46d61dac3e306249e729f45d25e33c3b4d7986e9)
2018-05-21Switch to project domain nameCédric Ollivier1-3/+5
At least Fuel, Apex and Joid don't export OS_PROJECT_DOMAIN_ID. https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-master/353/console Change-Id: I8441d2de1841d5a9ce78c6929fd78da01c45548c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 084dd47cdeb68f150165e009877ed995cb9db6dc)
2018-05-21Rewrite TempestResourcesManagerCédric Ollivier1-135/+97
It now leverages on OpenStack Shade to allow gating OpenStack master as asked by TSC. Switching to OpenStack master (OpenStack SDK Rocky) requires to change only one line. It stops dedicating the network to the user possibly created because the support is incomplete (the users have to be registed in tempest.conf and the multiprocessing has to be enforced). It also writes the second image and the second flavor in tempest.conf. Conflicts: functest/opnfv_tests/openstack/tempest/tempest.py functest/tests/unit/openstack/tempest/test_conf_utils.py functest/tests/unit/openstack/tempest/test_tempest.py Change-Id: I9fe2c90a6073ebc240e9aa8fdf2ef93d5dd5f15f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9fb702111ae63824a2e293d327437d0486a565d2)
2018-05-07Functest fail to parse tempest-based resultsxudan1-3/+6
1. The regular expression can't work well for all tempest-based results 2. Refer to http://testresults.opnfv.org/test/api/v1/results?project=functest&case_name=refstack_defcore&last=1 3. There are 88 success test cases but the list of success is empty 4. tempest test case name is not always begin with {0}, it maybe any number 5. The test case whose name has the keyword 'fail' will also be list in 'failures' Change-Id: I5187aaaabdca762256be9162cc85ba42263169bc Signed-off-by: xudan <xudan16@huawei.com> (cherry picked from commit 9286e052afe194c400479275edaa2aaf8ed52de8)
2018-04-12Fix role processing in PatroleCédric Ollivier1-2/+12
Role has to be listed in tempest.conf before calling testr. It also saves the right config file. Change-Id: Ifd122f31d0bfe3e9b3f93c6d62526acce96953d3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 71c4b3b719289929c226cae60eb1b23ada16eeb2)
2018-04-12Pass kwargs in TempestCommon:configure()Cédric Ollivier1-1/+1
It's required by Patrole. Change-Id: I07eb6f05b4776c3f9929b6f08c5cbfa131e09d8e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 51acd93bfe71e2cbd897f071e58b641ccd5ca1ac)
2018-03-20Update the details of tempest resultsLinda Wang1-4/+6
Change-Id: Idb18c7784e44030a48208ea64c21cde2536a3387 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2018-03-17Mute all rally messagesCédric Ollivier1-7/+4
The are now printed in functest.log only. JIRA: FUNCTEST-950 Change-Id: I1fdd0e0f1be19fc5e7536326acb6a66645610696 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-16Precise tempest regexCédric Ollivier1-1/+1
It stops running neutron tests which matched the former regex. Change-Id: Ica5a0d1f4858d01533156c98da87f9ed4662489f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-16Set api_extensions = all for NeutronTrunk onlyCédric Ollivier1-1/+9
Change-Id: I6c019094a9cea86f5990752463a3690d4f00df0c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-15Create 2 images and flavors for tempest testsLinda Wang1-28/+13
Also 1. remove the obsolete function: generate_test_accounts_file 2. Use patch mechanism to get config Change-Id: I455ac37de71519bcaf9a51e5f1de1b9bc83b9ba6 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2018-03-14Refactor TempestCommon to simplify subclassesCédric Ollivier1-18/+25
Change-Id: I8d265b6939861fd9d02a3d0f31aef53b9bfcb035 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-13Implement trunk ports testcasePanagiotis Karalis1-0/+13
The tempest testcases have been integrated from Neutron trunk testcases. It also enables every Neutron extension which may raise side effects [1]. [1] https://docs.openstack.org/tempest/latest/sampleconf.html JIRA: FUNCTEST-926 Change-Id: I05015acf119f504ce0a5539d677719de874594ad Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-03-13Fix tempest_customLinda Wang1-1/+1
Change-Id: I684c06eae899d9bc468e0be6a1537f9c942e86ce Signed-off-by: Linda Wang <wangwulin@huawei.com>
2018-03-10Improve tempest files managementCédric Ollivier1-1/+4
Calling apply_tempest_blacklist() is no longer mandatory. Change-Id: Id9a4e970c8acca066a36be89920745127867cd63 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-07Allow overriding tempest output dirCédric Ollivier1-12/+18
Change-Id: I31a5d2772bab168b3cb4e6b0713d6e95aa5b6fd7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-07Add a new env var for nameserverLinda Wang1-1/+2
We should still consider that the nameserver must be handled by all installers instead [1]: - by configuring one or more DNS resolvers in dhcp_agent.ini - by asking a local resolv It eases setting the nameserver in testcases and then allows gating them. 8.8.8.8 is still defined by default. [1] https://docs.openstack.org/neutron/pike/admin/config-dns-res.html Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com> Change-Id: Id58dd5ebee612a0fce2ebc40f089769e60899416 Signed-off-by: Linda Wang <wangwulin@huawei.com> Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-04Support domains when creating project/userLinda Wang1-2/+4
JIRA: FUNCTEST-945 Change-Id: Ibfce5a7d2ebb59a83ad5b2b6344c2fc87d7f3e84 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2018-03-02Leverage on XtestingCédric Ollivier1-11/+8
It removes all the files which have moved to Xtesting. Vnf inheritances and env management will be improved in other changes. It keeps the same tree thanks to a symlink to allow publishing artifacts. Change-Id: I551bbd3f344cdab0158a50b7b09e541576695631 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-24Automatically download defcore.txtCédric Ollivier1-4/+1
It stops writing files in Functest package dirs and cleans instance variables in refstack modules too. Change-Id: Iaddbe4fbaf12d1af207b86d4e44258efdc6d6f3a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-22Switch from CONST to CONFCédric Ollivier1-23/+27
It also removes constants.CONST and env.Environment which are now useless. Depends-On: I764a0a2a24447c941d1e726f3116593b29dd1c1e Depends-On: I6cfa832466dcefd737314633d807512e46267a69 Change-Id: Ife41c59d9f2e6ec4e49df38af962039f99554bc5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-19Generate tempest verification report (HTML)Juha Kosonen1-2/+12
JIRA: FUNCTEST-934 Change-Id: I2befbd3f2b2311f69b24321c7e9544fb43df3a9d Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>