aboutsummaryrefslogtreecommitdiffstats
path: root/functest/ci/testcases.yaml
AgeCommit message (Collapse)AuthorFilesLines
2017-05-30Define create_snapshot() and clean() in TestCaseCédric Ollivier1-33/+0
They replace the former calls in run_tests which are not suitable for all test cases. Now any test case can define how to clean its resources. If the snapshot cannot be created, the test case is considered as failed. Only a message is printed if any failure during cleaning. It also defines a new class called OSGCTestCase useful for test cases which don't clean their OpenStack resources. All test cases located in opnfv_tests/openstack inherit from it to keep the global behavior unchanged. It also deletes exit instructions in openstack_clean and openstack_snapshot, removes clean flags in testcases.yaml and updates the related utils. All Docs are modified as well. JIRA: FUNCTEST-438 Change-Id: I8938e6255708012380389763a24059ace4ce45d8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-29Merge "Re-Enable Promise testcases"Morgan Richomme1-1/+0
2017-05-19Activating SDNVPN test cases for master.Tim Irnich1-1/+0
Change-Id: I4ba995d412bfeadc708d5a3c643576097867d1ac Signed-off-by: Tim Irnich <tim.irnich@ericsson.com>
2017-05-19Re-Enable Promise testcasesGerald Kunzmann1-1/+0
Change-Id: I438c8f5ad9a517a19b675205964a427e8b57ac44 Signed-off-by: Gerald Kunzmann <kunzmann@docomolab-euro.com>
2017-05-16Add enabled flag in testcases.yamljose.lausuch1-16/+43
This will allow enable/disable test cases directly using true/false. Change-Id: Iaa7d051ebb8dd9e5a6f835a823f58f1e17a87b8d Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-05-03Updated basic attributes for opera testcasehelenyao1-3/+3
The original value as unknown is not reasonable. Change-Id: I23672ea70fff9b3badb2d2cf4f32dc3e038d7bb3 Signed-off-by: helenyao <yaohelan@huawei.com>
2017-04-26Manage criteria in TestCaseCédric Ollivier1-33/+33
It converts all criteria values to the corresponding percent in functest/ci/testcases.yaml. Result is expected to be equal or greater than criteria. If both are 0, result is considered as false. It is compatible with the old behavior but warns to update. It will allow a safer remove. It also fixes a bug in test_tempest to allow merging [1] and tier_handler.py which required that type criteria was str. [1] https://gerrit.opnfv.org/gerrit/#/c/27949/ Change-Id: Ib6edcfa3103b7d51b0bdc83119f1cea2a8be9fbc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-20Remove exec right for ci config filesCédric Ollivier1-0/+0
Change-Id: Ibf7e549252234fb69dd6e0cc7920b66759469aea Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-14Add copper section in testcases.yamlAimee Ukasick1-0/+17
JIRA: COPPER-34 Change-Id: I7301745718b9c035493c2e5c162e2a4e117982db Signed-off-by: Aimee Ukasick <aimeeu.opensource@gmail.com>
2017-04-14Remove deprecated feature modulesCédric Ollivier1-16/+32
All features running bash programs are now simply defined in testcases.yaml. All deprecated modules and unit tests are removed. Change-Id: I7047b6f7a1e43cb8ed5ba2d569d5dcecae68fb86 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-11Modify TestCase constructor attributesCédric Ollivier1-32/+64
Every feature/testcase now allows receiving the data defined in testcases.yaml as args (name is renamed to case_name). From the time being, only project and case names are handled. Next pending patches will add criteria, cmd and repo to this list. It keeps the default values for case names except for features which will be aggregated into Feature. Change-Id: Id742d100b8183d7f10894c24ae6879d1b2b60ac9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-08Write feature case names in testcases.yamlLinda Wang1-5/+5
It replaces some project names by the corresponding case names. Else some feature projects results failed to be pushed to DB. Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com> JIRA: FUNCTEST-792 Change-Id: Id81644406bafb361da46e397a92e60b4da2ec068 Signed-off-by: Linda Wang <wangwulin@huawei.com> Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-08Add unit tests for parserCédric Ollivier1-1/+1
It also fixes the corresponding case name in testcases.yaml. Change-Id: I2b93323a18bb192d9ad35823dc48b236cb6841b1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-31Make tempest cleanup resources after runningLinda Wang1-4/+4
JIRA: FUNCTEST-773 1. When running refstack_defcore, create tenant and user failed due to conflict with those created when runnning tempest_smoke_serial. 2. Clean all resources created for tempest_smoke_serial. Change-Id: Icf8ec95f6db0b2f93bf6c56ec10a6e8ea1a91ab1 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-03-30Re-enable rally and tempest full for weeklyjose.lausuch1-43/+31
Change-Id: I640531ebcc471adebe780c75627d822cf8fd956d Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-03-29Added test case for opera vimshelenyao1-4/+4
1. Created a ImsOnBoardingBase to wrap up all common ims operations 2. Created test case to test opera vims onboarding Change-Id: Idf3a82e10661271ad413f4dd5795c80ca4421d73 Signed-off-by: helenyao <yaohelan@huawei.com>
2017-03-28Remove vping_ssh from bgpvpn and gluon scenarioNikolas Hermanns1-1/+1
Cause of the bug: https://bugs.opendaylight.org/show_bug.cgi?id=5586 Floating ip is not working at the moment for us. We don't want to remove the vping_ssh test from fuel. the scenarios are called different in fuel and in apex. odl-bgpvpn in apex. odl_l2-bgpvpn in fuel. Change-Id: Iaa2e6cea8a4fdb8635f018427aaf30634fd8d2d5 Signed-off-by: Nikolas Hermanns <nikolas.hermanns@ericsson.com>
2017-03-22Disable blocking status for odl_netvirtJamo Luhrsen1-1/+1
Change-Id: I4a3f10a82d3109a44f8db237cbd7beda06a361da Signed-off-by: Jamo Luhrsen <jluhrsen@redhat.com>
2017-03-22Merge "Set refstack_defcore criteria to 100%"Morgan Richomme1-1/+1
2017-03-21Set refstack_defcore criteria to 100%Linda Wang1-1/+1
In order to be consistent with other tests, set refstack_defcore criteria to 100%. Change-Id: Iab8bc0cb66b834a661b860a032192d502fde578b Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-03-16Remove functest_defcore testcasesMorgan Richomme1-14/+0
refstack_defcore has been merged the testcase is no more needed (duplication) Change-Id: I287406b40cdd89ff2c4eadcde7e9d5a93e21953a Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-03-16Bugfix: fix the criteria of refstack_defcoreLinda Wang1-1/+1
The criteria of refstack_defcore is supposed to be ">= 80%" rather than "== 80%". Change-Id: I9f9e38c84925c3725289e8b427e5735d131459f1 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-03-15Parse refstack_defcore resultsLinda Wang1-1/+1
JIRA: DOVETAIL-366 JIRA: FUNCTEST-758 Parse refstack_defcore results, including: 1. show number of tests passed, skipped and failed 2. record the detailed info about testcases passed, skipped and failed. Change-Id: I355a907c5c9ea2739f845447144c62613ec3bb6a Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-03-15refstack client integrationMatthewLi1-0/+15
JIRA: DOVETAIL-366 JIRA: FUNCTEST-758 usage: 1,functest testcase run refstack_defcore, will run the default testcases in defcore_201608.txt 2,cd <path>/functest/functest/opnfv_tests/openstack/refstack_client python refstack_client.py -c <tempest-conf-file-path> --testlist <testlist-path> this has been worked with openstack-refstack experts, etc. tested workedi. result see http://paste.openstack.org/show/602173/ Change-Id: I1aaffea01dec9dc8d1c572885bdf516614a16894 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-03-10Exclude Doctor from Joid scnenariosMorgan Richomme1-1/+1
Doctor feature not available for Joid on Danube Change-Id: I1d39ff15a1fe003689828f6923b92c7e7c7d716f Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-03-09Change scenario for odl_netvirt tests to be os-odl_l3-nofeatureJamo Luhrsen1-2/+2
Change-Id: Ibf04adcaa0b14c4cdda8fba2ef2826a0c7f3b7d4 Signed-off-by: Jamo Luhrsen <jluhrsen@redhat.com>
2017-03-09Change the order of HealthCheck testsMorgan Richomme1-16/+16
currently the snaps healthcheck (dhcp) is tried first it should be connectivity, api, dhcp the reporting looks for scenario based on connectivity testing dhcp first => scenarios ok on connectivy but failed with DHCP are not considered, which could be misleading Change-Id: I0eb34b1f5ba163540281e5ef0dcb12c8cefa88bd Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-03-09Introduce smoke tests for FDSTomas Cechvala1-0/+19
Added: - fds repository to docker file - fds test to smoke tier in testcases - config values for fds tests A new robot var OS_AUTH_URL is defined as expected by FDS robot files. Method args switch from Keystone IP to OS_AUTH_URL which is more precised. The argument parsing is changed too. Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com> Change-Id: Ife27ab4a83f0110cbdcf7c9d47d9b3fb4c0ec705 Signed-off-by: Tomas Cechvala <tcechval@cisco.com> Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-08Exclude SNAPS smoke for lxd scenarioMorgan Richomme1-42/+14
SNAPS healthcheck is already excluded due to the use of QCOW2 images, not compatible with lxd scenarios 2 JIRAs have been cretaed to adapt the SNAPS cases to lxd in Euphrates https://jira.opnfv.org/browse/FUNCTEST-752 https://jira.opnfv.org/browse/FUNCTEST-753 Additionnal cleaning done on testcases.yaml - remove juju_epc case (no one from OAI available for Danube) - remove orchestra (as standalone test case of the MANO stack, postponed to Euphrates) - comment aaa (empty sample used to illustrate the VNF abstraction but not completed for Danube) Change-Id: Ibad6cad4c92ee9897b6665b29c09b62cb1986aae Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-03-02fix KeyError: 'clean_flag'Ryota MIBU1-0/+1
Change-Id: I26d24ed1db0668aba4c737058c82e2d655bbf818 Signed-off-by: Ryota MIBU <r-mibu@cq.jp.nec.com>
2017-03-02Merge "Add clean_flag to test cases"Jose Lausuch1-0/+33
2017-03-01Merge "Adopt TestcaseBase for ONOS and ONOS SFC"Jose Lausuch1-1/+6
2017-03-01Adopt TestcaseBase for ONOS and ONOS SFChelenyao1-1/+6
1. Adopted TestcaseBase for ONOS and ONOS SFC 2. Removed the ONOS testcase from exec_test.sh This patch is not meant to refactor the code which can be improved in a later patch. Change-Id: Ib8ac47742f598648c7575c9a8968477307629d42 Signed-off-by: helenyao <yaohelan@huawei.com>
2017-03-01Merge "Re-enable the tests for bgpvpn scenario wich use floating ips"Jose Lausuch1-2/+2
2017-03-01Merge "Add custom tempest test."Helen Yao1-0/+16
2017-03-01Add clean_flag to test casesjose.lausuch1-0/+33
Some test cases such as tempest or those using SNAPS have a built-in cleaup or teardown mechanism. This patch helps to save some time by bypassing the openstack snapshot and cleanup when running those kind of tests cases. Change-Id: I50e7a17937a813532b3756474158416d3e85b922 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-28Merge "Add domino testcase"Morgan Richomme1-0/+12
2017-02-27Merge "Adding vRouter test codes using TestCasesBase"Morgan Richomme1-0/+13
2017-02-26Merge "Remove shell Healtcheck test case"Jose Lausuch1-13/+1
2017-02-25Re-enable the tests for bgpvpn scenario wich use floating ipsjose.lausuch1-2/+2
Change-Id: Ia468b9c9ab9f74a3255aa3647916d1018cef2902 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-25Add custom tempest test.jose.lausuch1-0/+16
Some users need to run a specific list of tempest test cases. This patch allows them to do that by running the command: functest testcase run tempest_custom Change-Id: If8e7726b98c3a71398c7abdf80aea79f10653ebf Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-24Merge "Add Tempest Defcore test case"Jose Lausuch1-23/+35
2017-02-24Add Tempest Defcore test casejose.lausuch1-23/+35
Refstack is the OpenStack project that provides the tooling to run the approved specific tempest test cases (Defcore guidelines) by the Interop working group and by the OpenStack board for OpenStack certification purposes. For more info: https://refstack.openstack.org/ This adds the possibility for a Functest user to run this list of tempest test cases by just issuing the command: functest testcase run tempest_defcore The test will be run on daily loops with the scenario nosdn-nofeature on any installer. Change-Id: I920b2a44cff418116008fdf514425e69dc959412 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-24Adding vRouter test codes using TestCasesBaseh-hayashi1-0/+13
JIRA: FUNCTEST-530 1.setting "nosdn-nofeature" scinario in testcase.yaml. 1.adding process of setting result details for push DB 2.Delete of define "vrouter_data" in config_functest.yaml. 3.modify of vrouter git tag's name "stable". 4.modrify of testcase name "vyos_vrouter" In order to more specific. Change-Id: I0602c01dbf9a5e71bf85943c2b5e40ef51269ffd Signed-off-by: Hideyasu Hayashi <hideyasu.hayashi@okinawaopenlabs.org>
2017-02-24Integration of orchestra_ims testMorgan Richomme1-2/+2
JIRA: FUNCTEST-726 Change-Id: Ie6a35a817aa327a7e9143d93b5ea2582e99e0066 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-02-23Add domino testcaseUlas Kozat1-0/+12
Change-Id: I177820638c7b050764c5354cd7ed22a7ccf3acbc Signed-off-by: Ulas Kozat <ulas.kozat@gmail.com>
2017-02-23Remove shell Healtcheck test casejose.lausuch1-13/+1
The healthcheck tier has grown thanks to the adoption of SNAPs library. Now we have a SNAPs test case that checks that creating a Network, a VM and making sure it gets an IP from the DHCP agent works. Basically, we are duplicating both test cases. This patch removes the execution of the former test without eliminating the code. Change-Id: Iad6fec10fa1c247151c68313dc3760a55e511555 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-22Merge "Integrate BAROMETER feature test case"Jose Lausuch1-0/+15
2017-02-21Integrate BAROMETER feature test casejose.lausuch1-0/+15
Also, add a executor method in VNFBase to allow re-write it for some cases where the feature is another python script instead of shell script. Change-Id: Id734553dffe32fdc9a0befc3f4c0e29e5d56fc61 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-20Remove cloudify_ims from ci daily loopboucherv1-4/+4
JIRA: FUNCTEST-734 Change-Id: I018ba3b14d21b29935a16e1689015c01cb11ffd7 Signed-off-by: boucherv <valentin.boucher@orange.com>