summaryrefslogtreecommitdiffstats
path: root/dovetail/testcase
AgeCommit message (Collapse)AuthorFilesLines
2017-12-13Remove all alias tempest test cases in osinterop test specificaion docsxudan1-39/+5
1. Some tempest test cases have alias test cases because of the different versions of tempest. 2. However, Dovetail fixes the version of tempest to be 14.0.0 in its first release. So the number of test cases that can be executed in osinterop is always 205. And for different SUTs, the 205 test cases are all the same. 3. In this situation, there is no need to list the alias test cases. 4. Just list all the 205 test cases which will not confuse users when they reading dovetail docs. JIRA: DOVETAIL-566 Change-Id: I5e39dd82314833c8c0a6c1732c990f671ae64420 Signed-off-by: xudan <xudan16@huawei.com>
2017-11-28Remove one sub test case from dovetail.tempest.tc004xudan1-1/+0
1. Currently, dovetail.tempest.tc004 has 13 sub test cases defined in its config file and specification document of the test case. All these sub test cases are picked from tempest based on it's version 15.0.0. 2. For Dovetail 1st release, it fixes the version of Functest to be opnfv/dovetail:cvp.0.5.0, and Functest integretes tempest 14.0.0 in this version and it's E release. 3. There is one sub test case in dovetail.tempest.tc004 that is new in tempest 15.0.0, so it can not be recognized and executed by Functest D and E release. It is 'test_server_connectivity_cold_migration_revert'. 4. Remove this sub test case from dovetail.tempest.tc004. This means remove it from the test case config file and specification document. JIRA: DOVETAIL-561 Change-Id: Ia927af040794c7027a741a282ed75fe401f99ff1 Signed-off-by: xudan <xudan16@huawei.com>
2017-10-23Use functest clean module to do the sdnvpn resources cleanupxudan5-0/+15
1. If there throw an exception during the test of one sdnvpn test case, it will not do the clean. 2. Then there are many resources left, such as network, subnet, server... 3. This is a bug of sdnvpn see JIRA: SDNVPN-184 4. The patch for fixing this bug https://gerrit.opnfv.org/gerrit/#/c/44777/ 5. Currently, use functest module "functest openstack clean" to clean the SUT. JIRA: DOVETAIL-524 Change-Id: I629fc6fdc9e4c95960f6122358ff219a467d441b Signed-off-by: xudan <xudan16@huawei.com>
2017-09-28Bugfix: sdnvpn test cases failed due to wrong module namexudan5-0/+15
SDNVPN test cases failed due to wrong module name. The module name needs to be changed into "sdnvpn.test.functest.testcase_1" Refer to patch https://gerrit.opnfv.org/gerrit/#/c/41385/ Currently Dovetail uses functest:cvp.0.5.0 which uses module name "testcase_1". JIRA: DOVETAIL-510 Change-Id: I3232760d3af34125c7da71a5d2c62ca32693c82b Signed-off-by: xudan <xudan16@huawei.com>
2017-09-18Dovetail tool replaces defcore with osinteropxudan2-8/+8
Rename the "defcore" test area to "osinterop". The DefCore working group has been renamed OpenStack InterOp Working Group some time ago. JIRA: DOVETAIL-499 Change-Id: I144af767632c8fb83859e5e04b6427bb1a538348 Signed-off-by: xudan <xudan16@huawei.com>
2017-09-03Remove some tempest test cases that are not too commonxudan2-27/+0
Change-Id: I2a19b9f3e6c44bedef5d3cb20f480405e4d6d5e2 Signed-off-by: xudan <xudan16@huawei.com>
2017-08-31Add additional Tempest API test cases into test case tempest.tc005xudan1-0/+5
1. According to the latest dovetail test case specification document, there are 5 new Tempest api test cases. 2. Add them into tempest.tc005 so they can be tested in CI. JIRA: DOVETAIL-495 Change-Id: Ib8057d01b99472aeb91cf564d09b67407c572235 Signed-off-by: xudan <xudan16@huawei.com>
2017-08-15Adapt Functest cvp.0.5.0 docker imagexudan32-33/+95
In order to use functest cvp.0.5.0, Dovetail tool needs some adaptions. Change-Id: I9d0a23791c246a6c4961f229eaeb1f52f9b4995a Signed-off-by: xudan <xudan16@huawei.com>
2017-08-07Add Tempest test cases to Dovetail CI daily jobsxudan6-0/+111
JIRA: DOVETAIL-478 JIRA: DOVETAIL-473 adds 6 test areas: - forwarding packets through virtual networks in data path - filtering packets based on security rules and port security in data path - dynamic network runtime operations through the life of a VNF - correct behavior after common virtual machine life cycles events - simple virtual machine resource scheduling on multiple nodes - storage security Add Tempest test cases belong to this 6 test areas to Dovetail CI daily jobs. Change-Id: I1535d96f3a85cdd8ac4f0c4375c68ce47bd9df93 Signed-off-by: xudan <xudan16@huawei.com>
2017-08-03Add resiliency test caes into proposed_testsxudan1-0/+11
JIRA: DOVETAIL-474 Add Bottlenecks resiliency test case "posca_factor_ping" into proposed_tests. Change-Id: I76aea651014fb4722ddae07559f417d3f840eee2 Signed-off-by: xudan <xudan16@huawei.com>
2017-07-20mandatory vping test case added to proposed_testsMatthewLi4-22/+4
JIRA: DOVETAIL-446 1.add vping to proposed_tests 2.delete out-dated nfvi test cases Change-Id: I181b00b02abe6fed4ff51e947576c1b71321cdd2 Signed-off-by: MatthewLi <lijun_1203@126.com>
2017-06-30Change sdnvpn testcases file pathxudan5-5/+5
JIRA: DOVETAIL-458 1. Functest has changed the repo of sdnvpn. 2. Currently Dovetail will copy the testcase yaml file to /home/opnfv/repos/sdnvpn/sdnvpn/test/functest/config.yaml in Functest container. 3. The path needed to be replaced by /usr/local/lib/python2.7/dist-packages/sdnvpn/test/functest/config.yaml Change-Id: I1badba4120e96e2c9ed5a07db3189967cf24c564 Signed-off-by: xudan <xudan16@huawei.com>
2017-05-24SDNVPN test cases failed in CIxudan5-5/+5
JIRA: DOVETAIL-439 1. SDNVPN test cases run in CI failed, because the file /home/opnfv/repos/sdnvpn/sdnvpn/test/functest/config.yaml is empty. 2. copy the file userconfig/sdnvpn_config_testcase1/2/4/8.yaml to /home/opnfv/repos/sdnvpn/sdnvpn/test/functest/config.yaml in Functest container Change-Id: Ie904f83c755efc0d9e0abe5dcd44fe4a5056123e Signed-off-by: xudan <xudan16@huawei.com>
2017-05-15Merge "bugfix of wrong testcase name"hongbo tian1-1/+1
2017-05-11bugfix of wrong testcase nameMatthewLi1-1/+1
JIRA: DOVETAIL-430 this is wrong before(ipv6.tc023 same to ipv6.tc019), fixed according to https://gerrit.opnfv.org/gerrit/#/c/34577/ Change-Id: I3d996a7c2adffc305b0f8fa6bcd87e4e6e4c5b7e Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-05-11Add three HA test cases into test suite proposed_testsxudan11-70/+26
JIRA: DOVETAIL-427 There are another three HA test cases which are stable now for adding into test suite proposed_tests for CI daily run. Change-Id: I86ed38d2e3a2bcfda29af50164aadd0da89e6aaa Signed-off-by: xudan <xudan16@huawei.com>
2017-05-07vpn testcases added in proposed_testsMatthewLi5-0/+60
JIRA: DOVETAIL-212 after communicating with Tim/jose, help to add vpn testcase into proposed_tests reference: https://wiki.opnfv.org/display/sdnvpn/SDNVPN+Testing#SDNVPNTesting-TestcompliancetemplateforDovetail Change-Id: Id34a2cadba625abbdbc59ef11affeccba24fa10c Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-05-02xipv6 testcase adjustmentMatthewLi12-15/+8
JIRA: DOVETAIL-418 1) ipv6 tc002-tc005 error fix, delete wrong testcases 2) ipv6 tc018-tc025 os-nosdn-nofeature scenario dependent, make it known to the users Change-Id: I8e63131b795403ee63566ae59bf46782524acd5a Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-22skip ha and rename tempest.confMatthewLi1-1/+1
1) ha testcases need node configure file, skip it temporarily 2) tempest.conf-->refstack_tempest.conf, then it will not covered by other tempest testcases, can used for users to debug Change-Id: Ieb7101d8c406e294755629fd11a686f90b6c0a7f Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-21Merge "add defcore/ha testcases to proposed_tests"hongbo tian20-77/+134
2017-04-21use Functest tempest_custom test cases to run all Ipv6 test casesxudan25-61/+129
JIRA: DOVETAIL-401 1. modify all Ipv6 test cases, change tempest_smoke_serial/tempest_full_parallel into tempest_custom, add ID of each sub-testcases. 2. Some sub-testcases in ipv6.tc002, ipv6.tc003, ipv6.tc004 and ipv6.tc005 can not be found in the full tempest test case list. 2. disable testcases' script_result_acquired to enable tempest_custom run multiple times. 3. use build_tag-testcase_name(such as daily-master-79a6e392-28bb-4c40-9d88-a03bed99da2d-dovetail.ipv6.tc025) as the unique ID of each testcase. 4. comment out the progress bar, because it has some problems when print the report text. 5. Functest has changed the TEST_DB_URL from http://<testapi>:<port>/apv/v1 to http://<testapi>:<port>/apv/v1/results. 6. Functest has changed the TEST_DB_URL to file://<file_path>/<file_name>. see https://gerrit.opnfv.org/gerrit/#/c/33511/ Change-Id: I524f6702377f215f526158214842481eb44565be Signed-off-by: xudan <xudan16@huawei.com>
2017-04-21add defcore/ha testcases to proposed_testsMatthewLi20-77/+134
JIRA: DOVETAIL-398 JIRA: DOVETAIL-399 1. this testsuite will run in community CI, see https://gerrit.opnfv.org/gerrit/#/c/33509/ 2. HA testcases please see https://wiki.opnfv.org/pages/viewpage.action?pageId=8684344 3. vimops testcases deleted, since it's covered in defcore Change-Id: Ied9c8867e0aa99e33fd213030e6471aa5cfb86e8 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-10refstack: add manually method to let user can set its tempest.conf and ↵MatthewLi1-0/+18
defcore.txt JIRA: DOVETAIL-394 for refstack, userguide is listed in https://gerrit.opnfv.org/gerrit/#/c/33049/ in CI, use the automate way to generate tempest.conf to let refstack run against SUT, i.e., defcore.tc001.yml will provide a way to let user can let refstack run against SUT manually, i.e. defcore.tc002.yml since the tempest.conf autometely generated may not satisfied with SUT, users can adjust its tempest.conf to let the defcore testcases pass. Change-Id: I5ed212c9c6d864d352d1c497ea039cc553461158 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-01refstack result parseMatthewLi1-1/+1
JIRA: DOVETAIL-375 1, together with other testsuite from functest, such as tempest etc. this should merge after https://gerrit.opnfv.org/gerrit/#/c/31313/ 2, delete some unnecessary definition 3, bugfix of docker cp inside container, see https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-debug-master/361/console 4, disable tempest testcaes since changed to refstack_defcore Change-Id: I67503e348af4a3f697f1b4106bfe81c05add8aa0 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-03-30dovetail tool: refstack testcase integrationMatthewLi1-0/+266
JIRA: DOVETAIL-370 Change-Id: I60c9f431358f848e24abcb865c4b2ca8c3b2d843 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-01-25dovetail tool: functest different testsuite support and yardstick testcase addedMatthewLi4-2/+18
JIRA: DOVETAIL-186 Change-Id: Id0eec6652c9595c660436dec89a309a77b16bc03 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-01-16dovetail tool: format improvementMatthewLi36-2/+36
Change-Id: Ibf0bb781c0f0b6be73526bf45231040e68510f76 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-12-22[dovetail tool]check and get results for each cmdLeo Wang3-2/+17
JIRA: DOVETAIL-166 Check the results of each cmds executed in test case 1. the results of pre_condition, post_condition, cmds need to be checked, so it can get a quick fail, dont need to go through the next step 2. it's more accurate to show where error occurred as early as possible 3. get results from shell scripts Change-Id: I5c1e59839c55b92de0e83e7e1eb552aa364b3f80 Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
2016-12-13[dovetail tool] support shell scripts for testcase validationLeo Wang35-33/+107
JIRA: DOVETAIL-46 1. for now a testcase has two kinds of validation types(functest, yardstick), and it is not enough to check the complete funcionality 2. add new validation type(shell) for extra validation of the test case to make result more accurate and more convincing. Change-Id: I45dca6b8dbd888757da163189d261f6e4dba5034 Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
2016-11-10dovetail tool: 2 config files, compliance_set and proposed_testsMatthewLi33-1/+280
JIRA: DOVETAIL-50 1)'proposed_tests.yml' is added, which runs as the total tests run. The 'compliance_set.yml' will run as dovetail certification tests; 2)ipv6 test cases added accroding to 'docs/testsuites/ipv6/ipv6_all_testcases', tc026 missed temporarily,yardstick ipv6tc026 only supports 2 installers by now according to https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Cases 3)vimops001-006 added for VIM images operation test cases 4)nfvi001-002 added for vping ssh/userdata in fucntest 5)more test cases can be added in future in proposed_tests.yml, besides, the test cases can be adjusted in future in proposed_tests.yml Change-Id: I8a8634583549a802f6f64fbea651e7be4efa361c Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-10-12preparation for setup.pyMatthewLi1-0/+10
JIRA: DOVETAIL-26 1)change the directory scripts to dovetail, to make it unified in OPNFV and for setup.py 2)change the paths accordingly 3)this should be first merged, since docker is changed, then to add setup.py and run the local test Change-Id: I65974bde17f310ebbfe3c025532b05dae8752945 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>