summaryrefslogtreecommitdiffstats
path: root/dovetail/compliance
AgeCommit message (Collapse)AuthorFilesLines
2018-01-27Align container tags and test suite configuration yaml for initial releasexudan2-59/+2
Change default container tag to be: 1. Functest ovp.1.0.0 2. testapi ovp.1.0.0 Add test suite ovp.1.0.0 and set it to be the default test suite. Comment out bottlenecks CLI for initial release. JIRA: DOVETAIL-580 Change-Id: I956e49e43fc7ba0176c14079b7228089db5dc108 Signed-off-by: xudan <xudan16@huawei.com> (cherry picked from commit c47b8dde0e39e273372b091a3d8da1cc15b7ae9f)
2017-11-07Tag dovetail cvp.0.9.0xudan1-0/+57
JIRA: DOVETAIL-550 Change-Id: I871ec1fdf8d33efe3ed00602d0184c7b35896c17 Signed-off-by: xudan <xudan16@huawei.com>
2017-10-23Add testsuite cvp.0.8.0 for Beta testsxudan1-0/+57
1. Now there is a testsuite proposed_tests for beta tests which may cause some confusions. 2. Copy all test cases to testsuite cvp.0.8.0 for beta testers to use. 3. Update userguide with testsuite cvp.0.8.0 JIRA: DOVETAIL-528 Change-Id: I35bc7dee6e07a9125a4c1991b0049f6e2d705cc5 Signed-off-by: xudan <xudan16@huawei.com>
2017-10-16SDNVPN test cases resources left may effect the next test casesxudan1-5/+5
Move SDNVPN test cases after tempest test cases, then the resources left will not effect tempest test cases. JIRA: DOVETAIL-524 Change-Id: I815ae7f3a3be0225be1e9695f13d4552946bffb5 Signed-off-by: xudan <xudan16@huawei.com>
2017-09-25Support to run mandatory or optional test cases separatelyxudan1-9/+12
1. Currently, dovetail can run a test suite or a test area in that test suite. 2. The test areas in one test suite are not divided into optional and mandatory. 3. Split them and support to run just mandatory or optional. 4. Support to run multiple test areas. JIRA: DOVETAIL-505 Change-Id: I42cd7b4e11c3e3674c806e9bc999b782bf5c85c6 Signed-off-by: xudan <xudan16@huawei.com>
2017-09-18Dovetail tool replaces defcore with osinteropxudan2-7/+6
Rename the "defcore" test area to "osinterop". The DefCore working group has been renamed OpenStack InterOp Working Group some time ago. JIRA: DOVETAIL-499 Change-Id: I144af767632c8fb83859e5e04b6427bb1a538348 Signed-off-by: xudan <xudan16@huawei.com>
2017-09-03Remove some tempest test cases that are not too commonxudan1-1/+0
Change-Id: I2a19b9f3e6c44bedef5d3cb20f480405e4d6d5e2 Signed-off-by: xudan <xudan16@huawei.com>
2017-08-31Add additional Tempest API test cases into test case tempest.tc005xudan1-2/+2
1. According to the latest dovetail test case specification document, there are 5 new Tempest api test cases. 2. Add them into tempest.tc005 so they can be tested in CI. JIRA: DOVETAIL-495 Change-Id: Ib8057d01b99472aeb91cf564d09b67407c572235 Signed-off-by: xudan <xudan16@huawei.com>
2017-08-07Add Tempest test cases to Dovetail CI daily jobsxudan1-0/+7
JIRA: DOVETAIL-478 JIRA: DOVETAIL-473 adds 6 test areas: - forwarding packets through virtual networks in data path - filtering packets based on security rules and port security in data path - dynamic network runtime operations through the life of a VNF - correct behavior after common virtual machine life cycles events - simple virtual machine resource scheduling on multiple nodes - storage security Add Tempest test cases belong to this 6 test areas to Dovetail CI daily jobs. Change-Id: I1535d96f3a85cdd8ac4f0c4375c68ce47bd9df93 Signed-off-by: xudan <xudan16@huawei.com>
2017-08-03Add resiliency test caes into proposed_testsxudan1-0/+2
JIRA: DOVETAIL-474 Add Bottlenecks resiliency test case "posca_factor_ping" into proposed_tests. Change-Id: I76aea651014fb4722ddae07559f417d3f840eee2 Signed-off-by: xudan <xudan16@huawei.com>
2017-07-20mandatory vping test case added to proposed_testsMatthewLi2-3/+5
JIRA: DOVETAIL-446 1.add vping to proposed_tests 2.delete out-dated nfvi test cases Change-Id: I181b00b02abe6fed4ff51e947576c1b71321cdd2 Signed-off-by: MatthewLi <lijun_1203@126.com>
2017-05-25change the name of Yardstick results file and some log msgxudan1-4/+0
JIRA: DOVETAIL-428 1. Change the name of Yardstick result file from opnfv_yardstick_tc019.out to dovetail.ha.tc001.out. Because opnfv_yardstick_tc019 is the name defined in Yardstick and dovetail.ha.tc001 is the name defined in Dovetail. 2. Check whether the file exists before reporting the results are stored in file. 3. Delete test cases vping_ssh, vping_userdata and sdnvpn.tc003 from the testsuite proposed_tests. Change-Id: I4cb16947729479616a03abd9e91b76f9a41a949c Signed-off-by: xudan <xudan16@huawei.com>
2017-05-11Add three HA test cases into test suite proposed_testsxudan1-7/+5
JIRA: DOVETAIL-427 There are another three HA test cases which are stable now for adding into test suite proposed_tests for CI daily run. Change-Id: I86ed38d2e3a2bcfda29af50164aadd0da89e6aaa Signed-off-by: xudan <xudan16@huawei.com>
2017-05-07vpn testcases added in proposed_testsMatthewLi1-0/+6
JIRA: DOVETAIL-212 after communicating with Tim/jose, help to add vpn testcase into proposed_tests reference: https://wiki.opnfv.org/display/sdnvpn/SDNVPN+Testing#SDNVPNTesting-TestcompliancetemplateforDovetail Change-Id: Id34a2cadba625abbdbc59ef11affeccba24fa10c Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-05-02xipv6 testcase adjustmentMatthewLi1-4/+4
JIRA: DOVETAIL-418 1) ipv6 tc002-tc005 error fix, delete wrong testcases 2) ipv6 tc018-tc025 os-nosdn-nofeature scenario dependent, make it known to the users Change-Id: I8e63131b795403ee63566ae59bf46782524acd5a Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-27support HA test cases to run with keyxudan1-5/+5
JIRA: DOVETAIL-409 1. HA test cases need a file pod.yaml, in this file it can use the password or the key. 2. If use key, Dovetail need to pass the key file $DOVETAIL_HOME/dovetail/userconfig/id_rsa to yardstick Container. 3. Add a directory $DOVETAIL_HOME/dovetail/userconfig and a sample_pod.yaml 4. ha.tc001 tc003 tc005 tc006 tc009 are ready. Change-Id: I391cc211dba2cf0740dd0dcaee8a5c2d712eaee1 Signed-off-by: xudan <xudan16@huawei.com>
2017-04-26skip ha tc002 and tc012MatthewLi1-3/+1
since these two testcases will kill the host and can't restart not ready yet, skip. Change-Id: Icbc7e4f080d32d1ecb59df9fd95cb244d74e4cde Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-22skip ha and rename tempest.confMatthewLi1-12/+12
1) ha testcases need node configure file, skip it temporarily 2) tempest.conf-->refstack_tempest.conf, then it will not covered by other tempest testcases, can used for users to debug Change-Id: Ieb7101d8c406e294755629fd11a686f90b6c0a7f Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-21add defcore/ha testcases to proposed_testsMatthewLi2-17/+23
JIRA: DOVETAIL-398 JIRA: DOVETAIL-399 1. this testsuite will run in community CI, see https://gerrit.opnfv.org/gerrit/#/c/33509/ 2. HA testcases please see https://wiki.opnfv.org/pages/viewpage.action?pageId=8684344 3. vimops testcases deleted, since it's covered in defcore Change-Id: Ied9c8867e0aa99e33fd213030e6471aa5cfb86e8 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-10refstack: add manually method to let user can set its tempest.conf and ↵MatthewLi1-0/+1
defcore.txt JIRA: DOVETAIL-394 for refstack, userguide is listed in https://gerrit.opnfv.org/gerrit/#/c/33049/ in CI, use the automate way to generate tempest.conf to let refstack run against SUT, i.e., defcore.tc001.yml will provide a way to let user can let refstack run against SUT manually, i.e. defcore.tc002.yml since the tempest.conf autometely generated may not satisfied with SUT, users can adjust its tempest.conf to let the defcore testcases pass. Change-Id: I5ed212c9c6d864d352d1c497ea039cc553461158 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-01refstack result parseMatthewLi1-4/+4
JIRA: DOVETAIL-375 1, together with other testsuite from functest, such as tempest etc. this should merge after https://gerrit.opnfv.org/gerrit/#/c/31313/ 2, delete some unnecessary definition 3, bugfix of docker cp inside container, see https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-debug-master/361/console 4, disable tempest testcaes since changed to refstack_defcore Change-Id: I67503e348af4a3f697f1b4106bfe81c05add8aa0 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-03-30dovetail tool: refstack testcase integrationMatthewLi1-0/+1
JIRA: DOVETAIL-370 Change-Id: I60c9f431358f848e24abcb865c4b2ca8c3b2d843 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-02-07dovetail tool: report.py bugfix and tempest skipped testcase addedMatthewLi1-0/+2
JIRA: DOVETAIL-343 1, bugfix https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-debug-master/lastSuccessfulBuild/artifact/results/dovetail_report.txt test result error 2, skipped testcase in tempest added 3, two testcase in tempest full testsuite added in debug.yml to debug Change-Id: I6bf371d3384e3755d6a41d951cfc9d1a55e00ab3 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-01-25dovetail tool: functest different testsuite support and yardstick testcase addedMatthewLi1-1/+3
JIRA: DOVETAIL-186 Change-Id: Id0eec6652c9595c660436dec89a309a77b16bc03 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-01-16dovetail tool: format improvementMatthewLi3-0/+3
Change-Id: Ibf0bb781c0f0b6be73526bf45231040e68510f76 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-12-15dovetail tool: bugfix about setup dovetailxudan1-1/+1
1. change the entry_points of setup.cfg into dovetail.run:main 2. after setup the command 'dovetail -d true --testsuite debug' can just be successfully executed under folder dovetail/dovetail. That is mainly because of the load functions of testcase.py using relative paths. 3. using absolute path instead of relative path can support the command successfully executed under any path. JIRA: DOVETAIL-154 Change-Id: I4a2ad39a1a60e7e63efc55515781d27b823aa894 Signed-off-by: xudan <xudan16@huawei.com>
2016-12-14add example test areaLeo Wang1-8/+0
1. use this config to support example test case Change-Id: I3fda73c408ed4b2bfb75aac52e0292fa4de9161b Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
2016-12-13[dovetail tool] support shell scripts for testcase validationLeo Wang2-0/+9
JIRA: DOVETAIL-46 1. for now a testcase has two kinds of validation types(functest, yardstick), and it is not enough to check the complete funcionality 2. add new validation type(shell) for extra validation of the test case to make result more accurate and more convincing. Change-Id: I45dca6b8dbd888757da163189d261f6e4dba5034 Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
2016-12-02[WIP] dovetail report improvementMatthewLi1-0/+9
JIRA: DOVETAIL-49 1)details please refer to https://wiki.opnfv.org/display/dovetail/Dovetail+Reports 2)results please refer to one example https://etherpad.opnfv.org/p/dovetail_report Change-Id: I63a1ba3b6aa1ffe5fa043a11cd3777dd0e8d52d9 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-11-26dovetail tool: testarea argument addedMatthewLi2-0/+49
JIRA: DOVETAIL-71 1) argument testarea is added 2) terminology "scenario" change to "testsuite" 3) related documents amended 4) all word "certification" changed to "compliance" Change-Id: I038a9e04bf83cfdac20f0c59adb1841bf5470584 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>