summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-05-07vpn testcases added in proposed_testsMatthewLi13-4/+249
JIRA: DOVETAIL-212 after communicating with Tim/jose, help to add vpn testcase into proposed_tests reference: https://wiki.opnfv.org/display/sdnvpn/SDNVPN+Testing#SDNVPNTesting-TestcompliancetemplateforDovetail Change-Id: Id34a2cadba625abbdbc59ef11affeccba24fa10c Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-05-07Proposing to move "post conditions" as an optional topic for a test case.ChristopherPrice1-12/+2
It is not clear that there are post conditions to be documented for all test cases. In the case a test cse creates an image and deletes it, there is littöe need for a post condition to be documented. If a test case establishes a network to be used for a subsequent test ina sequence this information should be clearly dcumented. Change-Id: Ifb4bb45f240921c180077ab7273988d973417f6f Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
2017-05-05Merge "Get results json data from database"hongbo tian3-5/+11
2017-05-05Merge "Uploading the initial draft of the first VPN test case "hongbo tian2-5/+132
2017-05-04Merge "Drafting the first "refstack" version of the test scope document."hongbo tian1-0/+363
2017-05-04Get results json data from databasexudan3-5/+11
JIRA: DOVETAIL-417 1. If report results with files, all results will be stored in local files. 2. If report to database, will get results from database and stored with file results.json Change-Id: Ic30037bd66cd37042f82b75fac2043a876e21c6f Signed-off-by: xudan <xudan16@huawei.com>
2017-05-03Merge "local db support offline mode"hongbo tian6-38/+93
2017-05-03local db support offline modeLeo Wang6-38/+93
JIRA: DOVETAIL-415 The local db also need to support offline, move all the git clone/apt/yum/pip/wget/curl into the Dockerfile Change-Id: I2392dc9f4a6bd6f6a5a3f4849625a576c51a44f8 Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
2017-05-03Drafting the first "refstack" version of the test scope document.Georg Kunz1-0/+363
To be aligned with the committers and agreed as the "foundation" list of test scope for moving forward with DoveTail test selection. Requires a majority of committer approvals prior to merging. Change-Id: I5fb091d2190a183d24ba9e766d19aba9e6a7c094 Signed-off-by: ChristopherPrice <christopher.price@ericsson.com> Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2017-05-03Merge "xipv6 testcase adjustment"hongbo tian15-26/+16
2017-05-02xipv6 testcase adjustmentMatthewLi15-26/+16
JIRA: DOVETAIL-418 1) ipv6 tc002-tc005 error fix, delete wrong testcases 2) ipv6 tc018-tc025 os-nosdn-nofeature scenario dependent, make it known to the users Change-Id: I8e63131b795403ee63566ae59bf46782524acd5a Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-05-02Uploading the initial draft of the first VPN test case ChristopherPrice2-5/+132
This should be considered in the danube CVP phase as an optional test area for OPNFV compliance Change-Id: I99494cf2246e1cf079b87414ef1bb34d9b26e6b3 Co-Authored-By: Tim Irnich <tim.irnich@ericsson.com> Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
2017-05-01Tips on update docker to right version, > 1.8grkiss3-1/+30
JIRA: DOVETAIL-414 Dovetail requires docker at version > 1.8, so add some validation in code, and also versions of client and server should match with each other, or there will be unpredictable errors. Gives a notice to user to decide whether to update the docker to the latest version(1.17) Change-Id: I1331f5c19943a73e8fe25f76bd21628233e556f3 Signed-off-by: grakiss <grakiss.wanglei@huawei.com>
2017-04-27support HA test cases to run with keyxudan5-11/+68
JIRA: DOVETAIL-409 1. HA test cases need a file pod.yaml, in this file it can use the password or the key. 2. If use key, Dovetail need to pass the key file $DOVETAIL_HOME/dovetail/userconfig/id_rsa to yardstick Container. 3. Add a directory $DOVETAIL_HOME/dovetail/userconfig and a sample_pod.yaml 4. ha.tc001 tc003 tc005 tc006 tc009 are ready. Change-Id: I391cc211dba2cf0740dd0dcaee8a5c2d712eaee1 Signed-off-by: xudan <xudan16@huawei.com>
2017-04-27Setting the document directory structure according to the Plugfest outcome.ChristopherPrice23-13/+344
Implemented a directory and file structure that supports the requirements defined in the plugfest etherpad: https://etherpad.opnfv.org/p/cvp_tooling_sketch Change-Id: Icb7d2db4b0132f2039c680883e173ff5d7fc0120 Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
2017-04-27Adding a stub document for a CVP reviewer guideGeorg Kunz1-0/+35
This document shall describe guidelines and evaluation criteria for reviewers on how to handle the review process of results. DOVETAIL-408 Change-Id: I78e669544bc172b74f94443176a48082a4c28a2f Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2017-04-26Merge "skip ha tc002 and tc012"Dave Neary1-3/+1
2017-04-26Committing the test case requirements specification to GIT for committer review.ChristopherPrice1-0/+92
This doucment needs to be +2'd by the majority of committers prior to merging to ensure we are all in agreement with regard to the test case requirements. This is a pre-requisite to test case selection for the CVP. Updated by dneary 2017-04-25: List formatting, and reorganising requirements for installer and stack composition. Change-Id: Iabaf11bc9479a20a58f985233d460013e4690d7c Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
2017-04-26Merge "JIRA: DOVETAIL-352 Updating the test specification document with ↵Christopher Price4-48/+632
iterative inprovements."
2017-04-26skip ha tc002 and tc012MatthewLi1-3/+1
since these two testcases will kill the host and can't restart not ready yet, skip. Change-Id: Icbc7e4f080d32d1ecb59df9fd95cb244d74e4cde Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-26Merge "Bugfix: All HA testcases are SSH timeout in CI jobs"hongbo tian3-12/+20
2017-04-26JIRA: DOVETAIL-352 Updating the test specification document with iterative ↵ChristopherPrice4-48/+632
inprovements. JIRA tasks are being created for specific sections of the document. Refer to the doc for links to relevant JIRA tasks. Change-Id: I1717d56b8817c38802a227db320f30029f68fbd0 Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
2017-04-25Merge "AArch64 support for Dovetail docker"hongbo tian1-0/+32
2017-04-25Bugfix: All HA testcases are SSH timeout in CI jobsxudan3-12/+20
JIRA: DOVETAIL-405 1. All HA test cases running in CI are SSH timeout. 2. The reason is that HA test cases need a file pod.yaml providing the info of each node of Openstack. 3. This pod.yaml should be put in $DOVETAIL_HOME/dovetail/userconfig by users. 4. Dovetail's CI jobs should also do some adaptions. JIRA: DOVETAIL-407 5. modify file .gitignore to ignore directories results/ and userconfig/. Change-Id: I5c681445ec83c9d67a0151370f5b56840ffe444c Signed-off-by: xudan <xudan16@huawei.com>
2017-04-25Bugfix: KeyError when tempest_custom runs wrongxudan2-10/+14
JIRA: DOVETAIL-406 1. When there are something wrong with the test case tempest_custom, it's details json data are "details": {}, So it will throw KeyError exception in this situation. 2. Catch this exception and set all sub_testcases FAIL. 3. Disable the exceed_max_retry_times(). Change-Id: I6bdfc5944a535625b2211a3bd8096a13b81b339e Signed-off-by: xudan <xudan16@huawei.com>
2017-04-24AArch64 support for Dovetail dockerAlexandru Nemes1-0/+32
Add aarch64 patch for Dovetail docker file; The patch should be applied from releng Move to Ubuntu 16.04 in order, since docker.io package was not available for arm on Ubuntu 14.04. Added "pip install -U setuptools", since installation of other components failed because setuptools was missing. Change-Id: If00346d89c48be86c6d9b830f75c1792c39d6bb3 Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
2017-04-24Merge "skip ha and rename tempest.conf"hongbo tian2-13/+13
2017-04-22skip ha and rename tempest.confMatthewLi2-13/+13
1) ha testcases need node configure file, skip it temporarily 2) tempest.conf-->refstack_tempest.conf, then it will not covered by other tempest testcases, can used for users to debug Change-Id: Ieb7101d8c406e294755629fd11a686f90b6c0a7f Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-22Bugfix about the ipv6 results checkerxudan1-0/+6
JIRA: DOVETAIL-404 1. IPV6 sub-testcase's name includes [id], which will affects the regex. 2. Remove the [id] from the name of the sub_testcases. Change-Id: I51b567e8382fa4e701f8b32ba7ea195d49fd31eb Signed-off-by: xudan <xudan16@huawei.com>
2017-04-21Merge "add defcore/ha testcases to proposed_tests"hongbo tian23-94/+158
2017-04-21use Functest tempest_custom test cases to run all Ipv6 test casesxudan33-85/+166
JIRA: DOVETAIL-401 1. modify all Ipv6 test cases, change tempest_smoke_serial/tempest_full_parallel into tempest_custom, add ID of each sub-testcases. 2. Some sub-testcases in ipv6.tc002, ipv6.tc003, ipv6.tc004 and ipv6.tc005 can not be found in the full tempest test case list. 2. disable testcases' script_result_acquired to enable tempest_custom run multiple times. 3. use build_tag-testcase_name(such as daily-master-79a6e392-28bb-4c40-9d88-a03bed99da2d-dovetail.ipv6.tc025) as the unique ID of each testcase. 4. comment out the progress bar, because it has some problems when print the report text. 5. Functest has changed the TEST_DB_URL from http://<testapi>:<port>/apv/v1 to http://<testapi>:<port>/apv/v1/results. 6. Functest has changed the TEST_DB_URL to file://<file_path>/<file_name>. see https://gerrit.opnfv.org/gerrit/#/c/33511/ Change-Id: I524f6702377f215f526158214842481eb44565be Signed-off-by: xudan <xudan16@huawei.com>
2017-04-21add defcore/ha testcases to proposed_testsMatthewLi23-94/+158
JIRA: DOVETAIL-398 JIRA: DOVETAIL-399 1. this testsuite will run in community CI, see https://gerrit.opnfv.org/gerrit/#/c/33509/ 2. HA testcases please see https://wiki.opnfv.org/pages/viewpage.action?pageId=8684344 3. vimops testcases deleted, since it's covered in defcore Change-Id: Ied9c8867e0aa99e33fd213030e6471aa5cfb86e8 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-21fix tox errorMatthewLi1-2/+2
see logs https://build.opnfv.org/ci/job/dovetail-verify-master/560/console Change-Id: Ifa66087c4c8fcbaae809aae380587a4f03eded6c Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-21Merge "put dovetail test result back in log and not generte report file"hongbo tian2-6/+6
2017-04-19set dovetail.log level by command linexudan2-7/+10
JIRA: DOVETAIL-400 1. set both the dovetail.log and screen log level by command --debug. 2. if --report http://testapi_url:port/api/v1/, change it into --report http://testapi_url:port/api/v1 Change-Id: I7cf697d4ac9072fc2d302a67ccde0f452d84be8f Signed-off-by: xudan <xudan16@huawei.com>
2017-04-18put dovetail test result back in log and not generte report fileMatthewLi2-6/+6
Change-Id: I2da5e7844f5ce034d4c0cfaa4592e6cd47002049 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-18Merge "fix click and pytz package version"hongbo tian2-3/+2
2017-04-16fix click and pytz package versionMatthewLi2-3/+2
JIRA: DOVETAIL-376 Change-Id: Ia269ca7cdd34c71b4531b28e42e40415190ae398 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-14directory updatehongbotian15-304/+0
JIRA: DOVETAIL-395 1. since plugfest has its own plan, the files are removed Change-Id: I4989d1e0e1af483205733b3130e3f1152a8933c7 Signed-off-by: hongbotian <hongbo.tianhongbo@huawei.com>
2017-04-13Merge "dovetail tool: bugfix: test cases are erroneously judged"hongbo tian6-75/+97
2017-04-12dovetail tool: bugfix: test cases are erroneously judgedxudan6-75/+97
JIRA: DOVETAIL-375 JIRA: DOVETAIL-390 1. defcore test cases are erroneously judged because of: a) defcore results json data lack some keys b) some sub-testcases' names are the same as the prefix of another sub-testcases. 2. this patch removes the extra keys and judge sub-testcases accurately. 3. remove "store_type" in functest_config.yml and yardstick_config.yml. 4. change some log levels from info to debug. Change-Id: I5f126e70309409cac66ef9a0909f1573d325098b Signed-off-by: xudan <xudan16@huawei.com>
2017-04-12Merge "refstack: add manually method to let user can set its tempest.conf ↵hongbo tian5-2/+38
and defcore.txt"
2017-04-10refstack: add manually method to let user can set its tempest.conf and ↵MatthewLi5-2/+38
defcore.txt JIRA: DOVETAIL-394 for refstack, userguide is listed in https://gerrit.opnfv.org/gerrit/#/c/33049/ in CI, use the automate way to generate tempest.conf to let refstack run against SUT, i.e., defcore.tc001.yml will provide a way to let user can let refstack run against SUT manually, i.e. defcore.tc002.yml since the tempest.conf autometely generated may not satisfied with SUT, users can adjust its tempest.conf to let the defcore testcases pass. Change-Id: I5ed212c9c6d864d352d1c497ea039cc553461158 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-10dovetail tool: show progress during executing test casesxudan1-1/+6
JIRA: DOVETAIL-159 1. It takes long time to execute test cases that most of users think it hangs there, so a progress is needed to show the test is running. 2. add dynamic progress during execution 3. it' s a fake progress only showing the test is alive, not to show the percentage of completion. Change-Id: I0294659f79e8e09d1bcdd36d8f391d933e35b635 Signed-off-by: xudan <xudan16@huawei.com>
2017-04-05Merge "fix yardstick openrc location"hongbo tian1-1/+1
2017-04-05Merge "bugfix of pbr after updating"hongbo tian1-0/+8
2017-04-01fix yardstick openrc locationMatthewLi1-1/+1
since yardstick changes its openrc location see https://gerrit.opnfv.org/gerrit/#/c/32861/ Change-Id: Ib85910fe5521fb398c47e389e2baf127342b1c0b Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-01Merge "refstack result parse"hongbo tian8-48/+52
2017-04-01refstack result parseMatthewLi8-48/+52
JIRA: DOVETAIL-375 1, together with other testsuite from functest, such as tempest etc. this should merge after https://gerrit.opnfv.org/gerrit/#/c/31313/ 2, delete some unnecessary definition 3, bugfix of docker cp inside container, see https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-debug-master/361/console 4, disable tempest testcaes since changed to refstack_defcore Change-Id: I67503e348af4a3f697f1b4106bfe81c05add8aa0 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-04-01Merge "dovetail tool: check push_to_db success or fail"hongbo tian2-9/+38