Age | Commit message (Collapse) | Author | Files | Lines |
|
Change-Id: If111a8cc56c31ec3e22b090aca92ff56d41f7e01
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I918f85c5ac89cf2745efa8513946a2b5ab8638b1
Signed-off-by: xudan <xudan16@huawei.com>
|
|
There are 2 bugs found with CI results of functest.tempest.neutron_tempest_plugin_api
and all HA test cases.
Yardtsick conf file does't mapping the keyfile to Yardstick container, so all
HA test cases got SSHERROR.
Also remove some useless debug msg to make dovetail.log more concise.
Change-Id: I63c12d1b91ed5f8f8a893689384685ca9215f367
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Error to group skipped sub test cases to passed one.
Example:
Skipped sub test case will be parsed as passed in following case.
skipped sub test case:
tempest.api.network.test_routers.RoutersTest.test_create_show_list_update_delete_router
passed sub test case list includes this one:
tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_subnet_id
Then the skipped one will be checked as pass.
JIRA: DOVETAIL-775
Change-Id: I449f4968c23a9def1177a7c9645e5dcb89b2ec30
Signed-off-by: xudan <xudan16@huawei.com>
|
|
As Trevor Lovett suggests, the Heat file is a zip file. So users need to package
it different only for Dovetail testing.
It will be better to change it to be .zip to keep it the same as what they do.
JIRA: DOVETAIL-770
Change-Id: I5338720c5e1363081e1f8cdce33391a969f1cd97
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The results.json file generated by running
'dovetail run --testsuite xxx' is different from the one
generated by running 'dovetail run --testcase xxx'.
The former can be uploaded to ONAP portal successfully
but the latter one can't.
The main reason is that the latter one lacking of 'vnf_type'.
Keep the results.json with the same format for both running
one test suite or several test cases.
JIRA: DOVETAIL-769
Change-Id: I46af84f9f229f9e601439f68c9ed9df9477d002b
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
This patch adds check for no environment variables provided to
container configuration. Furthermore, it changes the check results
file to a list of check result files.
Change-Id: I2112ee1fd4b3156608d8a27dd3b9c90dbe35a24c
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
Co-Authored-By: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
This patch adds checksum information inside ONAP related test case run
results. The checksum is produced by the VNF input which can either be
a CSAR file or an archive of Heat templates.
Change-Id: I0ed58bdc9cc4031da08fd2ac220ef294520ef447
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch adds all Rally testcases of Functest to Dovetail.
Change-Id: I8b20bce9966e924fd5aac41bde5a30442394db05
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
When the sub-test case list is defined, in the dovetail results,
each test case is appeared with the its verdict next to its name.
Once a sub-testcase is passed, the verdict PASS is appeared next
to the name of sub testcase.
Once a sub-testcase is skipped or failed, the verdict FAIL is
appeared next to the name of sub-testcase.
This fix updates the code so that the verdict of skipped and failed
sub test cases to be appeared respectively next to its sub-tsetcase
name.
JIRA: DOVETAIL-763
Change-Id: Ib05b3793a57acbaa67b89321a840ae11251c2417
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
This patchs gives the ability to provide more than one check results
file. Furthermore, it does not alter the behavior of the existing tests
but prepares dovetail to be able to fetch information from many
sources.
Change-Id: I23cd23d2908ad6bdddd94b977813d0e81dc0c05a
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch removes the ONAP lifecycle deprecated test case.
Furthermore, it clears helper functions as well as unwanted
configuration.
Change-Id: Ib958457cf120ffb641a72be028d3a031741c44c6
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch makes it able to fetch the version from the testsuite.
Furthermore, in case of ONAP test cases it adds the vnf_type to the
produced results.json file in order to be parsed from the results
portal. Finally, it creates a testuite for ONAP TOSCA related tests.
Change-Id: I613358c445cfe62730b98fee5e8c6fba7d322e90
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
Change-Id: I9a3d56932ce41191901381831013768d1c57a749
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
Please refer to env_config.sh.onap.sample when testing it.
The guide of preparing local env for onap package validation can be found here
https://gerrit.onap.org/r/#/c/75119
Change-Id: I39d6b11544847756126623a7eb3b953dc800c470
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. fix the bug 'ShellRunner' object has no attribute 'archive_logs'
2. remove the testarea_supported because it's not very necessary for the test
cases themselves and can make it a little more simple to add a test case
with new test area
JIRA: DOVETAIL-750
Change-Id: I71ee74615200376adca2a0db040753e5fce329bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
In order to integrate k8s test cases, Dovetail framework should do some
refactor and make it more general for k8s test cases as well as ONAP ones.
Integrate 2 k8s test cases.
JIRA: DOVETAIL-748
Change-Id: Ibd87754ffb5fb29f6b4ce79232af860c2ed2da9c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for report file methods of Dovetail which were
missing.
Change-Id: I2ba7c8f9fcfb962988b2904b01538e9f89a0d849
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
Change-Id: Ia17bf7ef3e304eef7b3f0b2f7aa04e3be75bc8c3
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The LF pod used to run the CI verify jobs have updated the version of tox.
So the CI verify jobs failed.
See https://build.opnfv.org/ci/view/dovetail/job/dovetail-merge-master/476/console
Fix the python format errors
Ignore error 'W503 line break before binary operator'.
Change-Id: Ic2df070447bbea39c877ce7a394d733a95064bb1
Signed-off-by: xudan <xudan16@huawei.com>
|
|
If there is no pod.yaml, the results data looks like,
{"status": 2, "result": "[Errno 2] No such file or directory: u'/home/opnfv/userconfig/pre_config/pod.yaml'"}
It will get an exception when reading value from file,
File "/usr/local/lib/python2.7/dist-packages/dovetail/report.py", line 327, in crawl_from_file
criteria = data['result']['criteria']
TypeError: string indices must be integers
https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-baremetal-default-mandatory-master/52/console
Change-Id: I73ca3e5ae73433a2dc139789df4d67e8c583be08
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The official name of the next release is going to be 2018.09.
We need to rename all occurrences of 2018.08 in the code base,
the documentation, and the web portal.
JIRA: DOVETAIL-740
Change-Id: Ie0e532d6671b7590454e9afd960a2934f313a022
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The results.json file is currently not a properly formated json file,
but contains one json structure per line/testcase. Whit this change, all
test results are being collected at runtime and then written to a
properly formated json file at the end of a test run.
Moreover, this patch introduces 'mandatory' and 'optional' sections to
the test suites definition files in order to clearly indicate if a test
is part of the corresponding test scope.
The web portal needs to be adapted accordingly to be able to read the
new results file.
JIRA: DOVETAIL-672
Change-Id: Id0706c7e82622fc9c2c2fece26b2d6c07c1580be
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The 2 sections 'functest_testsuite' and 'functest_testcase' are used by
all Functest test cases.
They aim to parse the results of Tempest sub test cases.
Remove them, and check the sub test case list,
1. if it's empty, that means no need to check sub test cases
2. if it is not empty, check the sub test cases one by one.
Additional, remove some useless keys in dovetail_config.yml
JIRA: DOVETAIL-658
Change-Id: Ic829264b0c8eaedd8e2b3fa55b38a34644482c20
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Updating the licence headers to comply with the community guidelines [1]
and anteater checks.
[1] https://wiki.opnfv.org/display/DEV/Contribution+Guidelines#ContributionGuidelines-License
Change-Id: Idec787e32daf76c0b1bd6108977b78828125da5e
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
1. All detailed tempest test cases results are recorded with
file tempest-report.html rather than tempest.log now.
2. Archive all results files include functest.log, yardstick.log ...
for each test case. Otherwise, they will all be in one log file.
3. Support to set the source files and dest files need to be archived with
the test case config files rather than hardcode with the source code.
4. Give the file which used to be parse the results (PASS/FAIL) with the
test case config files rather than hardcode with the source code.
Change-Id: I30f435d91ab90cf20def47007f177fe98187484d
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The CLI of dovetail tool needs some improvement.
1. Add an option '--testcase' to support specify test case.
2. Add an option '--no-clean' to keep functest/yardstick/bottlenecks containers
for debugging after running the test cases.
3. Improve the local report section.
a. Remove some info that is not necessary
b. Make the info more clearly
JIRA: DOVETAIL-588
Change-Id: Ic769a6e2788b407c3df08ccd593a0ecdaa35e5ea
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Remove local DB and can just report without launch local DB.
Need to do a little modification with the web portal code.
JIRA: DOVETAIL-593
Change-Id: I595db39cce4156596ee0522d4b7e642a23d78ab9
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Use Functest Fraser docker images whose tag is fraser
2. Use Yardstick stable docker images
3. Functest fraser has moved testcases.yaml file from functest/ci to xtesting/ci
4. Functest fraser has removed 'functest env prepare' command
5. Functest uses rally to execute osinterop test cases
instead of refstack_client
Change-Id: I81ebefe6ea959ca77c2c87f1d75ef56ac18c165f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I6a1fa5f27df2f5127eb00156b3135dc79caf5e77
Issue-ID: DOVETAIL-629
Signed-off-by: Moshe <moshehoa@amdocs.com>
|
|
Add 3 new HA test cases into proposed_tests test suite and running them on CI.
For the current 8 HA test cases and 3 other HA test cases plans to add,
do some adaptions for the Yardstick Euphrates release.
1. Don't need the bugfix patch anymore.
2. The data format has changed, and has done some adaptions.
3. For Yardstick Euphrates, if setting OS_INSECURE=True, it needs to set
OS_CACERT=False
4. Test case tc011 needs a image and flavor to create the stack.
JIRA: DOVETAIL-614
JIRA: DOVETAIL-586
Change-Id: I9da8c8316246b9edee0d81ad18422ae6018becaf
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Use opnfv/functest-restapi:euphrates.
Adapt Functest E framework for all Functest test cases.
1. All path of functest and sdnvpn has been changed.
2. Sdnvpn can clean resources by itself with E release, no need to clean again.
3. Add a file testcases.yaml to overwrite the file in functest container,
then it can support to run tempest_custom i.e. ipv6 and tempest in OVP.
JIRA: DOVETAIL-584
Change-Id: I3ba9aa8e80becaa99cc89c8db4b39acbb89dfda4
Signed-off-by: xudan <xudan16@huawei.com>
|
|
When Tempest test cases failed at the SetupClass, the record is the class name
of the test cases rather than the name of the test cases.
The local summary will say they are passed wrongly.
JIRA: DOVETAIL-575
Change-Id: Iec879378770017eddd37e9a99346d980355bb6e2
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The name of the compressed result file included a timestamp according
to the pattern "day_hour_minute". This is a rather uncommon timestamp
and can cause confusion. This patch changes the timestamp to a more
common "YearMonthDay-HourMinute" format.
Please note: Please check if this change has an impact on the web
portal.
Change-Id: Ife41050bc1d5ca6942706b756d03a1d0d251ae41
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
|
|
SDNVPN test cases failed due to wrong module name.
The module name needs to be changed into "sdnvpn.test.functest.testcase_1"
Refer to patch https://gerrit.opnfv.org/gerrit/#/c/41385/
Currently Dovetail uses functest:cvp.0.5.0 which uses module name "testcase_1".
JIRA: DOVETAIL-510
Change-Id: I3232760d3af34125c7da71a5d2c62ca32693c82b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
|
|
The directory tree of results tar file is wrong, so web portal cannot parse it.
Reset the tar file.
JIRA: DOVETAIL-508
Change-Id: I8475323294e84ca51242b700570926d5c0177390
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, the log level of yardstick is DEBUG and cannot be modified.
2. Make it can be changed from CLI.
3. When use dovetail run --debug, can see all Yardstick logs (including DEBUG)
in dovetail.log.
4. When run without debug option, cannot see Yardstick debug logs in
dovetail.logs.
5. yardstick.log always doesn't include debug logs.
JIRA: DOVETAIL-506
Change-Id: Iaafb61232a22e16fff2fa4605bf7e262b022085e
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, dovetail can run a test suite or a test area in that test suite.
2. The test areas in one test suite are not divided into optional and mandatory.
3. Split them and support to run just mandatory or optional.
4. Support to run multiple test areas.
JIRA: DOVETAIL-505
Change-Id: I42cd7b4e11c3e3674c806e9bc999b782bf5c85c6
Signed-off-by: xudan <xudan16@huawei.com>
|
|
In order to use functest cvp.0.5.0, Dovetail tool needs some adaptions.
Change-Id: I9d0a23791c246a6c4961f229eaeb1f52f9b4995a
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-479
User may want to upload all logs along with the results,
so others may check the running logs to make sure what' s going on during the test.
Change-Id: Ice906264bbdf091d3353d6d2e146653aeb057f5f
Signed-off-by: grakiss <grakiss.wanglei@huawei.com>
|
|
JIRA: DOVETAIL-479
User may want to upload all logs along with the results,
so others may check the running logs to make sure what' s going on during the test.
Change-Id: Ia5862f99c129b8207aa91a391567ffdd5ee0829a
Signed-off-by: grakiss <grakiss.wanglei@huawei.com>
|
|
JIRA: DOVETAIL-474
Add Bottlenecks resiliency test case "posca_factor_ping" into proposed_tests.
Change-Id: I76aea651014fb4722ddae07559f417d3f840eee2
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-450
Change-Id: I795679c9559d1bead3aecefff28d21b6567aac2f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-428
1. Change the name of Yardstick result file from opnfv_yardstick_tc019.out to
dovetail.ha.tc001.out. Because opnfv_yardstick_tc019 is the name defined in
Yardstick and dovetail.ha.tc001 is the name defined in Dovetail.
2. Check whether the file exists before reporting the results are stored in file.
3. Delete test cases vping_ssh, vping_userdata and sdnvpn.tc003 from the testsuite
proposed_tests.
Change-Id: I4cb16947729479616a03abd9e91b76f9a41a949c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-425
Change-Id: Ifee2473956e0a1a3ad9c98beff92d94327251614
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-406
1. When there are something wrong with the test case tempest_custom,
it's details json data are "details": {}, So it will throw KeyError
exception in this situation.
2. Catch this exception and set all sub_testcases FAIL.
3. Disable the exceed_max_retry_times().
Change-Id: I6bdfc5944a535625b2211a3bd8096a13b81b339e
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-404
1. IPV6 sub-testcase's name includes [id], which will affects the regex.
2. Remove the [id] from the name of the sub_testcases.
Change-Id: I51b567e8382fa4e701f8b32ba7ea195d49fd31eb
Signed-off-by: xudan <xudan16@huawei.com>
|