Age | Commit message (Collapse) | Author | Files | Lines |
|
JIRA: DOVETAIL-774
Change-Id: Ifb934eaeda52c1d316d80d9bb9c1890ffb71049c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
There are 2 bugs found with CI results of functest.tempest.neutron_tempest_plugin_api
and all HA test cases.
Yardtsick conf file does't mapping the keyfile to Yardstick container, so all
HA test cases got SSHERROR.
Also remove some useless debug msg to make dovetail.log more concise.
Change-Id: I63c12d1b91ed5f8f8a893689384685ca9215f367
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Error to group skipped sub test cases to passed one.
Example:
Skipped sub test case will be parsed as passed in following case.
skipped sub test case:
tempest.api.network.test_routers.RoutersTest.test_create_show_list_update_delete_router
passed sub test case list includes this one:
tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_subnet_id
Then the skipped one will be checked as pass.
JIRA: DOVETAIL-775
Change-Id: I449f4968c23a9def1177a7c9645e5dcb89b2ec30
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Many files in Dovetail repo are lacking of License after LF scanning.
Add all of them except the following files:
1. docker/Dockerfile.aarch64.patch
2. 0001-Allow-additional-properties-in-API-responses.patch
3. docs/images/dovetail_offline_mode.xcf
4. docs/images/dovetail_onfline_mode.xcf
5. all png files
6. all empty files
Change-Id: If1a0b7c2d35d6682c87c4eb9106e1998dc4ca3bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
As Trevor Lovett suggests, the Heat file is a zip file. So users need to package
it different only for Dovetail testing.
It will be better to change it to be .zip to keep it the same as what they do.
JIRA: DOVETAIL-770
Change-Id: I5338720c5e1363081e1f8cdce33391a969f1cd97
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The results.json file generated by running
'dovetail run --testsuite xxx' is different from the one
generated by running 'dovetail run --testcase xxx'.
The former can be uploaded to ONAP portal successfully
but the latter one can't.
The main reason is that the latter one lacking of 'vnf_type'.
Keep the results.json with the same format for both running
one test suite or several test cases.
JIRA: DOVETAIL-769
Change-Id: I46af84f9f229f9e601439f68c9ed9df9477d002b
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
In order to use OPNFV test results page to do the ayalysis of all OVP test cases,
it needs to push all results generated by Dovetail CI jobs to test DB and then
using http://testresults.opnfv.org/test/#/results to check all results.
The following data must contain in the POST body:
1. project_name
2. case_name
3. details
4. installer
5. scenario
6. pod_name
7. build_tag
8. criteria
9. start_date
10. stop_date
11. version
JIRA: DOVETAIL-767
Change-Id: I925ae249e24efd7bfb1c68a69150e9c22f0cdf36
Signed-off-by: xudan <xudan16@huawei.com>
|
|
This patch adds check for no environment variables provided to
container configuration. Furthermore, it changes the check results
file to a list of check result files.
Change-Id: I2112ee1fd4b3156608d8a27dd3b9c90dbe35a24c
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
Co-Authored-By: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
This patch improves the Dockerfile which produces the Dovetail image.
Change-Id: I65e65f155afe237b11c668e6486e48b55cc7e96c
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-752
This patch replaces all python subprocess commands with the
Docker SDK relevant commands.
Change-Id: Iac7caffd80a863a8a022247d735b2a7f2792e49d
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch adds checksum information inside ONAP related test case run
results. The checksum is produced by the VNF input which can either be
a CSAR file or an archive of Heat templates.
Change-Id: I0ed58bdc9cc4031da08fd2ac220ef294520ef447
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch adds all Rally testcases of Functest to Dovetail.
Change-Id: I8b20bce9966e924fd5aac41bde5a30442394db05
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
When the sub-test case list is defined, in the dovetail results,
each test case is appeared with the its verdict next to its name.
Once a sub-testcase is passed, the verdict PASS is appeared next
to the name of sub testcase.
Once a sub-testcase is skipped or failed, the verdict FAIL is
appeared next to the name of sub-testcase.
This fix updates the code so that the verdict of skipped and failed
sub test cases to be appeared respectively next to its sub-tsetcase
name.
JIRA: DOVETAIL-763
Change-Id: Ib05b3793a57acbaa67b89321a840ae11251c2417
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
This patchs gives the ability to provide more than one check results
file. Furthermore, it does not alter the behavior of the existing tests
but prepares dovetail to be able to fetch information from many
sources.
Change-Id: I23cd23d2908ad6bdddd94b977813d0e81dc0c05a
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch removes the ONAP lifecycle deprecated test case.
Furthermore, it clears helper functions as well as unwanted
configuration.
Change-Id: Ib958457cf120ffb641a72be028d3a031741c44c6
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch makes it able to fetch the version from the testsuite.
Furthermore, in case of ONAP test cases it adds the vnf_type to the
produced results.json file in order to be parsed from the results
portal. Finally, it creates a testuite for ONAP TOSCA related tests.
Change-Id: I613358c445cfe62730b98fee5e8c6fba7d322e90
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
Change-Id: I9a3d56932ce41191901381831013768d1c57a749
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
Please refer to env_config.sh.onap.sample when testing it.
The guide of preparing local env for onap package validation can be found here
https://gerrit.onap.org/r/#/c/75119
Change-Id: I39d6b11544847756126623a7eb3b953dc800c470
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. use copy commands instead of pre_copy
2. remove redundant items in project conf files
3. make the volume mapping clearer
Change-Id: I6c6aa58fac65d7e40105e0a54f6544ee5c47db31
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The item 'extra_container' used to be available for Bottlenecks only.
Make it open for Dovetail framework.
Change-Id: If4f2c032924447b97e401d497aca0309c0f5d847
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. fix the bug 'ShellRunner' object has no attribute 'archive_logs'
2. remove the testarea_supported because it's not very necessary for the test
cases themselves and can make it a little more simple to add a test case
with new test area
JIRA: DOVETAIL-750
Change-Id: I71ee74615200376adca2a0db040753e5fce329bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
In order to integrate k8s test cases, Dovetail framework should do some
refactor and make it more general for k8s test cases as well as ONAP ones.
Integrate 2 k8s test cases.
JIRA: DOVETAIL-748
Change-Id: Ibd87754ffb5fb29f6b4ce79232af860c2ed2da9c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for report file methods of Dovetail which were
missing.
Change-Id: I2ba7c8f9fcfb962988b2904b01538e9f89a0d849
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for run file methods of Dovetail which were
missing.
Change-Id: I1700c8c97430899abdc2b752a3dcbd4d09a334ac
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for Tescases classes methods of Dovetail
which were missing.
Change-Id: Ie1b665249ce8afaa76b59b0dbe7f132e404a5569
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for Runners classes methods of Dovetail
which were missing.
Change-Id: Ib985a44db86be02a7b9cefb8deca187bb50e78cc
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for container class methods of Dovetail
which were missing.
Change-Id: Iaba3bbfbb8838278e20ac9f63ef6e01f9ada1cbc
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for cli methods of Dovetail which were
missing.
Change-Id: I99c584007659401e298e58aebb1764df1c543894
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests which were missing from dovetail code base.
Furthermore it updates the test_parser existing unit test. Finally,
it fixes some minor issues in dovetail_utils itself.
Change-Id: I8fd7cd4f6b1ede11664914746d2279f062511639
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
As we discussed in Dovetail weekly meeting, there is no need to maintain
all the offline scripts.
Change-Id: I360d1a211856626af91310f142c1777a77398a12
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: Ia17bf7ef3e304eef7b3f0b2f7aa04e3be75bc8c3
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Remove the functions which are not called anywhere
2. Change some class methods to be static methods and instance methods
Change-Id: I2e754cdff61aa34ffe899e352653ff767d4ac69d
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I4fb24f86c6d15aea5a62baaca20531b2891437ff
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The LF pod used to run the CI verify jobs have updated the version of tox.
So the CI verify jobs failed.
See https://build.opnfv.org/ci/view/dovetail/job/dovetail-merge-master/476/console
Fix the python format errors
Ignore error 'W503 line break before binary operator'.
Change-Id: Ic2df070447bbea39c877ce7a394d733a95064bb1
Signed-off-by: xudan <xudan16@huawei.com>
|
|
If there is no pod.yaml, the results data looks like,
{"status": 2, "result": "[Errno 2] No such file or directory: u'/home/opnfv/userconfig/pre_config/pod.yaml'"}
It will get an exception when reading value from file,
File "/usr/local/lib/python2.7/dist-packages/dovetail/report.py", line 327, in crawl_from_file
criteria = data['result']['criteria']
TypeError: string indices must be integers
https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-baremetal-default-mandatory-master/52/console
Change-Id: I73ca3e5ae73433a2dc139789df4d67e8c583be08
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The official name of the next release is going to be 2018.09.
We need to rename all occurrences of 2018.08 in the code base,
the documentation, and the web portal.
JIRA: DOVETAIL-740
Change-Id: Ie0e532d6671b7590454e9afd960a2934f313a022
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The endpoints info for the 2018.08 has been changed.
The web portal needs some adaptions for the new data format.
It keeps the same as 2018.01 and doesn't need to change dovetail-webportal.
JIRA: DOVETAIL-725
Change-Id: I74cde3aa6032c7afac4b6ce1d2146e09a0f99fe5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
If mapping to host's /tmp, it won't clean all .log and .out files before
run a new test. It causes the results file including all previous results.
See https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-baremetal-default-mandatory-master/26/artifact/results/stress_logs/bottlenecks.stress.ping.out/*view*/
It will affect the results check.
Mapping to DOVETAIL_HOME/results/bottlenecks, which will always be cleaned
before running another test.
Change-Id: I22bfc871d6f041ae38308bf8e8f685993a376973
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Replacing the generic "dovetail" prefix of the testcase definitions with
the name of the OPNFV test framework which hosts the test.
JIRA: DOVETAIL-719
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
Change-Id: I8f90423b8908197bdc7adddf82c61ff394893f52
|
|
The results.json file is currently not a properly formated json file,
but contains one json structure per line/testcase. Whit this change, all
test results are being collected at runtime and then written to a
properly formated json file at the end of a test run.
Moreover, this patch introduces 'mandatory' and 'optional' sections to
the test suites definition files in order to clearly indicate if a test
is part of the corresponding test scope.
The web portal needs to be adapted accordingly to be able to read the
new results file.
JIRA: DOVETAIL-672
Change-Id: Id0706c7e82622fc9c2c2fece26b2d6c07c1580be
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The 2 sections 'functest_testsuite' and 'functest_testcase' are used by
all Functest test cases.
They aim to parse the results of Tempest sub test cases.
Remove them, and check the sub test case list,
1. if it's empty, that means no need to check sub test cases
2. if it is not empty, check the sub test cases one by one.
Additional, remove some useless keys in dovetail_config.yml
JIRA: DOVETAIL-658
Change-Id: Ic829264b0c8eaedd8e2b3fa55b38a34644482c20
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Update test suite ovp.next according to the latest OVP scope.
Modify Dovetail daily jobs to run all the scope.
There should be 2 jobs.
One for all mandatory test cases and the other for all optional ones.
That's mainly because of that the total executed time will be too large
(more than 300 minutes). Then the job will always failed because of time out.
It's hard to enlarge the time because it's already larger than common 3 hours.
Split it into 2 jobs can avoid the time out issue and make the results
clearer for reviewing.
JIRA: DOVETAIL-694
Change-Id: Ie0ea6221868941781af1477f7c7719f7cb4351a4
Signed-off-by: xudan <xudan16@huawei.com>
|
|
These 2 commands can't work now.
Also there is no need to support them, so remove them.
JIRA: DOVETAIL-695
Change-Id: I6707c7fc1d19fe06690cd14ef39f9d421d00a2bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
DPDK-enabled NFVIs require support for hugepages which
in turn is configured by means of extra_specs in flavors.
Currently, Functest enables hugepage support if the name
of the scenario contains the substring "ovs".
Dovetail allows end users to run all tests on DPDK-enabled systems
and the documentation has been updates accordingly.
JIRA: DOVETAIL-665
Change-Id: If4898da168f8b4c69a045ef3b00b8a8862f29b0c
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
Proposed solution by CI server:
"Avoid dangerous file parsing & serialization libs, use yaml.safe_load"
Change-Id: Iff9764601bc135cec271e16444d0bbbb4da8d459
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
Change-Id: I9f33bb320452efab79be5a22e625efac820183eb
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
Updating the licence headers to comply with the community guidelines [1]
and anteater checks.
[1] https://wiki.opnfv.org/display/DEV/Contribution+Guidelines#ContributionGuidelines-License
Change-Id: Idec787e32daf76c0b1bd6108977b78828125da5e
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
Use openstack shade to get endpoint infomation rather than executing
openstack commands directly.
The next step can replace openstack commands by shade for some test cases that
need to create openstack resources (images, flavors) if we agree to use shade.
JIRA: DOVETAIL-659
Change-Id: I72b302fb2d4a61e9249a7e53f81ea9464944dcd3
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-625
Change-Id: Ib96e6e39bad708f495af97ed43884909e394da10
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
|
|
This commit fixes CLI run command to exit with non-zero exit status
when bad arguments are given for '--testsuite' or '--testarea'.
JIRA: DOVETAIL-660
Change-Id: I4539b19978d9e93d8a90be9000f36d0aa1895f71
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
|