Age | Commit message (Collapse) | Author | Files | Lines |
|
This patchs gives the ability to provide more than one check results
file. Furthermore, it does not alter the behavior of the existing tests
but prepares dovetail to be able to fetch information from many
sources.
Change-Id: I23cd23d2908ad6bdddd94b977813d0e81dc0c05a
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch removes the ONAP lifecycle deprecated test case.
Furthermore, it clears helper functions as well as unwanted
configuration.
Change-Id: Ib958457cf120ffb641a72be028d3a031741c44c6
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
This patch makes it able to fetch the version from the testsuite.
Furthermore, in case of ONAP test cases it adds the vnf_type to the
produced results.json file in order to be parsed from the results
portal. Finally, it creates a testuite for ONAP TOSCA related tests.
Change-Id: I613358c445cfe62730b98fee5e8c6fba7d322e90
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
Change-Id: I9a3d56932ce41191901381831013768d1c57a749
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
Please refer to env_config.sh.onap.sample when testing it.
The guide of preparing local env for onap package validation can be found here
https://gerrit.onap.org/r/#/c/75119
Change-Id: I39d6b11544847756126623a7eb3b953dc800c470
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. use copy commands instead of pre_copy
2. remove redundant items in project conf files
3. make the volume mapping clearer
Change-Id: I6c6aa58fac65d7e40105e0a54f6544ee5c47db31
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The item 'extra_container' used to be available for Bottlenecks only.
Make it open for Dovetail framework.
Change-Id: If4f2c032924447b97e401d497aca0309c0f5d847
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. fix the bug 'ShellRunner' object has no attribute 'archive_logs'
2. remove the testarea_supported because it's not very necessary for the test
cases themselves and can make it a little more simple to add a test case
with new test area
JIRA: DOVETAIL-750
Change-Id: I71ee74615200376adca2a0db040753e5fce329bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
In order to integrate k8s test cases, Dovetail framework should do some
refactor and make it more general for k8s test cases as well as ONAP ones.
Integrate 2 k8s test cases.
JIRA: DOVETAIL-748
Change-Id: Ibd87754ffb5fb29f6b4ce79232af860c2ed2da9c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for report file methods of Dovetail which were
missing.
Change-Id: I2ba7c8f9fcfb962988b2904b01538e9f89a0d849
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for run file methods of Dovetail which were
missing.
Change-Id: I1700c8c97430899abdc2b752a3dcbd4d09a334ac
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for Tescases classes methods of Dovetail
which were missing.
Change-Id: Ie1b665249ce8afaa76b59b0dbe7f132e404a5569
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for Runners classes methods of Dovetail
which were missing.
Change-Id: Ib985a44db86be02a7b9cefb8deca187bb50e78cc
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for container class methods of Dovetail
which were missing.
Change-Id: Iaba3bbfbb8838278e20ac9f63ef6e01f9ada1cbc
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests for cli methods of Dovetail which were
missing.
Change-Id: I99c584007659401e298e58aebb1764df1c543894
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
JIRA: DOVETAIL-724
This patch adds unit tests which were missing from dovetail code base.
Furthermore it updates the test_parser existing unit test. Finally,
it fixes some minor issues in dovetail_utils itself.
Change-Id: I8fd7cd4f6b1ede11664914746d2279f062511639
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
As we discussed in Dovetail weekly meeting, there is no need to maintain
all the offline scripts.
Change-Id: I360d1a211856626af91310f142c1777a77398a12
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: Ia17bf7ef3e304eef7b3f0b2f7aa04e3be75bc8c3
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Remove the functions which are not called anywhere
2. Change some class methods to be static methods and instance methods
Change-Id: I2e754cdff61aa34ffe899e352653ff767d4ac69d
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I4fb24f86c6d15aea5a62baaca20531b2891437ff
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The LF pod used to run the CI verify jobs have updated the version of tox.
So the CI verify jobs failed.
See https://build.opnfv.org/ci/view/dovetail/job/dovetail-merge-master/476/console
Fix the python format errors
Ignore error 'W503 line break before binary operator'.
Change-Id: Ic2df070447bbea39c877ce7a394d733a95064bb1
Signed-off-by: xudan <xudan16@huawei.com>
|
|
If there is no pod.yaml, the results data looks like,
{"status": 2, "result": "[Errno 2] No such file or directory: u'/home/opnfv/userconfig/pre_config/pod.yaml'"}
It will get an exception when reading value from file,
File "/usr/local/lib/python2.7/dist-packages/dovetail/report.py", line 327, in crawl_from_file
criteria = data['result']['criteria']
TypeError: string indices must be integers
https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-baremetal-default-mandatory-master/52/console
Change-Id: I73ca3e5ae73433a2dc139789df4d67e8c583be08
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The official name of the next release is going to be 2018.09.
We need to rename all occurrences of 2018.08 in the code base,
the documentation, and the web portal.
JIRA: DOVETAIL-740
Change-Id: Ie0e532d6671b7590454e9afd960a2934f313a022
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The endpoints info for the 2018.08 has been changed.
The web portal needs some adaptions for the new data format.
It keeps the same as 2018.01 and doesn't need to change dovetail-webportal.
JIRA: DOVETAIL-725
Change-Id: I74cde3aa6032c7afac4b6ce1d2146e09a0f99fe5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
If mapping to host's /tmp, it won't clean all .log and .out files before
run a new test. It causes the results file including all previous results.
See https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-baremetal-default-mandatory-master/26/artifact/results/stress_logs/bottlenecks.stress.ping.out/*view*/
It will affect the results check.
Mapping to DOVETAIL_HOME/results/bottlenecks, which will always be cleaned
before running another test.
Change-Id: I22bfc871d6f041ae38308bf8e8f685993a376973
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Replacing the generic "dovetail" prefix of the testcase definitions with
the name of the OPNFV test framework which hosts the test.
JIRA: DOVETAIL-719
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
Change-Id: I8f90423b8908197bdc7adddf82c61ff394893f52
|
|
The results.json file is currently not a properly formated json file,
but contains one json structure per line/testcase. Whit this change, all
test results are being collected at runtime and then written to a
properly formated json file at the end of a test run.
Moreover, this patch introduces 'mandatory' and 'optional' sections to
the test suites definition files in order to clearly indicate if a test
is part of the corresponding test scope.
The web portal needs to be adapted accordingly to be able to read the
new results file.
JIRA: DOVETAIL-672
Change-Id: Id0706c7e82622fc9c2c2fece26b2d6c07c1580be
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The 2 sections 'functest_testsuite' and 'functest_testcase' are used by
all Functest test cases.
They aim to parse the results of Tempest sub test cases.
Remove them, and check the sub test case list,
1. if it's empty, that means no need to check sub test cases
2. if it is not empty, check the sub test cases one by one.
Additional, remove some useless keys in dovetail_config.yml
JIRA: DOVETAIL-658
Change-Id: Ic829264b0c8eaedd8e2b3fa55b38a34644482c20
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Update test suite ovp.next according to the latest OVP scope.
Modify Dovetail daily jobs to run all the scope.
There should be 2 jobs.
One for all mandatory test cases and the other for all optional ones.
That's mainly because of that the total executed time will be too large
(more than 300 minutes). Then the job will always failed because of time out.
It's hard to enlarge the time because it's already larger than common 3 hours.
Split it into 2 jobs can avoid the time out issue and make the results
clearer for reviewing.
JIRA: DOVETAIL-694
Change-Id: Ie0ea6221868941781af1477f7c7719f7cb4351a4
Signed-off-by: xudan <xudan16@huawei.com>
|
|
These 2 commands can't work now.
Also there is no need to support them, so remove them.
JIRA: DOVETAIL-695
Change-Id: I6707c7fc1d19fe06690cd14ef39f9d421d00a2bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
DPDK-enabled NFVIs require support for hugepages which
in turn is configured by means of extra_specs in flavors.
Currently, Functest enables hugepage support if the name
of the scenario contains the substring "ovs".
Dovetail allows end users to run all tests on DPDK-enabled systems
and the documentation has been updates accordingly.
JIRA: DOVETAIL-665
Change-Id: If4898da168f8b4c69a045ef3b00b8a8862f29b0c
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
Proposed solution by CI server:
"Avoid dangerous file parsing & serialization libs, use yaml.safe_load"
Change-Id: Iff9764601bc135cec271e16444d0bbbb4da8d459
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
Change-Id: I9f33bb320452efab79be5a22e625efac820183eb
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
Updating the licence headers to comply with the community guidelines [1]
and anteater checks.
[1] https://wiki.opnfv.org/display/DEV/Contribution+Guidelines#ContributionGuidelines-License
Change-Id: Idec787e32daf76c0b1bd6108977b78828125da5e
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
Use openstack shade to get endpoint infomation rather than executing
openstack commands directly.
The next step can replace openstack commands by shade for some test cases that
need to create openstack resources (images, flavors) if we agree to use shade.
JIRA: DOVETAIL-659
Change-Id: I72b302fb2d4a61e9249a7e53f81ea9464944dcd3
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-625
Change-Id: Ib96e6e39bad708f495af97ed43884909e394da10
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
|
|
This commit fixes CLI run command to exit with non-zero exit status
when bad arguments are given for '--testsuite' or '--testarea'.
JIRA: DOVETAIL-660
Change-Id: I4539b19978d9e93d8a90be9000f36d0aa1895f71
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
|
|
1. All detailed tempest test cases results are recorded with
file tempest-report.html rather than tempest.log now.
2. Archive all results files include functest.log, yardstick.log ...
for each test case. Otherwise, they will all be in one log file.
3. Support to set the source files and dest files need to be archived with
the test case config files rather than hardcode with the source code.
4. Give the file which used to be parse the results (PASS/FAIL) with the
test case config files rather than hardcode with the source code.
Change-Id: I30f435d91ab90cf20def47007f177fe98187484d
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Return empty test case list when test suite is empty.
2. Don't quite understand why comment out all proposed_tests.
3. Move stress ping test case at the end of the list,
because it may broken the message queue.
Change-Id: Ib717fc91e3763be03f243f82e53679462ecdae04
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Adding the Neutron trunk port tempest tests to Dovetail and the
proposed_tests test suite.
JIRA: DOVETAIL-648
Change-Id: Ia77980a245fd1bca6e5f0e567467432e12bf974e
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The CLI of dovetail tool needs some improvement.
1. Add an option '--testcase' to support specify test case.
2. Add an option '--no-clean' to keep functest/yardstick/bottlenecks containers
for debugging after running the test cases.
3. Improve the local report section.
a. Remove some info that is not necessary
b. Make the info more clearly
JIRA: DOVETAIL-588
Change-Id: Ic769a6e2788b407c3df08ccd593a0ecdaa35e5ea
Signed-off-by: xudan <xudan16@huawei.com>
|
|
There are some special Docker setting items when creating
Bottlenecks Docker Cntainers.
Move the special settings to config files.
Disable the function to push results to DB.
JIRA: DOVETAIL-645
Change-Id: Icc033222edf74c0fe4853dede2794f4226fd4252
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Remove local DB and can just report without launch local DB.
Need to do a little modification with the web portal code.
JIRA: DOVETAIL-593
Change-Id: I595db39cce4156596ee0522d4b7e642a23d78ab9
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Adding the new Patrole test case of Functest to Dovetail and to the
proposed_tests test suite.
JIRA: DOVETAIL-649
Change-Id: I2c3f583cdbd3af75cec020b0fe97f287d23498a5
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The original refstack results file is refstack/refstack.log
For Fraser, it has been changed into refstack/tempest.log
Change-Id: I7d40abe0c5a366b5fbb1912ec58d15109f22c0d0
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Use Functest Fraser docker images whose tag is fraser
2. Use Yardstick stable docker images
3. Functest fraser has moved testcases.yaml file from functest/ci to xtesting/ci
4. Functest fraser has removed 'functest env prepare' command
5. Functest uses rally to execute osinterop test cases
instead of refstack_client
Change-Id: I81ebefe6ea959ca77c2c87f1d75ef56ac18c165f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
This patch adds the ability to Dovetail to disable strict API
response validation in Tempest-based test cases run by Functest.
Corresponding updates of the web portal and backporting of this patch to
OVP 1.0.0 (Danube) will follow.
JIRA: DOVETAIL-633
Change-Id: Iace99ea1b6224ea907a2c3af8676e9285e6ad3ee
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
There are some special Docker setting items when creating
Yardstick Docker Containers.
1. Remove the extra settings which are required by Yardstick Danube.
2. Remove the settings for pushing results to DB because Dovetail
plan to disable this function.
3. Move the other special Functest settings to config files.
JIRA: DOVETAIL-640
Change-Id: Ic2d90393e227d7886b14d805eaa307b7aef56e4a
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently each type of test cases will use one docker image.
2. For example,
Functest test cases use opnfv/functest-restapi:euphrates
Yardstick uses opnfv/yardstick:opnfv-5.1.0
3. It needs to support to specify the docker image for each test case.
4. For example,
vping test cases use opnfv/functest-smoke:euphrates
sdnvpn uses opnfv/functest-features:euphrates
5. The benefit of this is that these docker images are well verified
by Functest CI and from Functest plan, they may not support
functest-restapi docker image in its future release.
JIRA: DOVETAIL-638
Change-Id: I8a30eea2233aeff809af2e241b6c939194397832
Signed-off-by: xudan <xudan16@huawei.com>
|
|
There are some special Docker setting item when creating Functest Containers.
1. Remove the extra settings which are required by Functest Danube.
2. Remove the settings for pushing results to DB because
Dovetail plans to disable this function.
3. Move the other special Functest settings to config files.
JIRA: DOVETAIL-639
Change-Id: Ibc4090b5c0aafdf9ed64e65a9a92e34a12a16df5
Signed-off-by: xudan <xudan16@huawei.com>
|