Age | Commit message (Collapse) | Author | Files | Lines |
|
These 2 commands can't work now.
Also there is no need to support them, so remove them.
JIRA: DOVETAIL-695
Change-Id: I6707c7fc1d19fe06690cd14ef39f9d421d00a2bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
DPDK-enabled NFVIs require support for hugepages which
in turn is configured by means of extra_specs in flavors.
Currently, Functest enables hugepage support if the name
of the scenario contains the substring "ovs".
Dovetail allows end users to run all tests on DPDK-enabled systems
and the documentation has been updates accordingly.
JIRA: DOVETAIL-665
Change-Id: If4898da168f8b4c69a045ef3b00b8a8862f29b0c
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
Change-Id: I9f33bb320452efab79be5a22e625efac820183eb
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
JIRA: DOVETAIL-625
Change-Id: Ib96e6e39bad708f495af97ed43884909e394da10
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
|
|
This commit fixes CLI run command to exit with non-zero exit status
when bad arguments are given for '--testsuite' or '--testarea'.
JIRA: DOVETAIL-660
Change-Id: I4539b19978d9e93d8a90be9000f36d0aa1895f71
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
|
|
1. All detailed tempest test cases results are recorded with
file tempest-report.html rather than tempest.log now.
2. Archive all results files include functest.log, yardstick.log ...
for each test case. Otherwise, they will all be in one log file.
3. Support to set the source files and dest files need to be archived with
the test case config files rather than hardcode with the source code.
4. Give the file which used to be parse the results (PASS/FAIL) with the
test case config files rather than hardcode with the source code.
Change-Id: I30f435d91ab90cf20def47007f177fe98187484d
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The CLI of dovetail tool needs some improvement.
1. Add an option '--testcase' to support specify test case.
2. Add an option '--no-clean' to keep functest/yardstick/bottlenecks containers
for debugging after running the test cases.
3. Improve the local report section.
a. Remove some info that is not necessary
b. Make the info more clearly
JIRA: DOVETAIL-588
Change-Id: Ic769a6e2788b407c3df08ccd593a0ecdaa35e5ea
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Remove local DB and can just report without launch local DB.
Need to do a little modification with the web portal code.
JIRA: DOVETAIL-593
Change-Id: I595db39cce4156596ee0522d4b7e642a23d78ab9
Signed-off-by: xudan <xudan16@huawei.com>
|
|
This patch adds the ability to Dovetail to disable strict API
response validation in Tempest-based test cases run by Functest.
Corresponding updates of the web portal and backporting of this patch to
OVP 1.0.0 (Danube) will follow.
JIRA: DOVETAIL-633
Change-Id: Iace99ea1b6224ea907a2c3af8676e9285e6ad3ee
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
Change-Id: I6a1fa5f27df2f5127eb00156b3135dc79caf5e77
Issue-ID: DOVETAIL-629
Signed-off-by: Moshe <moshehoa@amdocs.com>
|
|
Change-Id: Ie470d03bd168c909fd02c316523049c67a6bb095
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
Specifying separate valid docker tags for functest and yardstick for
input validation.
DOVETAIL-592
Change-Id: I3365cdbd9ab521bf1368750c3f34b6863b3f7ec8
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The function 'check_tc_result' may return 'None'.
In this case it will throw an exception because of
https://github.com/opnfv/dovetail/blob/master/dovetail/run.py#L65
The exception is like
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 339, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in _call_
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 304, in main
duration = run_test(testsuite_yaml, testarea, logger, kwargs)
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 65, in run_test
if stop_on_fail['criteria'] == "FAIL" and kwargs['stop']:
TypeError: 'NoneType' object has no attribute '_getitem_'
JIRA: DOVETAIL-622
Change-Id: I9e612b380c3dfb148692074a3bf8eb31875213ac
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. mv conf/ userconfig/ compliance/ patch to etc/
2. after installation etc/ will be put under /etc/dovetail
3. leveraging /etc/dovetail in coding
Change-Id: Icf9cb00ff7fc7cf35755f60d3e258b23e2c70839
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
- Improve test execution control by adding stop on failure flag
- Modify test check function to return value regardless of whether
report or local file option is employed
- Modify run test loop to check for failure after every test case
if stop flag set
Change-Id: I27a4f3aaaa8b8cb8c52eb3f092f52048980abb44
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- Try new license header to deal with anteater failure
- Ability to specify 'default' as argument to --report, which
fetches host IP automatically and forms URL with 8000 as testapi
port
- This should not break the previous behavior of report option,
if default option not specified
Change-Id: I1e05bb52cf4fc211c59796d4e763e2e04177b406
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
Change default container tag to be:
1. Functest ovp.1.0.0
2. testapi ovp.1.0.0
Add test suite ovp.1.0.0 and set it to be the default test suite.
Comment out bottlenecks CLI for initial release.
JIRA: DOVETAIL-580
Change-Id: I956e49e43fc7ba0176c14079b7228089db5dc108
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Get the admin endpoints of all service.
JIRA: DOVETAIL-546
Change-Id: Ie0ac3bc5b4eb0885046e207b73217994a4f0e358
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Hardware info:
Use ansible to get all hardware info from all nodes.
JIRA: DOVETAIL-540
Change-Id: I6a2dde1b2ebae2af8f702008c23110ebe63cc06f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
|
|
1. Dovetail generate a tar file for uploading results to CVP web portal.
2. If users don't run with option --report, it shouldn't generate the tar file.
Because users can not upload results to CVP if they don't run with --report.
3. If it generates the tar file when user run dovetail without --report,
the tar file is invalid because it lacks of results.json.
JIRA: DOVETAIL-511
Change-Id: I4d21f21f0b278962564d810369df03100b38f003
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, the log level of yardstick is DEBUG and cannot be modified.
2. Make it can be changed from CLI.
3. When use dovetail run --debug, can see all Yardstick logs (including DEBUG)
in dovetail.log.
4. When run without debug option, cannot see Yardstick debug logs in
dovetail.logs.
5. yardstick.log always doesn't include debug logs.
JIRA: DOVETAIL-506
Change-Id: Iaafb61232a22e16fff2fa4605bf7e262b022085e
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, dovetail can run a test suite or a test area in that test suite.
2. The test areas in one test suite are not divided into optional and mandatory.
3. Split them and support to run just mandatory or optional.
4. Support to run multiple test areas.
JIRA: DOVETAIL-505
Change-Id: I42cd7b4e11c3e3674c806e9bc999b782bf5c85c6
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. According to ha test specification doc, the SLA of process recover time
is 30 seconds.
2. However, the SLA in Yardstick test cases are all 20 seconds.
3. Add a patch to change the SLA in yardstick test cases.
JIRA: DOVETAIL-496
Change-Id: Ib8f229fdfbb3289bf70d6ee900132b3db3a21b17
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-479
User may want to upload all logs along with the results,
so others may check the running logs to make sure what' s going on during the test.
Change-Id: Ice906264bbdf091d3353d6d2e146653aeb057f5f
Signed-off-by: grakiss <grakiss.wanglei@huawei.com>
|
|
JIRA: DOVETAIL-474
Add Bottlenecks resiliency test case "posca_factor_ping" into proposed_tests.
Change-Id: I76aea651014fb4722ddae07559f417d3f840eee2
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-469
uuid1 is more unique than uuid4
Change-Id: Ic29ec7586ba518e67cb1750df1001cd390fcf313
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-450
Change-Id: I795679c9559d1bead3aecefff28d21b6567aac2f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JITA: DOVETAIL-449
1. Dovetail cvp.0.1.0 link dovetail/dovetail to
/usr/local/lib/python2.7/dist-packages/dovetail/
2. The directory dovetail/userconfig is lost in this image
3. Move dovetail/userconfig into dovetail/dovetail/userconfig
Change-Id: I9da8389e02d92f35879ff5c6a1db1b7d984dd862
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
|
|
|
|
JIRA: DOVETAIL-440
Change-Id: I69fb96673755545e8e0d78d940870195dbf20a0b
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-437
JIRA: DOVETAIL-444
1. Docker client version and server version should be 1.12.3 or later.
2. If any of them is older than 1.12.3, there will be an ERROR message.
3. Delete the “rm -rf results” message.
Change-Id: I0c10cf256ef139cac4d43f36108bf5f3f9da8398
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-445
https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/19/console
Change-Id: If933156bb3931e7bfc8ec31eb03ca5a3cb0bf8b0
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
JIRA: DOVETAIL-428
1. Change the name of Yardstick result file from opnfv_yardstick_tc019.out to
dovetail.ha.tc001.out. Because opnfv_yardstick_tc019 is the name defined in
Yardstick and dovetail.ha.tc001 is the name defined in Dovetail.
2. Check whether the file exists before reporting the results are stored in file.
3. Delete test cases vping_ssh, vping_userdata and sdnvpn.tc003 from the testsuite
proposed_tests.
Change-Id: I4cb16947729479616a03abd9e91b76f9a41a949c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-438
1. Need an env variable DOVETAIL_HOME in env_config.sh
2. env_config.sh, pod.yaml and id_rsa should be put in path DOVETAIL_HOME/pre_config
3. the results will also be put in this path
4. no need to give the path of openstack credential file while running a testsuite
5. if running with the code, source the env_config.sh file before running
source $DOVETAIL_HOME/env_config.sh
dovetail run --testsuite <testsuite_name>
6. if running with Dovetail docker image
docker run -it --privileged=true -e DOVETAIL_HOME=<dovetail_home_path>
-v <dovetail_home_path>:<dovetail_home_path>
-v /var/run/docker.sock:/var/run/docker.sock
opnfv/dovetail:latest /bin/bash
Change-Id: Ia23078f27a34e293d0075e49c673930f41069cb6
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-421
1. Yardstick is working on supporting push results to mongoDB.
see https://gerrit.opnfv.org/gerrit/#/c/34091/
2. make Dovetail supporting push results to database too when
user choose to report to database.
Change-Id: Iee06ad6be59dad62ac4f985f4a982fb5ff93a592
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-417
1. If report results with files, all results will be stored in local files.
2. If report to database, will get results from database and stored with file results.json
Change-Id: Ic30037bd66cd37042f82b75fac2043a876e21c6f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-414
Dovetail requires docker at version > 1.8, so add some validation in code, and also versions of client and server should match
with each other, or there will be unpredictable errors.
Gives a notice to user to decide whether to update the docker to the latest version(1.17)
Change-Id: I1331f5c19943a73e8fe25f76bd21628233e556f3
Signed-off-by: grakiss <grakiss.wanglei@huawei.com>
|
|
JIRA: DOVETAIL-406
1. When there are something wrong with the test case tempest_custom,
it's details json data are "details": {}, So it will throw KeyError
exception in this situation.
2. Catch this exception and set all sub_testcases FAIL.
3. Disable the exceed_max_retry_times().
Change-Id: I6bdfc5944a535625b2211a3bd8096a13b81b339e
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-401
1. modify all Ipv6 test cases, change tempest_smoke_serial/tempest_full_parallel
into tempest_custom, add ID of each sub-testcases.
2. Some sub-testcases in ipv6.tc002, ipv6.tc003, ipv6.tc004 and ipv6.tc005 can not
be found in the full tempest test case list.
2. disable testcases' script_result_acquired to enable tempest_custom run multiple times.
3. use build_tag-testcase_name(such as daily-master-79a6e392-28bb-4c40-9d88-a03bed99da2d-dovetail.ipv6.tc025)
as the unique ID of each testcase.
4. comment out the progress bar, because it has some problems when print the report text.
5. Functest has changed the TEST_DB_URL from http://<testapi>:<port>/apv/v1 to
http://<testapi>:<port>/apv/v1/results.
6. Functest has changed the TEST_DB_URL to file://<file_path>/<file_name>.
see https://gerrit.opnfv.org/gerrit/#/c/33511/
Change-Id: I524f6702377f215f526158214842481eb44565be
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
JIRA: DOVETAIL-400
1. set both the dovetail.log and screen log level by command --debug.
2. if --report http://testapi_url:port/api/v1/, change it into
--report http://testapi_url:port/api/v1
Change-Id: I7cf697d4ac9072fc2d302a67ccde0f452d84be8f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I2da5e7844f5ce034d4c0cfaa4592e6cd47002049
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
JIRA: DOVETAIL-375
JIRA: DOVETAIL-390
1. defcore test cases are erroneously judged because of:
a) defcore results json data lack some keys
b) some sub-testcases' names are the same as the prefix of another sub-testcases.
2. this patch removes the extra keys and judge sub-testcases accurately.
3. remove "store_type" in functest_config.yml and yardstick_config.yml.
4. change some log levels from info to debug.
Change-Id: I5f126e70309409cac66ef9a0909f1573d325098b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
defcore.txt
JIRA: DOVETAIL-394
for refstack, userguide is listed in https://gerrit.opnfv.org/gerrit/#/c/33049/
in CI, use the automate way to generate tempest.conf to let refstack run against SUT, i.e., defcore.tc001.yml
will provide a way to let user can let refstack run against SUT manually, i.e. defcore.tc002.yml
since the tempest.conf autometely generated may not satisfied with SUT, users can adjust its tempest.conf to let the defcore testcases pass.
Change-Id: I5ed212c9c6d864d352d1c497ea039cc553461158
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|