Age | Commit message (Collapse) | Author | Files | Lines |
|
This patch adds the ability to Dovetail to disable strict API
response validation in Tempest-based test cases run by Functest. This is
a backport of the changes from master, targeting OVP 1.0.1
Corresponding updates of the web portal will follow.
JIRA: DOVETAIL-633
Change-Id: Iace99ea1b6224ea907a2c3af8676e9285e6ad3ee
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
Change default container tag to be:
1. Functest ovp.1.0.0
2. testapi ovp.1.0.0
Add test suite ovp.1.0.0 and set it to be the default test suite.
Comment out bottlenecks CLI for initial release.
JIRA: DOVETAIL-580
Change-Id: I956e49e43fc7ba0176c14079b7228089db5dc108
Signed-off-by: xudan <xudan16@huawei.com>
(cherry picked from commit c47b8dde0e39e273372b091a3d8da1cc15b7ae9f)
|
|
Get the admin endpoints of all service.
JIRA: DOVETAIL-546
Change-Id: Ie0ac3bc5b4eb0885046e207b73217994a4f0e358
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Hardware info:
Use ansible to get all hardware info from all nodes.
JIRA: DOVETAIL-540
Change-Id: I6a2dde1b2ebae2af8f702008c23110ebe63cc06f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
|
|
1. Dovetail generate a tar file for uploading results to CVP web portal.
2. If users don't run with option --report, it shouldn't generate the tar file.
Because users can not upload results to CVP if they don't run with --report.
3. If it generates the tar file when user run dovetail without --report,
the tar file is invalid because it lacks of results.json.
JIRA: DOVETAIL-511
Change-Id: I4d21f21f0b278962564d810369df03100b38f003
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, the log level of yardstick is DEBUG and cannot be modified.
2. Make it can be changed from CLI.
3. When use dovetail run --debug, can see all Yardstick logs (including DEBUG)
in dovetail.log.
4. When run without debug option, cannot see Yardstick debug logs in
dovetail.logs.
5. yardstick.log always doesn't include debug logs.
JIRA: DOVETAIL-506
Change-Id: Iaafb61232a22e16fff2fa4605bf7e262b022085e
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, dovetail can run a test suite or a test area in that test suite.
2. The test areas in one test suite are not divided into optional and mandatory.
3. Split them and support to run just mandatory or optional.
4. Support to run multiple test areas.
JIRA: DOVETAIL-505
Change-Id: I42cd7b4e11c3e3674c806e9bc999b782bf5c85c6
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. According to ha test specification doc, the SLA of process recover time
is 30 seconds.
2. However, the SLA in Yardstick test cases are all 20 seconds.
3. Add a patch to change the SLA in yardstick test cases.
JIRA: DOVETAIL-496
Change-Id: Ib8f229fdfbb3289bf70d6ee900132b3db3a21b17
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-479
User may want to upload all logs along with the results,
so others may check the running logs to make sure what' s going on during the test.
Change-Id: Ice906264bbdf091d3353d6d2e146653aeb057f5f
Signed-off-by: grakiss <grakiss.wanglei@huawei.com>
|
|
JIRA: DOVETAIL-474
Add Bottlenecks resiliency test case "posca_factor_ping" into proposed_tests.
Change-Id: I76aea651014fb4722ddae07559f417d3f840eee2
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-469
uuid1 is more unique than uuid4
Change-Id: Ic29ec7586ba518e67cb1750df1001cd390fcf313
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-450
Change-Id: I795679c9559d1bead3aecefff28d21b6567aac2f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JITA: DOVETAIL-449
1. Dovetail cvp.0.1.0 link dovetail/dovetail to
/usr/local/lib/python2.7/dist-packages/dovetail/
2. The directory dovetail/userconfig is lost in this image
3. Move dovetail/userconfig into dovetail/dovetail/userconfig
Change-Id: I9da8389e02d92f35879ff5c6a1db1b7d984dd862
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
|
|
|
|
JIRA: DOVETAIL-440
Change-Id: I69fb96673755545e8e0d78d940870195dbf20a0b
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-437
JIRA: DOVETAIL-444
1. Docker client version and server version should be 1.12.3 or later.
2. If any of them is older than 1.12.3, there will be an ERROR message.
3. Delete the “rm -rf results” message.
Change-Id: I0c10cf256ef139cac4d43f36108bf5f3f9da8398
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-445
https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/19/console
Change-Id: If933156bb3931e7bfc8ec31eb03ca5a3cb0bf8b0
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
JIRA: DOVETAIL-428
1. Change the name of Yardstick result file from opnfv_yardstick_tc019.out to
dovetail.ha.tc001.out. Because opnfv_yardstick_tc019 is the name defined in
Yardstick and dovetail.ha.tc001 is the name defined in Dovetail.
2. Check whether the file exists before reporting the results are stored in file.
3. Delete test cases vping_ssh, vping_userdata and sdnvpn.tc003 from the testsuite
proposed_tests.
Change-Id: I4cb16947729479616a03abd9e91b76f9a41a949c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-438
1. Need an env variable DOVETAIL_HOME in env_config.sh
2. env_config.sh, pod.yaml and id_rsa should be put in path DOVETAIL_HOME/pre_config
3. the results will also be put in this path
4. no need to give the path of openstack credential file while running a testsuite
5. if running with the code, source the env_config.sh file before running
source $DOVETAIL_HOME/env_config.sh
dovetail run --testsuite <testsuite_name>
6. if running with Dovetail docker image
docker run -it --privileged=true -e DOVETAIL_HOME=<dovetail_home_path>
-v <dovetail_home_path>:<dovetail_home_path>
-v /var/run/docker.sock:/var/run/docker.sock
opnfv/dovetail:latest /bin/bash
Change-Id: Ia23078f27a34e293d0075e49c673930f41069cb6
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-421
1. Yardstick is working on supporting push results to mongoDB.
see https://gerrit.opnfv.org/gerrit/#/c/34091/
2. make Dovetail supporting push results to database too when
user choose to report to database.
Change-Id: Iee06ad6be59dad62ac4f985f4a982fb5ff93a592
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-417
1. If report results with files, all results will be stored in local files.
2. If report to database, will get results from database and stored with file results.json
Change-Id: Ic30037bd66cd37042f82b75fac2043a876e21c6f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-414
Dovetail requires docker at version > 1.8, so add some validation in code, and also versions of client and server should match
with each other, or there will be unpredictable errors.
Gives a notice to user to decide whether to update the docker to the latest version(1.17)
Change-Id: I1331f5c19943a73e8fe25f76bd21628233e556f3
Signed-off-by: grakiss <grakiss.wanglei@huawei.com>
|
|
JIRA: DOVETAIL-406
1. When there are something wrong with the test case tempest_custom,
it's details json data are "details": {}, So it will throw KeyError
exception in this situation.
2. Catch this exception and set all sub_testcases FAIL.
3. Disable the exceed_max_retry_times().
Change-Id: I6bdfc5944a535625b2211a3bd8096a13b81b339e
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-401
1. modify all Ipv6 test cases, change tempest_smoke_serial/tempest_full_parallel
into tempest_custom, add ID of each sub-testcases.
2. Some sub-testcases in ipv6.tc002, ipv6.tc003, ipv6.tc004 and ipv6.tc005 can not
be found in the full tempest test case list.
2. disable testcases' script_result_acquired to enable tempest_custom run multiple times.
3. use build_tag-testcase_name(such as daily-master-79a6e392-28bb-4c40-9d88-a03bed99da2d-dovetail.ipv6.tc025)
as the unique ID of each testcase.
4. comment out the progress bar, because it has some problems when print the report text.
5. Functest has changed the TEST_DB_URL from http://<testapi>:<port>/apv/v1 to
http://<testapi>:<port>/apv/v1/results.
6. Functest has changed the TEST_DB_URL to file://<file_path>/<file_name>.
see https://gerrit.opnfv.org/gerrit/#/c/33511/
Change-Id: I524f6702377f215f526158214842481eb44565be
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
JIRA: DOVETAIL-400
1. set both the dovetail.log and screen log level by command --debug.
2. if --report http://testapi_url:port/api/v1/, change it into
--report http://testapi_url:port/api/v1
Change-Id: I7cf697d4ac9072fc2d302a67ccde0f452d84be8f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I2da5e7844f5ce034d4c0cfaa4592e6cd47002049
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
JIRA: DOVETAIL-375
JIRA: DOVETAIL-390
1. defcore test cases are erroneously judged because of:
a) defcore results json data lack some keys
b) some sub-testcases' names are the same as the prefix of another sub-testcases.
2. this patch removes the extra keys and judge sub-testcases accurately.
3. remove "store_type" in functest_config.yml and yardstick_config.yml.
4. change some log levels from info to debug.
Change-Id: I5f126e70309409cac66ef9a0909f1573d325098b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
defcore.txt
JIRA: DOVETAIL-394
for refstack, userguide is listed in https://gerrit.opnfv.org/gerrit/#/c/33049/
in CI, use the automate way to generate tempest.conf to let refstack run against SUT, i.e., defcore.tc001.yml
will provide a way to let user can let refstack run against SUT manually, i.e. defcore.tc002.yml
since the tempest.conf autometely generated may not satisfied with SUT, users can adjust its tempest.conf to let the defcore testcases pass.
Change-Id: I5ed212c9c6d864d352d1c497ea039cc553461158
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
JIRA: DOVETAIL-381
Now DoveTail's results dir is defined in $DOVETAIL_HOME/dovetail/conf/dovetail_config.yml,
and its default value is /home/opnfv/dovetail/results.
Since customer's jumphost may not have this dir and they don't want to create it,
Dovetail needs to change this dir into $DOVETAIL_HOME/results.
Change-Id: I616fb2da87bcfaabfd78d8762897e378168f694d
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-380
If Functest fails to push results to db, there is just an error log in functest.log
Dovetail needs to know the failure.
Dovetail will get the results from db with keys, build_tag and testcase.
If it can get the results data, it means push_to_db success.
Fix the duration time wrong.
Change-Id: If03bdf0ceb2f1a9d422381cdf06e1fd207358bd5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-379
1. create a uuid for each test, and set build_tag=daily-master-<uuid>
2. the reason it starts with "daily_master" is that it's the requirement of
functest when push data to DB.
Change-Id: I9a59d5695a9f0d83bf4330379f1949a2a04f876c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-164
usage:
dovetail run --testsuite <testsuite> --offline
1,if --offline given, can run offline by using local docker images
(if there is no images, will get an error in log, images can be
pulled from dockerhub or downloaded from artifacts.opnfv.org )
2,if --offline not given, will run online, which means
pull docker image from dockerhub
Change-Id: I87ab35c82e59fb13a7a88c38d233e89285efc4d3
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-373
1. add an option --report/-r to choose the report type.
2. "-r file" will store results with files.
3. "-r http://192.168.135.2:8000/api/v1" will push results into DB.
The DB url can replaced by user's local DB or official DB.
4. Default report type is defined as "report_dest" in dovetail_config.yml.
5. Yardstick can't push results of one testcase into DB now, so both "-r file"
and "-r http://..." will be regarded as "file".
Change-Id: Ieaa017ab59c5b15235bf306ef57cdc56beb22af8
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-371
1. remove all the envs in functest_config.yml and yardstick_config.yml
2. remove all relative cmds in cmd_config.yml
3. remove all the envs when running a new functest/yardstick container
4. add some variables just for functest pushing results to db
5. get the external network name for yardstick
6. make --debug/-d just control the log level shown on the screen
7. set requests=2.10.0 since python-openstackclient needs
'requests!=2.12.2,!=2.13.0,>=2.10.0'
Change-Id: I9f941e09d067fc95a14c3c9be1374f41df4a9a16
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-178
Change-Id: Ica814d4ff634c9159e4ecca888b8cc0f6c16e624
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-166
Check the results of each cmds executed in test case
1. the results of pre_condition, post_condition, cmds need to be checked,
so it can get a quick fail, dont need to go through the next step
2. it's more accurate to show where error occurred as early as possible
3. get results from shell scripts
Change-Id: I5c1e59839c55b92de0e83e7e1eb552aa364b3f80
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
|
|
|
|
JIRA: DOVETAIL-162
Change-Id: Idc78790e49ab3e3552d45b902cfdf8cc37fd2847
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
1. move conf/dovetail_config.py into utils/
2. change the open file path in dovetail_config.py
3. do some modification about all 'from conf.dovetail_config import ***'
JIRA: DOVETAIL-158
Change-Id: I7670e6de993899e4e2a604cbc501dc3339896ef1
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. modify cmd_config.yml
a. change 'env' and 'non-env' into 'config' and 'control'
b. add subsection 'path' for each block in 'config'
c. delete 'DEPLOY_SCENARIO' and 'DEPLOY_TYPE'
d. use 'func_tag' and 'yard_tag' to control container's tag
2. if cmd in config
a. if its path is envs, it will be used to update envs in dovetail_config
b. if its path is non_envs, it will be used to update other value in the dict
JIRA: DOVETAIL-70
Change-Id: I1cf9bfcd1a19294a390a85bce6e458ce50672b7f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-46
1. for now a testcase has two kinds of validation types(functest, yardstick),
and it is not enough to check the complete funcionality
2. add new validation type(shell) for extra validation of the test case to make
result more accurate and more convincing.
Change-Id: I45dca6b8dbd888757da163189d261f6e4dba5034
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
|
|
screen"
|