Age | Commit message (Collapse) | Author | Files | Lines |
|
This doucment needs to be +2'd by the majority of committers prior to merging
to ensure we are all in agreement with regard to the test case requirements.
This is a pre-requisite to test case selection for the CVP.
Updated by dneary 2017-04-25: List formatting, and reorganising requirements for installer and stack composition.
Change-Id: Iabaf11bc9479a20a58f985233d460013e4690d7c
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
|
|
iterative inprovements."
|
|
|
|
inprovements.
JIRA tasks are being created for specific sections of the document.
Refer to the doc for links to relevant JIRA tasks.
Change-Id: I1717d56b8817c38802a227db320f30029f68fbd0
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
|
|
|
|
JIRA: DOVETAIL-405
1. All HA test cases running in CI are SSH timeout.
2. The reason is that HA test cases need a file pod.yaml providing the info of
each node of Openstack.
3. This pod.yaml should be put in $DOVETAIL_HOME/dovetail/userconfig by users.
4. Dovetail's CI jobs should also do some adaptions. JIRA: DOVETAIL-407
5. modify file .gitignore to ignore directories results/ and userconfig/.
Change-Id: I5c681445ec83c9d67a0151370f5b56840ffe444c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-406
1. When there are something wrong with the test case tempest_custom,
it's details json data are "details": {}, So it will throw KeyError
exception in this situation.
2. Catch this exception and set all sub_testcases FAIL.
3. Disable the exceed_max_retry_times().
Change-Id: I6bdfc5944a535625b2211a3bd8096a13b81b339e
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Add aarch64 patch for Dovetail docker file;
The patch should be applied from releng
Move to Ubuntu 16.04 in order, since docker.io package was not
available for arm on Ubuntu 14.04.
Added "pip install -U setuptools", since installation of other
components failed because setuptools was missing.
Change-Id: If00346d89c48be86c6d9b830f75c1792c39d6bb3
Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
|
|
|
|
1) ha testcases need node configure file, skip it temporarily
2) tempest.conf-->refstack_tempest.conf, then it will not covered
by other tempest testcases, can used for users to debug
Change-Id: Ieb7101d8c406e294755629fd11a686f90b6c0a7f
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-404
1. IPV6 sub-testcase's name includes [id], which will affects the regex.
2. Remove the [id] from the name of the sub_testcases.
Change-Id: I51b567e8382fa4e701f8b32ba7ea195d49fd31eb
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
JIRA: DOVETAIL-401
1. modify all Ipv6 test cases, change tempest_smoke_serial/tempest_full_parallel
into tempest_custom, add ID of each sub-testcases.
2. Some sub-testcases in ipv6.tc002, ipv6.tc003, ipv6.tc004 and ipv6.tc005 can not
be found in the full tempest test case list.
2. disable testcases' script_result_acquired to enable tempest_custom run multiple times.
3. use build_tag-testcase_name(such as daily-master-79a6e392-28bb-4c40-9d88-a03bed99da2d-dovetail.ipv6.tc025)
as the unique ID of each testcase.
4. comment out the progress bar, because it has some problems when print the report text.
5. Functest has changed the TEST_DB_URL from http://<testapi>:<port>/apv/v1 to
http://<testapi>:<port>/apv/v1/results.
6. Functest has changed the TEST_DB_URL to file://<file_path>/<file_name>.
see https://gerrit.opnfv.org/gerrit/#/c/33511/
Change-Id: I524f6702377f215f526158214842481eb44565be
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-398
JIRA: DOVETAIL-399
1. this testsuite will run in community CI, see
https://gerrit.opnfv.org/gerrit/#/c/33509/
2. HA testcases please see
https://wiki.opnfv.org/pages/viewpage.action?pageId=8684344
3. vimops testcases deleted, since it's covered in defcore
Change-Id: Ied9c8867e0aa99e33fd213030e6471aa5cfb86e8
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
see logs
https://build.opnfv.org/ci/job/dovetail-verify-master/560/console
Change-Id: Ifa66087c4c8fcbaae809aae380587a4f03eded6c
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
JIRA: DOVETAIL-400
1. set both the dovetail.log and screen log level by command --debug.
2. if --report http://testapi_url:port/api/v1/, change it into
--report http://testapi_url:port/api/v1
Change-Id: I7cf697d4ac9072fc2d302a67ccde0f452d84be8f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I2da5e7844f5ce034d4c0cfaa4592e6cd47002049
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
JIRA: DOVETAIL-376
Change-Id: Ia269ca7cdd34c71b4531b28e42e40415190ae398
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-395
1. since plugfest has its own plan, the files are removed
Change-Id: I4989d1e0e1af483205733b3130e3f1152a8933c7
Signed-off-by: hongbotian <hongbo.tianhongbo@huawei.com>
|
|
|
|
JIRA: DOVETAIL-375
JIRA: DOVETAIL-390
1. defcore test cases are erroneously judged because of:
a) defcore results json data lack some keys
b) some sub-testcases' names are the same as the prefix of another sub-testcases.
2. this patch removes the extra keys and judge sub-testcases accurately.
3. remove "store_type" in functest_config.yml and yardstick_config.yml.
4. change some log levels from info to debug.
Change-Id: I5f126e70309409cac66ef9a0909f1573d325098b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
and defcore.txt"
|
|
defcore.txt
JIRA: DOVETAIL-394
for refstack, userguide is listed in https://gerrit.opnfv.org/gerrit/#/c/33049/
in CI, use the automate way to generate tempest.conf to let refstack run against SUT, i.e., defcore.tc001.yml
will provide a way to let user can let refstack run against SUT manually, i.e. defcore.tc002.yml
since the tempest.conf autometely generated may not satisfied with SUT, users can adjust its tempest.conf to let the defcore testcases pass.
Change-Id: I5ed212c9c6d864d352d1c497ea039cc553461158
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-159
1. It takes long time to execute test cases that most of users think it hangs there,
so a progress is needed to show the test is running.
2. add dynamic progress during execution
3. it' s a fake progress only showing the test is alive,
not to show the percentage of completion.
Change-Id: I0294659f79e8e09d1bcdd36d8f391d933e35b635
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
|
|
since yardstick changes its openrc location
see
https://gerrit.opnfv.org/gerrit/#/c/32861/
Change-Id: Ib85910fe5521fb398c47e389e2baf127342b1c0b
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
JIRA: DOVETAIL-375
1, together with other testsuite from functest, such as tempest etc.
this should merge after https://gerrit.opnfv.org/gerrit/#/c/31313/
2, delete some unnecessary definition
3, bugfix of docker cp inside container, see
https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-debug-master/361/console
4, disable tempest testcaes since changed to refstack_defcore
Change-Id: I67503e348af4a3f697f1b4106bfe81c05add8aa0
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
JIRA: DOVETAIL-381
Now DoveTail's results dir is defined in $DOVETAIL_HOME/dovetail/conf/dovetail_config.yml,
and its default value is /home/opnfv/dovetail/results.
Since customer's jumphost may not have this dir and they don't want to create it,
Dovetail needs to change this dir into $DOVETAIL_HOME/results.
Change-Id: I616fb2da87bcfaabfd78d8762897e378168f694d
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-380
If Functest fails to push results to db, there is just an error log in functest.log
Dovetail needs to know the failure.
Dovetail will get the results from db with keys, build_tag and testcase.
If it can get the results data, it means push_to_db success.
Fix the duration time wrong.
Change-Id: If03bdf0ceb2f1a9d422381cdf06e1fd207358bd5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-388
Change-Id: I3611624c955fa735aabfd00b456bd7f8455080e5
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
|
|
|
|
JIRA: DOVETAIL-370
Change-Id: I60c9f431358f848e24abcb865c4b2ca8c3b2d843
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-383
The script launch_db.sh can just support ubuntu now.
Make it support centos 7.
Change-Id: Ic6123ac82cb81bacb2d0c2d394619b8d8d7b3296
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-379
1. create a uuid for each test, and set build_tag=daily-master-<uuid>
2. the reason it starts with "daily_master" is that it's the requirement of
functest when push data to DB.
Change-Id: I9a59d5695a9f0d83bf4330379f1949a2a04f876c
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-378
1. test_runner checks the result of cmd "pull image". If fail to pull,
then break the running.
2. pull_image will return None when there are errors, otherwise return image name.
3. if there is an old image named the same as the new one, the old one's tag
will be changed into <none>. In this case, the old image needs to be removed.
4. if there are some containers using this old image, the remove will be skipped
rather than logging error.
Change-Id: I940cb18911112d5e13f3c14f87ddbdbaced3538a
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-352
Needs a lot more work, but the directory tree and layout are according to our
needs in order to have this published as a CVP document. Will push patches
to restructure the repo according to the new requirements later.
Change-Id: I9d4316407c94bc3096d322f6206eccf236d349da
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
|
|
JIRA: DOVETAIL-164
usage:
dovetail run --testsuite <testsuite> --offline
1,if --offline given, can run offline by using local docker images
(if there is no images, will get an error in log, images can be
pulled from dockerhub or downloaded from artifacts.opnfv.org )
2,if --offline not given, will run online, which means
pull docker image from dockerhub
Change-Id: I87ab35c82e59fb13a7a88c38d233e89285efc4d3
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: DOVETAIL-373
1. add an option --report/-r to choose the report type.
2. "-r file" will store results with files.
3. "-r http://192.168.135.2:8000/api/v1" will push results into DB.
The DB url can replaced by user's local DB or official DB.
4. Default report type is defined as "report_dest" in dovetail_config.yml.
5. Yardstick can't push results of one testcase into DB now, so both "-r file"
and "-r http://..." will be regarded as "file".
Change-Id: Ieaa017ab59c5b15235bf306ef57cdc56beb22af8
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-374
User may want to dry run the test, and does not intend to upload test results to OPNFV DB.
So dovetail provide a way to launch a local TESTAPI DB instance, then user can upload test results to this local db, and browse test results locally.
Change-Id: I17b6153c9a308bd3cd59a2450c415200496cb855
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
|
|
JIRA: DOVETAIL-376
Change-Id: I91e0efff01d445e3f74718d730c8ff48c6413b26
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
yardstick.log"
|
|
JIRA: DOVETAIL-371
1. remove all the envs in functest_config.yml and yardstick_config.yml
2. remove all relative cmds in cmd_config.yml
3. remove all the envs when running a new functest/yardstick container
4. add some variables just for functest pushing results to db
5. get the external network name for yardstick
6. make --debug/-d just control the log level shown on the screen
7. set requests=2.10.0 since python-openstackclient needs
'requests!=2.12.2,!=2.13.0,>=2.10.0'
Change-Id: I9f941e09d067fc95a14c3c9be1374f41df4a9a16
Signed-off-by: xudan <xudan16@huawei.com>
|
|
yardstick.log
1. For Fuel, yardstick prepare_env.sh will fail when
pwd != /home/opnfv/repos/yardstick
2. Modify yardstick_config.yml in Dovetail to change pwd to that path.
3. Use log file /tmp/yardstick/yardstick.log created by cmd "yardstick task start"
rather than redirecting the console output to yardstick.log.
JIRA: DOVETAIL-363
Change-Id: Ie0475cfabc66e342be868f0924a5fccd6fa74b58
Signed-off-by: xudan <xudan16@huawei.com>
|