Age | Commit message (Collapse) | Author | Files | Lines |
|
1. Many HA test cases try to kill special processes with their name.
2. Then they check corresponding openstack services and
the status of processes' recovery.
3. For different SUTs, the names of the processes and the hosts' names
may be different.
4. So there is a requirement for Dovetail tool to provide one way to
allow users to specify all the infos about process name and its host name.
5. All process infos can be added into file DOVETAIL_HOME/user_config/pod.yaml
6. The infos include 'attack_process' and 'attack_host' for each HA test cases.
7. If not given in this file, will use Yardtsick default values.
JIRA: DOVETAIL-627
Change-Id: I83cee991f72a8685080ed562597c70d73002623a
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Specifying separate valid docker tags for functest and yardstick for
input validation.
DOVETAIL-592
Change-Id: I3365cdbd9ab521bf1368750c3f34b6863b3f7ec8
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The function 'check_tc_result' may return 'None'.
In this case it will throw an exception because of
https://github.com/opnfv/dovetail/blob/master/dovetail/run.py#L65
The exception is like
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 339, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in _call_
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 304, in main
duration = run_test(testsuite_yaml, testarea, logger, kwargs)
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 65, in run_test
if stop_on_fail['criteria'] == "FAIL" and kwargs['stop']:
TypeError: 'NoneType' object has no attribute '_getitem_'
JIRA: DOVETAIL-622
Change-Id: I9e612b380c3dfb148692074a3bf8eb31875213ac
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I89d15e18c588c27ab41596119d5288cba6e4e02d
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
1. mv conf/ userconfig/ compliance/ patch to etc/
2. after installation etc/ will be put under /etc/dovetail
3. leveraging /etc/dovetail in coding
Change-Id: Icf9cb00ff7fc7cf35755f60d3e258b23e2c70839
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
Adding the SDNVPN Tempest API test cases of the SDNVPN project to the
proposed_tests suite to allow for running these tests in the CI pipeline for
validating their suitability for inclusion in the OVP Euphrates release.
JIRA: DOVETAIL-608
Change-Id: Ic12db1584a2540a14979094c65fc128c6020ecea
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
If the hosts.yaml file is like,
—
hosts_info:
192.168.141.101:
it will raise an exception,
File "/usr/local/lib/python2.7/dist-packages/dovetail/utils/dovetail_utils.py",
line 227, in add_hosts_info
names=hostnames)
File "/usr/local/lib/python2.7/dist-packages/python_hosts/hosts.py",
line 54, in _init_
raise Exception('Address and Name(s) must be specified.')
Exception: Address and Name(s) must be specified.
If the hosts.yaml file is empty, it will crash with,
File "/usr/local/lib/python2.7/dist-packages/dovetail/container.py",
line 174, in create
if hosts_yaml['hosts_info']:
TypeError: 'NoneType' object has no attribute '_getitem_'
JIRA: DOVETAIL-621
Change-Id: I547928e514b3294c048379459c07df89879fbb03
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Bottlenecks stress test needs some adaptions about Bottlenecks
Euphrates release.
JIRA: DOVETAIL-587
Change-Id: Ia447143c8f08924ac1b190ee0173f39e462c5dfa
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Add cloudify_ims test case into proposed_tests test suite.
Then this test case can be run by Dovetail on CI.
JIRA: DOVETAIL-603
Change-Id: I5ab832002388f0916e04beef9d630de2cb37df59
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Add 3 new HA test cases into proposed_tests test suite and running them on CI.
For the current 8 HA test cases and 3 other HA test cases plans to add,
do some adaptions for the Yardstick Euphrates release.
1. Don't need the bugfix patch anymore.
2. The data format has changed, and has done some adaptions.
3. For Yardstick Euphrates, if setting OS_INSECURE=True, it needs to set
OS_CACERT=False
4. Test case tc011 needs a image and flavor to create the stack.
JIRA: DOVETAIL-614
JIRA: DOVETAIL-586
Change-Id: I9da8c8316246b9edee0d81ad18422ae6018becaf
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently it won't pass the cacert file to other containers if the
OS_AUTH_URL is http not https.
2. However, even though the OS_AUTH_URL is http, it still needs cacert file
somewhere for Functest and Rally.
3. This problem is found on Fuel Euphrates.
4. Add a function to check whether the cacert file exists.
5. If exists, pass it to testing project containers.
JIRA: DOVETAIL-616
Change-Id: Ied7bcc72e8f1c738bbce32c18096ca13641d3cd7
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Add snaps_smoke into proposed_tests test suite.
Then this test case can be run by Dovetail on CI.
JIRA: DOVETAIL-596
Change-Id: I8fc4bb656bcac551ae4b693dbc4d817b665c6111
Signed-off-by: xudan <xudan16@huawei.com>
|
|
- Improve test execution control by adding stop on failure flag
- Modify test check function to return value regardless of whether
report or local file option is employed
- Modify run test loop to check for failure after every test case
if stop flag set
Change-Id: I27a4f3aaaa8b8cb8c52eb3f092f52048980abb44
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- EXTERNAL_NETWORK env variable can be explicitly set
- If set, the value will be used in Yardstack configuration
- Otherwise, previous method of using output from
'openstack network list' will still be employed
Change-Id: I9d8b4a69ff3f5093c034823cc95e92e151bfade6
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- Try new license header to deal with anteater failure
- Ability to specify 'default' as argument to --report, which
fetches host IP automatically and forms URL with 8000 as testapi
port
- This should not break the previous behavior of report option,
if default option not specified
Change-Id: I1e05bb52cf4fc211c59796d4e763e2e04177b406
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
Add all tempest smoke test cases into debug test suite.
Then these test cases can be run by Dovetail on CI.
The results of CI will help to choose sub test cases from tempest smoke.
Here use tempest_custom to run all tempest smoke test cases because it can
support to run a custom defined sub test cases list.
for Functest tempest_smoke_serial, it always runs whole tempest smoke
test cases and can't be a subset.
JIRA: DOVETAIL-595
Change-Id: Id6358691d2bdef149f16775ce84dcfbf12f5e568
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Use opnfv/functest-restapi:euphrates.
Adapt Functest E framework for all Functest test cases.
1. All path of functest and sdnvpn has been changed.
2. Sdnvpn can clean resources by itself with E release, no need to clean again.
3. Add a file testcases.yaml to overwrite the file in functest container,
then it can support to run tempest_custom i.e. ipv6 and tempest in OVP.
JIRA: DOVETAIL-584
Change-Id: I3ba9aa8e80becaa99cc89c8db4b39acbb89dfda4
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change default container tag to be:
1. Functest ovp.1.0.0
2. testapi ovp.1.0.0
Add test suite ovp.1.0.0 and set it to be the default test suite.
Comment out bottlenecks CLI for initial release.
JIRA: DOVETAIL-580
Change-Id: I956e49e43fc7ba0176c14079b7228089db5dc108
Signed-off-by: xudan <xudan16@huawei.com>
|
|
When Tempest test cases failed at the SetupClass, the record is the class name
of the test cases rather than the name of the test cases.
The local summary will say they are passed wrongly.
JIRA: DOVETAIL-575
Change-Id: Iec879378770017eddd37e9a99346d980355bb6e2
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I9c585f132c10ef726cd859e4abe3c9a874cbe0bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Some tempest test cases have alias test cases because of the different
versions of tempest.
2. However, Dovetail fixes the version of tempest to be 14.0.0 in its first
release. So the number of test cases that can be executed in osinterop
is always 205. And for different SUTs, the 205 test cases are all the same.
3. In this situation, there is no need to list the alias test cases.
4. Just list all the 205 test cases which will not confuse users when they
reading dovetail docs.
JIRA: DOVETAIL-566
Change-Id: I5e39dd82314833c8c0a6c1732c990f671ae64420
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, dovetail.tempest.tc004 has 13 sub test cases defined in its
config file and specification document of the test case. All these
sub test cases are picked from tempest based on it's version 15.0.0.
2. For Dovetail 1st release, it fixes the version of Functest to be
opnfv/dovetail:cvp.0.5.0, and Functest integretes tempest 14.0.0
in this version and it's E release.
3. There is one sub test case in dovetail.tempest.tc004 that is new in
tempest 15.0.0, so it can not be recognized and executed by Functest D
and E release. It is 'test_server_connectivity_cold_migration_revert'.
4. Remove this sub test case from dovetail.tempest.tc004. This means remove it
from the test case config file and specification document.
JIRA: DOVETAIL-561
Change-Id: Ia927af040794c7027a741a282ed75fe401f99ff1
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The name of the compressed result file included a timestamp according
to the pattern "day_hour_minute". This is a rather uncommon timestamp
and can cause confusion. This patch changes the timestamp to a more
common "YearMonthDay-HourMinute" format.
Please note: Please check if this change has an impact on the web
portal.
Change-Id: Ife41050bc1d5ca6942706b756d03a1d0d251ae41
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
changes:
1. add usage example in hosts.yaml.sample
2. fix /etc/hosts from wrong format {'hostname': 'ip'} to
the right 'ip hostnames'
3. fix --add-hosts from wrong usage --add-host {'hostname': 'ip'} to
the right --add-host='hostnames':ip
4. support per ip mapping mutiple hostnames, which is also
in line with /etc/hosts schema
JIRA: DOVETAIL-556
Change-Id: I92e8cb0fd3476b61f5c73a69e34c4a66aef51d6e
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
The script launch_db.sh will print "Already Exists" logs when the DB has been
initialized by the data stored in JumpHost.
Ignore this kind of logs because it won't affect the launch of the DB but
it will confuse users.
JIRA: DOVETAIL-557
Change-Id: I23616b10bbadc49888731e943d0a4ee4425045f5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-550
Change-Id: I871ec1fdf8d33efe3ed00602d0184c7b35896c17
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. The scripts for building local DB and testapi service just work on Ubuntu.
2. For Centos, they fails to do that.
3. This is mainly because of that Centos can't use the IP of the JumpHost
to connect the DB and testapi. It needs to use the internal IPs of the
DB container and the testapi container.
4. Change the IPs in the scripts to be the containers' internal IPs.
JIRA: DOVETAIL-548
Change-Id: I71aed82cb620b58bcd43ace61497db65c5bd7ecc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Get the admin endpoints of all service.
JIRA: DOVETAIL-546
Change-Id: Ie0ac3bc5b4eb0885046e207b73217994a4f0e358
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Hardware info:
Use ansible to get all hardware info from all nodes.
JIRA: DOVETAIL-540
Change-Id: I6a2dde1b2ebae2af8f702008c23110ebe63cc06f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. If there throw an exception during the test of one sdnvpn test case,
it will not do the clean.
2. Then there are many resources left, such as network, subnet, server...
3. This is a bug of sdnvpn see JIRA: SDNVPN-184
4. The patch for fixing this bug https://gerrit.opnfv.org/gerrit/#/c/44777/
5. Currently, use functest module "functest openstack clean" to clean the SUT.
JIRA: DOVETAIL-524
Change-Id: I629fc6fdc9e4c95960f6122358ff219a467d441b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. The SDNVPN DB initialization information lacks two test cases,
functest_testcase_4 and functest_testcase_8.
2. So the results of these 2 test cases cannot be pushed into DB.
3. Add these 2 test cases info to DB.
JIRA: DOVETAIL-535
Change-Id: I1c6d049b942806924f15dd106452fd9b4d554454
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Now there is a testsuite proposed_tests for beta tests which may
cause some confusions.
2. Copy all test cases to testsuite cvp.0.8.0 for beta testers to use.
3. Update userguide with testsuite cvp.0.8.0
JIRA: DOVETAIL-528
Change-Id: I35bc7dee6e07a9125a4c1991b0049f6e2d705cc5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Move SDNVPN test cases after tempest test cases, then the resources left will
not effect tempest test cases.
JIRA: DOVETAIL-524
Change-Id: I815ae7f3a3be0225be1e9695f13d4552946bffb5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
SDNVPN test cases failed due to wrong module name.
The module name needs to be changed into "sdnvpn.test.functest.testcase_1"
Refer to patch https://gerrit.opnfv.org/gerrit/#/c/41385/
Currently Dovetail uses functest:cvp.0.5.0 which uses module name "testcase_1".
JIRA: DOVETAIL-510
Change-Id: I3232760d3af34125c7da71a5d2c62ca32693c82b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
|
|
|
|
1. Dovetail generate a tar file for uploading results to CVP web portal.
2. If users don't run with option --report, it shouldn't generate the tar file.
Because users can not upload results to CVP if they don't run with --report.
3. If it generates the tar file when user run dovetail without --report,
the tar file is invalid because it lacks of results.json.
JIRA: DOVETAIL-511
Change-Id: I4d21f21f0b278962564d810369df03100b38f003
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
1. When test dovetail, users can build local DB and testapi to store and
report the results.
2. The script for building local DB and testapi needs some updates.
JIRA: DOVETAIL-507
Change-Id: I8bc96bfea27f424468cafaa2e05894065ef44ad9
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The directory tree of results tar file is wrong, so web portal cannot parse it.
Reset the tar file.
JIRA: DOVETAIL-508
Change-Id: I8475323294e84ca51242b700570926d5c0177390
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, the log level of yardstick is DEBUG and cannot be modified.
2. Make it can be changed from CLI.
3. When use dovetail run --debug, can see all Yardstick logs (including DEBUG)
in dovetail.log.
4. When run without debug option, cannot see Yardstick debug logs in
dovetail.logs.
5. yardstick.log always doesn't include debug logs.
JIRA: DOVETAIL-506
Change-Id: Iaafb61232a22e16fff2fa4605bf7e262b022085e
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, dovetail can run a test suite or a test area in that test suite.
2. The test areas in one test suite are not divided into optional and mandatory.
3. Split them and support to run just mandatory or optional.
4. Support to run multiple test areas.
JIRA: DOVETAIL-505
Change-Id: I42cd7b4e11c3e3674c806e9bc999b782bf5c85c6
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Rename the "defcore" test area to "osinterop".
The DefCore working group has been
renamed OpenStack InterOp Working Group some time ago.
JIRA: DOVETAIL-499
Change-Id: I144af767632c8fb83859e5e04b6427bb1a538348
Signed-off-by: xudan <xudan16@huawei.com>
|
|
docker image used
opnfv/yardstick: danube.3.2
opnfv/functest: cvp.0.5.0
opnfv/bottlenecks: cvp.0.4.0
opnfv/testapi: cvp.0.6.0
opnfv/dovetail:web.cvp.0.6.0
opnfv/dovetail:api.cvp.0.6.0
opnfv/dovetail: cvp.0.6.0
main changes
Merge all test specification docs about test area tempest and defcore
Merge addendum doc and workflow doc
Remake aarch64 patch for Dovetail container
Remove some tempest test cases that are not too common
Add additional Tempest API test cases into test case tempest.tc005
persist db file in host for mongo container
add nginx to host static files
add a sample tempest_conf.yaml file into userconfig
Add a patch to set ha SLA to be 30 seconds
Modify logs
Bugfix: the config files in useeconfig will cover user's config
Bugfix: functest only keeps the last test case's log
Bugfix: pre_condition cmds is just executed once
For more information, refer to https://wiki.opnfv.org/display/dovetail/Running+history+for+the+dovetail+tool
JIRA: DOVETAIL-497
Change-Id: I74626bbf65ce0121cc7b53394c85694e15df8484
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
JIRA: DOVETAIL-498
Change-Id: Ie98fc89cc840b557aa4186c6c2bee2d610432af0
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. According to ha test specification doc, the SLA of process recover time
is 30 seconds.
2. However, the SLA in Yardstick test cases are all 20 seconds.
3. Add a patch to change the SLA in yardstick test cases.
JIRA: DOVETAIL-496
Change-Id: Ib8f229fdfbb3289bf70d6ee900132b3db3a21b17
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|