Age | Commit message (Collapse) | Author | Files | Lines |
|
The CLI of dovetail tool needs some improvement.
1. Add an option '--testcase' to support specify test case.
2. Add an option '--no-clean' to keep functest/yardstick/bottlenecks containers
for debugging after running the test cases.
3. Improve the local report section.
a. Remove some info that is not necessary
b. Make the info more clearly
JIRA: DOVETAIL-588
Change-Id: Ic769a6e2788b407c3df08ccd593a0ecdaa35e5ea
Signed-off-by: xudan <xudan16@huawei.com>
|
|
There are some special Docker setting items when creating
Bottlenecks Docker Cntainers.
Move the special settings to config files.
Disable the function to push results to DB.
JIRA: DOVETAIL-645
Change-Id: Icc033222edf74c0fe4853dede2794f4226fd4252
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Remove local DB and can just report without launch local DB.
Need to do a little modification with the web portal code.
JIRA: DOVETAIL-593
Change-Id: I595db39cce4156596ee0522d4b7e642a23d78ab9
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Adding the new Patrole test case of Functest to Dovetail and to the
proposed_tests test suite.
JIRA: DOVETAIL-649
Change-Id: I2c3f583cdbd3af75cec020b0fe97f287d23498a5
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The original refstack results file is refstack/refstack.log
For Fraser, it has been changed into refstack/tempest.log
Change-Id: I7d40abe0c5a366b5fbb1912ec58d15109f22c0d0
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Use Functest Fraser docker images whose tag is fraser
2. Use Yardstick stable docker images
3. Functest fraser has moved testcases.yaml file from functest/ci to xtesting/ci
4. Functest fraser has removed 'functest env prepare' command
5. Functest uses rally to execute osinterop test cases
instead of refstack_client
Change-Id: I81ebefe6ea959ca77c2c87f1d75ef56ac18c165f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
This patch adds the ability to Dovetail to disable strict API
response validation in Tempest-based test cases run by Functest.
Corresponding updates of the web portal and backporting of this patch to
OVP 1.0.0 (Danube) will follow.
JIRA: DOVETAIL-633
Change-Id: Iace99ea1b6224ea907a2c3af8676e9285e6ad3ee
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
There are some special Docker setting items when creating
Yardstick Docker Containers.
1. Remove the extra settings which are required by Yardstick Danube.
2. Remove the settings for pushing results to DB because Dovetail
plan to disable this function.
3. Move the other special Functest settings to config files.
JIRA: DOVETAIL-640
Change-Id: Ic2d90393e227d7886b14d805eaa307b7aef56e4a
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently each type of test cases will use one docker image.
2. For example,
Functest test cases use opnfv/functest-restapi:euphrates
Yardstick uses opnfv/yardstick:opnfv-5.1.0
3. It needs to support to specify the docker image for each test case.
4. For example,
vping test cases use opnfv/functest-smoke:euphrates
sdnvpn uses opnfv/functest-features:euphrates
5. The benefit of this is that these docker images are well verified
by Functest CI and from Functest plan, they may not support
functest-restapi docker image in its future release.
JIRA: DOVETAIL-638
Change-Id: I8a30eea2233aeff809af2e241b6c939194397832
Signed-off-by: xudan <xudan16@huawei.com>
|
|
There are some special Docker setting item when creating Functest Containers.
1. Remove the extra settings which are required by Functest Danube.
2. Remove the settings for pushing results to DB because
Dovetail plans to disable this function.
3. Move the other special Functest settings to config files.
JIRA: DOVETAIL-639
Change-Id: Ibc4090b5c0aafdf9ed64e65a9a92e34a12a16df5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I6a1fa5f27df2f5127eb00156b3135dc79caf5e77
Issue-ID: DOVETAIL-629
Signed-off-by: Moshe <moshehoa@amdocs.com>
|
|
1. The original testcases.yaml file of Functest restricts sdnvpn test cases
to run on Fuel, Apex or Netvirt on bgpvpn scenario.
https://github.com/opnfv/functest/blob/master/functest/ci/testcases.yaml#L278
2. Dovetail has a testcases.yaml file to allow sdnvpn test cases run on
any commercial SUTs.
https://github.com/opnfv/dovetail/blob/master/etc/userconfig/testcases.yaml#L113
Change-Id: I0d5ed5b22e55dc3dc0fe584dc57b28239334b7de
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: Ie470d03bd168c909fd02c316523049c67a6bb095
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
The directory of userconfig has changed from
/home/opnfv/dovetail/dovetail/userconfig to /etc/dovetail/userconfig.
Change-Id: Id3e8b1f3e9299e9263a4adf3456b6733fdb62e11
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Many HA test cases try to kill special processes with their name.
2. Then they check corresponding openstack services and
the status of processes' recovery.
3. For different SUTs, the names of the processes and the hosts' names
may be different.
4. So there is a requirement for Dovetail tool to provide one way to
allow users to specify all the infos about process name and its host name.
5. All process infos can be added into file DOVETAIL_HOME/user_config/pod.yaml
6. The infos include 'attack_process' and 'attack_host' for each HA test cases.
7. If not given in this file, will use Yardtsick default values.
JIRA: DOVETAIL-627
Change-Id: I83cee991f72a8685080ed562597c70d73002623a
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Specifying separate valid docker tags for functest and yardstick for
input validation.
DOVETAIL-592
Change-Id: I3365cdbd9ab521bf1368750c3f34b6863b3f7ec8
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The function 'check_tc_result' may return 'None'.
In this case it will throw an exception because of
https://github.com/opnfv/dovetail/blob/master/dovetail/run.py#L65
The exception is like
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 339, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in _call_
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 304, in main
duration = run_test(testsuite_yaml, testarea, logger, kwargs)
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 65, in run_test
if stop_on_fail['criteria'] == "FAIL" and kwargs['stop']:
TypeError: 'NoneType' object has no attribute '_getitem_'
JIRA: DOVETAIL-622
Change-Id: I9e612b380c3dfb148692074a3bf8eb31875213ac
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I89d15e18c588c27ab41596119d5288cba6e4e02d
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
1. mv conf/ userconfig/ compliance/ patch to etc/
2. after installation etc/ will be put under /etc/dovetail
3. leveraging /etc/dovetail in coding
Change-Id: Icf9cb00ff7fc7cf35755f60d3e258b23e2c70839
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
Adding the SDNVPN Tempest API test cases of the SDNVPN project to the
proposed_tests suite to allow for running these tests in the CI pipeline for
validating their suitability for inclusion in the OVP Euphrates release.
JIRA: DOVETAIL-608
Change-Id: Ic12db1584a2540a14979094c65fc128c6020ecea
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
If the hosts.yaml file is like,
—
hosts_info:
192.168.141.101:
it will raise an exception,
File "/usr/local/lib/python2.7/dist-packages/dovetail/utils/dovetail_utils.py",
line 227, in add_hosts_info
names=hostnames)
File "/usr/local/lib/python2.7/dist-packages/python_hosts/hosts.py",
line 54, in _init_
raise Exception('Address and Name(s) must be specified.')
Exception: Address and Name(s) must be specified.
If the hosts.yaml file is empty, it will crash with,
File "/usr/local/lib/python2.7/dist-packages/dovetail/container.py",
line 174, in create
if hosts_yaml['hosts_info']:
TypeError: 'NoneType' object has no attribute '_getitem_'
JIRA: DOVETAIL-621
Change-Id: I547928e514b3294c048379459c07df89879fbb03
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Bottlenecks stress test needs some adaptions about Bottlenecks
Euphrates release.
JIRA: DOVETAIL-587
Change-Id: Ia447143c8f08924ac1b190ee0173f39e462c5dfa
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Add cloudify_ims test case into proposed_tests test suite.
Then this test case can be run by Dovetail on CI.
JIRA: DOVETAIL-603
Change-Id: I5ab832002388f0916e04beef9d630de2cb37df59
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Add 3 new HA test cases into proposed_tests test suite and running them on CI.
For the current 8 HA test cases and 3 other HA test cases plans to add,
do some adaptions for the Yardstick Euphrates release.
1. Don't need the bugfix patch anymore.
2. The data format has changed, and has done some adaptions.
3. For Yardstick Euphrates, if setting OS_INSECURE=True, it needs to set
OS_CACERT=False
4. Test case tc011 needs a image and flavor to create the stack.
JIRA: DOVETAIL-614
JIRA: DOVETAIL-586
Change-Id: I9da8c8316246b9edee0d81ad18422ae6018becaf
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently it won't pass the cacert file to other containers if the
OS_AUTH_URL is http not https.
2. However, even though the OS_AUTH_URL is http, it still needs cacert file
somewhere for Functest and Rally.
3. This problem is found on Fuel Euphrates.
4. Add a function to check whether the cacert file exists.
5. If exists, pass it to testing project containers.
JIRA: DOVETAIL-616
Change-Id: Ied7bcc72e8f1c738bbce32c18096ca13641d3cd7
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Add snaps_smoke into proposed_tests test suite.
Then this test case can be run by Dovetail on CI.
JIRA: DOVETAIL-596
Change-Id: I8fc4bb656bcac551ae4b693dbc4d817b665c6111
Signed-off-by: xudan <xudan16@huawei.com>
|
|
- Improve test execution control by adding stop on failure flag
- Modify test check function to return value regardless of whether
report or local file option is employed
- Modify run test loop to check for failure after every test case
if stop flag set
Change-Id: I27a4f3aaaa8b8cb8c52eb3f092f52048980abb44
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- EXTERNAL_NETWORK env variable can be explicitly set
- If set, the value will be used in Yardstack configuration
- Otherwise, previous method of using output from
'openstack network list' will still be employed
Change-Id: I9d8b4a69ff3f5093c034823cc95e92e151bfade6
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- Try new license header to deal with anteater failure
- Ability to specify 'default' as argument to --report, which
fetches host IP automatically and forms URL with 8000 as testapi
port
- This should not break the previous behavior of report option,
if default option not specified
Change-Id: I1e05bb52cf4fc211c59796d4e763e2e04177b406
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
Add all tempest smoke test cases into debug test suite.
Then these test cases can be run by Dovetail on CI.
The results of CI will help to choose sub test cases from tempest smoke.
Here use tempest_custom to run all tempest smoke test cases because it can
support to run a custom defined sub test cases list.
for Functest tempest_smoke_serial, it always runs whole tempest smoke
test cases and can't be a subset.
JIRA: DOVETAIL-595
Change-Id: Id6358691d2bdef149f16775ce84dcfbf12f5e568
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Use opnfv/functest-restapi:euphrates.
Adapt Functest E framework for all Functest test cases.
1. All path of functest and sdnvpn has been changed.
2. Sdnvpn can clean resources by itself with E release, no need to clean again.
3. Add a file testcases.yaml to overwrite the file in functest container,
then it can support to run tempest_custom i.e. ipv6 and tempest in OVP.
JIRA: DOVETAIL-584
Change-Id: I3ba9aa8e80becaa99cc89c8db4b39acbb89dfda4
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change default container tag to be:
1. Functest ovp.1.0.0
2. testapi ovp.1.0.0
Add test suite ovp.1.0.0 and set it to be the default test suite.
Comment out bottlenecks CLI for initial release.
JIRA: DOVETAIL-580
Change-Id: I956e49e43fc7ba0176c14079b7228089db5dc108
Signed-off-by: xudan <xudan16@huawei.com>
|
|
When Tempest test cases failed at the SetupClass, the record is the class name
of the test cases rather than the name of the test cases.
The local summary will say they are passed wrongly.
JIRA: DOVETAIL-575
Change-Id: Iec879378770017eddd37e9a99346d980355bb6e2
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I9c585f132c10ef726cd859e4abe3c9a874cbe0bc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Some tempest test cases have alias test cases because of the different
versions of tempest.
2. However, Dovetail fixes the version of tempest to be 14.0.0 in its first
release. So the number of test cases that can be executed in osinterop
is always 205. And for different SUTs, the 205 test cases are all the same.
3. In this situation, there is no need to list the alias test cases.
4. Just list all the 205 test cases which will not confuse users when they
reading dovetail docs.
JIRA: DOVETAIL-566
Change-Id: I5e39dd82314833c8c0a6c1732c990f671ae64420
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently, dovetail.tempest.tc004 has 13 sub test cases defined in its
config file and specification document of the test case. All these
sub test cases are picked from tempest based on it's version 15.0.0.
2. For Dovetail 1st release, it fixes the version of Functest to be
opnfv/dovetail:cvp.0.5.0, and Functest integretes tempest 14.0.0
in this version and it's E release.
3. There is one sub test case in dovetail.tempest.tc004 that is new in
tempest 15.0.0, so it can not be recognized and executed by Functest D
and E release. It is 'test_server_connectivity_cold_migration_revert'.
4. Remove this sub test case from dovetail.tempest.tc004. This means remove it
from the test case config file and specification document.
JIRA: DOVETAIL-561
Change-Id: Ia927af040794c7027a741a282ed75fe401f99ff1
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The name of the compressed result file included a timestamp according
to the pattern "day_hour_minute". This is a rather uncommon timestamp
and can cause confusion. This patch changes the timestamp to a more
common "YearMonthDay-HourMinute" format.
Please note: Please check if this change has an impact on the web
portal.
Change-Id: Ife41050bc1d5ca6942706b756d03a1d0d251ae41
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
changes:
1. add usage example in hosts.yaml.sample
2. fix /etc/hosts from wrong format {'hostname': 'ip'} to
the right 'ip hostnames'
3. fix --add-hosts from wrong usage --add-host {'hostname': 'ip'} to
the right --add-host='hostnames':ip
4. support per ip mapping mutiple hostnames, which is also
in line with /etc/hosts schema
JIRA: DOVETAIL-556
Change-Id: I92e8cb0fd3476b61f5c73a69e34c4a66aef51d6e
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
The script launch_db.sh will print "Already Exists" logs when the DB has been
initialized by the data stored in JumpHost.
Ignore this kind of logs because it won't affect the launch of the DB but
it will confuse users.
JIRA: DOVETAIL-557
Change-Id: I23616b10bbadc49888731e943d0a4ee4425045f5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
JIRA: DOVETAIL-550
Change-Id: I871ec1fdf8d33efe3ed00602d0184c7b35896c17
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. The scripts for building local DB and testapi service just work on Ubuntu.
2. For Centos, they fails to do that.
3. This is mainly because of that Centos can't use the IP of the JumpHost
to connect the DB and testapi. It needs to use the internal IPs of the
DB container and the testapi container.
4. Change the IPs in the scripts to be the containers' internal IPs.
JIRA: DOVETAIL-548
Change-Id: I71aed82cb620b58bcd43ace61497db65c5bd7ecc
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Get the admin endpoints of all service.
JIRA: DOVETAIL-546
Change-Id: Ie0ac3bc5b4eb0885046e207b73217994a4f0e358
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Hardware info:
Use ansible to get all hardware info from all nodes.
JIRA: DOVETAIL-540
Change-Id: I6a2dde1b2ebae2af8f702008c23110ebe63cc06f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. If there throw an exception during the test of one sdnvpn test case,
it will not do the clean.
2. Then there are many resources left, such as network, subnet, server...
3. This is a bug of sdnvpn see JIRA: SDNVPN-184
4. The patch for fixing this bug https://gerrit.opnfv.org/gerrit/#/c/44777/
5. Currently, use functest module "functest openstack clean" to clean the SUT.
JIRA: DOVETAIL-524
Change-Id: I629fc6fdc9e4c95960f6122358ff219a467d441b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. The SDNVPN DB initialization information lacks two test cases,
functest_testcase_4 and functest_testcase_8.
2. So the results of these 2 test cases cannot be pushed into DB.
3. Add these 2 test cases info to DB.
JIRA: DOVETAIL-535
Change-Id: I1c6d049b942806924f15dd106452fd9b4d554454
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Now there is a testsuite proposed_tests for beta tests which may
cause some confusions.
2. Copy all test cases to testsuite cvp.0.8.0 for beta testers to use.
3. Update userguide with testsuite cvp.0.8.0
JIRA: DOVETAIL-528
Change-Id: I35bc7dee6e07a9125a4c1991b0049f6e2d705cc5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Move SDNVPN test cases after tempest test cases, then the resources left will
not effect tempest test cases.
JIRA: DOVETAIL-524
Change-Id: I815ae7f3a3be0225be1e9695f13d4552946bffb5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|
|
SDNVPN test cases failed due to wrong module name.
The module name needs to be changed into "sdnvpn.test.functest.testcase_1"
Refer to patch https://gerrit.opnfv.org/gerrit/#/c/41385/
Currently Dovetail uses functest:cvp.0.5.0 which uses module name "testcase_1".
JIRA: DOVETAIL-510
Change-Id: I3232760d3af34125c7da71a5d2c62ca32693c82b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
|