summaryrefslogtreecommitdiffstats
path: root/dovetail
AgeCommit message (Collapse)AuthorFilesLines
2018-05-01Integrating Functest Patrole test caseGeorg Kunz1-0/+2
Adding the new Patrole test case of Functest to Dovetail and to the proposed_tests test suite. JIRA: DOVETAIL-649 Change-Id: I2c3f583cdbd3af75cec020b0fe97f287d23498a5 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-04-24The refstack file name has been changedxudan1-1/+1
The original refstack results file is refstack/refstack.log For Fraser, it has been changed into refstack/tempest.log Change-Id: I7d40abe0c5a366b5fbb1912ec58d15109f22c0d0 Signed-off-by: xudan <xudan16@huawei.com>
2018-04-17Switch to OPNFV Fraser releasexudan1-3/+3
1. Use Functest Fraser docker images whose tag is fraser 2. Use Yardstick stable docker images 3. Functest fraser has moved testcases.yaml file from functest/ci to xtesting/ci 4. Functest fraser has removed 'functest env prepare' command 5. Functest uses rally to execute osinterop test cases instead of refstack_client Change-Id: I81ebefe6ea959ca77c2c87f1d75ef56ac18c165f Signed-off-by: xudan <xudan16@huawei.com>
2018-04-17Implementation of API validaton exemption in dovetailGeorg Kunz3-3/+28
This patch adds the ability to Dovetail to disable strict API response validation in Tempest-based test cases run by Functest. Corresponding updates of the web portal and backporting of this patch to OVP 1.0.0 (Danube) will follow. JIRA: DOVETAIL-633 Change-Id: Iace99ea1b6224ea907a2c3af8676e9285e6ad3ee Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-04-10Decoupling Yardstick Docker config items from source codexudan3-61/+21
There are some special Docker setting items when creating Yardstick Docker Containers. 1. Remove the extra settings which are required by Yardstick Danube. 2. Remove the settings for pushing results to DB because Dovetail plan to disable this function. 3. Move the other special Functest settings to config files. JIRA: DOVETAIL-640 Change-Id: Ic2d90393e227d7886b14d805eaa307b7aef56e4a Signed-off-by: xudan <xudan16@huawei.com>
2018-04-10Support to specify docker image for each test casexudan3-28/+41
1. Currently each type of test cases will use one docker image. 2. For example, Functest test cases use opnfv/functest-restapi:euphrates Yardstick uses opnfv/yardstick:opnfv-5.1.0 3. It needs to support to specify the docker image for each test case. 4. For example, vping test cases use opnfv/functest-smoke:euphrates sdnvpn uses opnfv/functest-features:euphrates 5. The benefit of this is that these docker images are well verified by Functest CI and from Functest plan, they may not support functest-restapi docker image in its future release. JIRA: DOVETAIL-638 Change-Id: I8a30eea2233aeff809af2e241b6c939194397832 Signed-off-by: xudan <xudan16@huawei.com>
2018-04-04Decoupling Functest Docker config items from source codexudan2-30/+20
There are some special Docker setting item when creating Functest Containers. 1. Remove the extra settings which are required by Functest Danube. 2. Remove the settings for pushing results to DB because Dovetail plans to disable this function. 3. Move the other special Functest settings to config files. JIRA: DOVETAIL-639 Change-Id: Ibc4090b5c0aafdf9ed64e65a9a92e34a12a16df5 Signed-off-by: xudan <xudan16@huawei.com>
2018-03-29Integrate dovetail with ONAPMoshe5-3/+129
Change-Id: I6a1fa5f27df2f5127eb00156b3135dc79caf5e77 Issue-ID: DOVETAIL-629 Signed-off-by: Moshe <moshehoa@amdocs.com>
2018-03-15Disable restricts of sdnvpn test casesxudan1-7/+0
1. The original testcases.yaml file of Functest restricts sdnvpn test cases to run on Fuel, Apex or Netvirt on bgpvpn scenario. https://github.com/opnfv/functest/blob/master/functest/ci/testcases.yaml#L278 2. Dovetail has a testcases.yaml file to allow sdnvpn test cases run on any commercial SUTs. https://github.com/opnfv/dovetail/blob/master/etc/userconfig/testcases.yaml#L113 Change-Id: I0d5ed5b22e55dc3dc0fe584dc57b28239334b7de Signed-off-by: xudan <xudan16@huawei.com>
2018-03-15substitute image copy with docker volumeSerenaFeng3-21/+12
Change-Id: Ie470d03bd168c909fd02c316523049c67a6bb095 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2018-03-14Bugfix: Failed to get the hardware infoxudan1-3/+4
The directory of userconfig has changed from /home/opnfv/dovetail/dovetail/userconfig to /etc/dovetail/userconfig. Change-Id: Id3e8b1f3e9299e9263a4adf3456b6733fdb62e11 Signed-off-by: xudan <xudan16@huawei.com>
2018-03-14Add process info to pod.yaml to specify the info of a special processxudan3-7/+90
1. Many HA test cases try to kill special processes with their name. 2. Then they check corresponding openstack services and the status of processes' recovery. 3. For different SUTs, the names of the processes and the hosts' names may be different. 4. So there is a requirement for Dovetail tool to provide one way to allow users to specify all the infos about process name and its host name. 5. All process infos can be added into file DOVETAIL_HOME/user_config/pod.yaml 6. The infos include 'attack_process' and 'attack_host' for each HA test cases. 7. If not given in this file, will use Yardtsick default values. JIRA: DOVETAIL-627 Change-Id: I83cee991f72a8685080ed562597c70d73002623a Signed-off-by: xudan <xudan16@huawei.com>
2018-03-06Separate valid docker tags for functest and yardstickGeorg Kunz1-10/+12
Specifying separate valid docker tags for functest and yardstick for input validation. DOVETAIL-592 Change-Id: I3365cdbd9ab521bf1368750c3f34b6863b3f7ec8 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-03-06Bugfix: it crashes when check_tc_result returns Nonexudan1-2/+6
The function 'check_tc_result' may return 'None'. In this case it will throw an exception because of https://github.com/opnfv/dovetail/blob/master/dovetail/run.py#L65 The exception is like Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 339, in <module> main() File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in _call_ return self.main(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main rv = self.invoke(ctx) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke return callback(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 304, in main duration = run_test(testsuite_yaml, testarea, logger, kwargs) File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 65, in run_test if stop_on_fail['criteria'] == "FAIL" and kwargs['stop']: TypeError: 'NoneType' object has no attribute '_getitem_' JIRA: DOVETAIL-622 Change-Id: I9e612b380c3dfb148692074a3bf8eb31875213ac Signed-off-by: xudan <xudan16@huawei.com>
2018-03-06simplify file existence check with os.path.isfile()SerenaFeng1-16/+11
Change-Id: I89d15e18c588c27ab41596119d5288cba6e4e02d Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2018-03-06leverage data_files to eliminate so many hardcoded abs_path_seekingsSerenaFeng86-2108/+52
1. mv conf/ userconfig/ compliance/ patch to etc/ 2. after installation etc/ will be put under /etc/dovetail 3. leveraging /etc/dovetail in coding Change-Id: Icf9cb00ff7fc7cf35755f60d3e258b23e2c70839 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2018-03-02Adding SDNVPN tempest test cases to proposed_tests suiteGeorg Kunz3-0/+23
Adding the SDNVPN Tempest API test cases of the SDNVPN project to the proposed_tests suite to allow for running these tests in the CI pipeline for validating their suitability for inclusion in the OVP Euphrates release. JIRA: DOVETAIL-608 Change-Id: Ic12db1584a2540a14979094c65fc128c6020ecea Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-03-01Bugfix: Dovetail tool will crash when hosts.yaml file with incorrect formatxudan2-16/+35
If the hosts.yaml file is like, — hosts_info: 192.168.141.101: it will raise an exception, File "/usr/local/lib/python2.7/dist-packages/dovetail/utils/dovetail_utils.py", line 227, in add_hosts_info names=hostnames) File "/usr/local/lib/python2.7/dist-packages/python_hosts/hosts.py", line 54, in _init_ raise Exception('Address and Name(s) must be specified.') Exception: Address and Name(s) must be specified. If the hosts.yaml file is empty, it will crash with, File "/usr/local/lib/python2.7/dist-packages/dovetail/container.py", line 174, in create if hosts_yaml['hosts_info']: TypeError: 'NoneType' object has no attribute '_getitem_' JIRA: DOVETAIL-621 Change-Id: I547928e514b3294c048379459c07df89879fbb03 Signed-off-by: xudan <xudan16@huawei.com>
2018-02-24Adapt Bottlenecks Euphrates frameworkxudan6-16/+35
Bottlenecks stress test needs some adaptions about Bottlenecks Euphrates release. JIRA: DOVETAIL-587 Change-Id: Ia447143c8f08924ac1b190ee0173f39e462c5dfa Signed-off-by: xudan <xudan16@huawei.com>
2018-02-24Add Functest cloudify_ims test case into proposed_tests test suitexudan4-8/+27
Add cloudify_ims test case into proposed_tests test suite. Then this test case can be run by Dovetail on CI. JIRA: DOVETAIL-603 Change-Id: I5ab832002388f0916e04beef9d630de2cb37df59 Signed-off-by: xudan <xudan16@huawei.com>
2018-02-19Add 3 new HA test cases into proposed_tests test suitexudan7-24/+63
Add 3 new HA test cases into proposed_tests test suite and running them on CI. For the current 8 HA test cases and 3 other HA test cases plans to add, do some adaptions for the Yardstick Euphrates release. 1. Don't need the bugfix patch anymore. 2. The data format has changed, and has done some adaptions. 3. For Yardstick Euphrates, if setting OS_INSECURE=True, it needs to set OS_CACERT=False 4. Test case tc011 needs a image and flavor to create the stack. JIRA: DOVETAIL-614 JIRA: DOVETAIL-586 Change-Id: I9da8c8316246b9edee0d81ad18422ae6018becaf Signed-off-by: xudan <xudan16@huawei.com>
2018-02-16Add a function to check the cacert filexudan2-20/+29
1. Currently it won't pass the cacert file to other containers if the OS_AUTH_URL is http not https. 2. However, even though the OS_AUTH_URL is http, it still needs cacert file somewhere for Functest and Rally. 3. This problem is found on Fuel Euphrates. 4. Add a function to check whether the cacert file exists. 5. If exists, pass it to testing project containers. JIRA: DOVETAIL-616 Change-Id: Ied7bcc72e8f1c738bbce32c18096ca13641d3cd7 Signed-off-by: xudan <xudan16@huawei.com>
2018-02-16Add Functest test case snaps_smoke into proposed_tests test suitexudan4-56/+21
Add snaps_smoke into proposed_tests test suite. Then this test case can be run by Dovetail on CI. JIRA: DOVETAIL-596 Change-Id: I8fc4bb656bcac551ae4b693dbc4d817b665c6111 Signed-off-by: xudan <xudan16@huawei.com>
2018-02-14Add stop on failure logic and CLI optionEddie Arrage2-7/+18
- Improve test execution control by adding stop on failure flag - Modify test check function to return value regardless of whether report or local file option is employed - Modify run test loop to check for failure after every test case if stop flag set Change-Id: I27a4f3aaaa8b8cb8c52eb3f092f52048980abb44 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-02-08Add ablity to config EXTERNAL_NETWORK in env_config.shEddie Arrage1-23/+28
- EXTERNAL_NETWORK env variable can be explicitly set - If set, the value will be used in Yardstack configuration - Otherwise, previous method of using output from 'openstack network list' will still be employed Change-Id: I9d8b4a69ff3f5093c034823cc95e92e151bfade6 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-02-05Add default to --report CLI optionEddie Arrage1-6/+13
- Try new license header to deal with anteater failure - Ability to specify 'default' as argument to --report, which fetches host IP automatically and forms URL with 8000 as testapi port - This should not break the previous behavior of report option, if default option not specified Change-Id: I1e05bb52cf4fc211c59796d4e763e2e04177b406 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-02-05Add all tempest smoke test cases into debug test suitexudan3-11/+137
Add all tempest smoke test cases into debug test suite. Then these test cases can be run by Dovetail on CI. The results of CI will help to choose sub test cases from tempest smoke. Here use tempest_custom to run all tempest smoke test cases because it can support to run a custom defined sub test cases list. for Functest tempest_smoke_serial, it always runs whole tempest smoke test cases and can't be a subset. JIRA: DOVETAIL-595 Change-Id: Id6358691d2bdef149f16775ce84dcfbf12f5e568 Signed-off-by: xudan <xudan16@huawei.com>
2018-02-05Adapt Functest E release Frameworkxudan40-107/+256
Use opnfv/functest-restapi:euphrates. Adapt Functest E framework for all Functest test cases. 1. All path of functest and sdnvpn has been changed. 2. Sdnvpn can clean resources by itself with E release, no need to clean again. 3. Add a file testcases.yaml to overwrite the file in functest container, then it can support to run tempest_custom i.e. ipv6 and tempest in OVP. JIRA: DOVETAIL-584 Change-Id: I3ba9aa8e80becaa99cc89c8db4b39acbb89dfda4 Signed-off-by: xudan <xudan16@huawei.com>
2018-01-26Align container tags and test suite configuration yaml for initial releasexudan7-87/+20
Change default container tag to be: 1. Functest ovp.1.0.0 2. testapi ovp.1.0.0 Add test suite ovp.1.0.0 and set it to be the default test suite. Comment out bottlenecks CLI for initial release. JIRA: DOVETAIL-580 Change-Id: I956e49e43fc7ba0176c14079b7228089db5dc108 Signed-off-by: xudan <xudan16@huawei.com>
2017-12-25[Bugfix] Local summary with wrong results when Tempest failed at the SetupClassxudan1-11/+14
When Tempest test cases failed at the SetupClass, the record is the class name of the test cases rather than the name of the test cases. The local summary will say they are passed wrongly. JIRA: DOVETAIL-575 Change-Id: Iec879378770017eddd37e9a99346d980355bb6e2 Signed-off-by: xudan <xudan16@huawei.com>
2017-12-18Add License for the python scripts in folder offlinexudan2-2/+16
Change-Id: I9c585f132c10ef726cd859e4abe3c9a874cbe0bc Signed-off-by: xudan <xudan16@huawei.com>
2017-12-13Remove all alias tempest test cases in osinterop test specificaion docsxudan1-39/+5
1. Some tempest test cases have alias test cases because of the different versions of tempest. 2. However, Dovetail fixes the version of tempest to be 14.0.0 in its first release. So the number of test cases that can be executed in osinterop is always 205. And for different SUTs, the 205 test cases are all the same. 3. In this situation, there is no need to list the alias test cases. 4. Just list all the 205 test cases which will not confuse users when they reading dovetail docs. JIRA: DOVETAIL-566 Change-Id: I5e39dd82314833c8c0a6c1732c990f671ae64420 Signed-off-by: xudan <xudan16@huawei.com>
2017-11-28Remove one sub test case from dovetail.tempest.tc004xudan1-1/+0
1. Currently, dovetail.tempest.tc004 has 13 sub test cases defined in its config file and specification document of the test case. All these sub test cases are picked from tempest based on it's version 15.0.0. 2. For Dovetail 1st release, it fixes the version of Functest to be opnfv/dovetail:cvp.0.5.0, and Functest integretes tempest 14.0.0 in this version and it's E release. 3. There is one sub test case in dovetail.tempest.tc004 that is new in tempest 15.0.0, so it can not be recognized and executed by Functest D and E release. It is 'test_server_connectivity_cold_migration_revert'. 4. Remove this sub test case from dovetail.tempest.tc004. This means remove it from the test case config file and specification document. JIRA: DOVETAIL-561 Change-Id: Ia927af040794c7027a741a282ed75fe401f99ff1 Signed-off-by: xudan <xudan16@huawei.com>
2017-11-20Include full date in name of compressed result fileGeorg Kunz1-1/+1
The name of the compressed result file included a timestamp according to the pattern "day_hour_minute". This is a rather uncommon timestamp and can cause confusion. This patch changes the timestamp to a more common "YearMonthDay-HourMinute" format. Please note: Please check if this change has an impact on the web portal. Change-Id: Ife41050bc1d5ca6942706b756d03a1d0d251ae41 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2017-11-20bugfix: hosts info is wrongly processedSerenaFeng3-11/+22
changes: 1. add usage example in hosts.yaml.sample 2. fix /etc/hosts from wrong format {'hostname': 'ip'} to the right 'ip hostnames' 3. fix --add-hosts from wrong usage --add-host {'hostname': 'ip'} to the right --add-host='hostnames':ip 4. support per ip mapping mutiple hostnames, which is also in line with /etc/hosts schema JIRA: DOVETAIL-556 Change-Id: I92e8cb0fd3476b61f5c73a69e34c4a66aef51d6e Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-11-16Remove redundant log when launch local DBxudan1-3/+9
The script launch_db.sh will print "Already Exists" logs when the DB has been initialized by the data stored in JumpHost. Ignore this kind of logs because it won't affect the launch of the DB but it will confuse users. JIRA: DOVETAIL-557 Change-Id: I23616b10bbadc49888731e943d0a4ee4425045f5 Signed-off-by: xudan <xudan16@huawei.com>
2017-11-07Tag dovetail cvp.0.9.0xudan3-1/+59
JIRA: DOVETAIL-550 Change-Id: I871ec1fdf8d33efe3ed00602d0184c7b35896c17 Signed-off-by: xudan <xudan16@huawei.com>
2017-11-06Bugfix: Failed to build local DB on Centosxudan2-17/+17
1. The scripts for building local DB and testapi service just work on Ubuntu. 2. For Centos, they fails to do that. 3. This is mainly because of that Centos can't use the IP of the JumpHost to connect the DB and testapi. It needs to use the internal IPs of the DB container and the testapi container. 4. Change the IPs in the scripts to be the containers' internal IPs. JIRA: DOVETAIL-548 Change-Id: I71aed82cb620b58bcd43ace61497db65c5bd7ecc Signed-off-by: xudan <xudan16@huawei.com>
2017-11-02Get SUT's endpoint infoxudan2-0/+28
Get the admin endpoints of all service. JIRA: DOVETAIL-546 Change-Id: Ie0ac3bc5b4eb0885046e207b73217994a4f0e358 Signed-off-by: xudan <xudan16@huawei.com>
2017-11-02Get hardware info of SUTxudan3-0/+86
Hardware info: Use ansible to get all hardware info from all nodes. JIRA: DOVETAIL-540 Change-Id: I6a2dde1b2ebae2af8f702008c23110ebe63cc06f Signed-off-by: xudan <xudan16@huawei.com>
2017-10-23Use functest clean module to do the sdnvpn resources cleanupxudan5-0/+15
1. If there throw an exception during the test of one sdnvpn test case, it will not do the clean. 2. Then there are many resources left, such as network, subnet, server... 3. This is a bug of sdnvpn see JIRA: SDNVPN-184 4. The patch for fixing this bug https://gerrit.opnfv.org/gerrit/#/c/44777/ 5. Currently, use functest module "functest openstack clean" to clean the SUT. JIRA: DOVETAIL-524 Change-Id: I629fc6fdc9e4c95960f6122358ff219a467d441b Signed-off-by: xudan <xudan16@huawei.com>
2017-10-23Bugfix: the sdnvpn DB initialization information lacks two test casesxudan1-0/+2
1. The SDNVPN DB initialization information lacks two test cases, functest_testcase_4 and functest_testcase_8. 2. So the results of these 2 test cases cannot be pushed into DB. 3. Add these 2 test cases info to DB. JIRA: DOVETAIL-535 Change-Id: I1c6d049b942806924f15dd106452fd9b4d554454 Signed-off-by: xudan <xudan16@huawei.com>
2017-10-23Add testsuite cvp.0.8.0 for Beta testsxudan3-1/+59
1. Now there is a testsuite proposed_tests for beta tests which may cause some confusions. 2. Copy all test cases to testsuite cvp.0.8.0 for beta testers to use. 3. Update userguide with testsuite cvp.0.8.0 JIRA: DOVETAIL-528 Change-Id: I35bc7dee6e07a9125a4c1991b0049f6e2d705cc5 Signed-off-by: xudan <xudan16@huawei.com>
2017-10-16SDNVPN test cases resources left may effect the next test casesxudan1-5/+5
Move SDNVPN test cases after tempest test cases, then the resources left will not effect tempest test cases. JIRA: DOVETAIL-524 Change-Id: I815ae7f3a3be0225be1e9695f13d4552946bffb5 Signed-off-by: xudan <xudan16@huawei.com>
2017-09-29Merge "Bugfix: sdnvpn test cases failed due to wrong module name"Leo wang14-41/+38
2017-09-28Bugfix: sdnvpn test cases failed due to wrong module namexudan14-41/+38
SDNVPN test cases failed due to wrong module name. The module name needs to be changed into "sdnvpn.test.functest.testcase_1" Refer to patch https://gerrit.opnfv.org/gerrit/#/c/41385/ Currently Dovetail uses functest:cvp.0.5.0 which uses module name "testcase_1". JIRA: DOVETAIL-510 Change-Id: I3232760d3af34125c7da71a5d2c62ca32693c82b Signed-off-by: xudan <xudan16@huawei.com>
2017-09-29Merge "Support to run mandatory or optional test cases separately"Leo wang6-27/+71
2017-09-28Merge "Don't generate tar file when run without pushing results to DB"Leo wang1-1/+2
2017-09-28Merge "Make Yardstick log level can be set by CLI"Leo wang5-14/+38
2017-09-28Don't generate tar file when run without pushing results to DBxudan1-1/+2
1. Dovetail generate a tar file for uploading results to CVP web portal. 2. If users don't run with option --report, it shouldn't generate the tar file. Because users can not upload results to CVP if they don't run with --report. 3. If it generates the tar file when user run dovetail without --report, the tar file is invalid because it lacks of results.json. JIRA: DOVETAIL-511 Change-Id: I4d21f21f0b278962564d810369df03100b38f003 Signed-off-by: xudan <xudan16@huawei.com>