summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2019-09-10Introduce XCI installerPanagiotis Karalis2-0/+227
After the expression of the need to run the dovetail tool over the XCI installer, the investigation to that direction has been started. The aim of this patch is to introduce the relevant impact either on dovetail's code or on dovetail's documentation. JIRA: DOVETAIL-766 Change-Id: I6931506cb80baa4fa4288141ba86a858bc4215b0 Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2019-09-10Use assertEqual instead of assertEqualsxudan9-212/+211
DeprecationWarning: Please use assertEqual instead Change-Id: I84db84e85a8a43486109bb948258e57c8b86fdfb Signed-off-by: xudan <xudan16@huawei.com>
2019-09-02Add portal_key_file into all test case yml filesxudan53-0/+70
Change-Id: If111a8cc56c31ec3e22b090aca92ff56d41f7e01 Signed-off-by: xudan <xudan16@huawei.com>
2019-08-27Add validation disabled/enabled to results.jsonxudan6-11/+28
Change-Id: I918f85c5ac89cf2745efa8513946a2b5ab8638b1 Signed-off-by: xudan <xudan16@huawei.com>
2019-08-23Add run tests APIxudan8-14/+634
Change-Id: I456d5d1459faa20d9dd8afd0c235a4bb6cbfbcce Signed-off-by: xudan <xudan16@huawei.com>
2019-08-05Update Dovetail for XCI and Functest Hunterxudan27-336/+19
All info are recorded in etherpad https://etherpad.opnfv.org/p/Dovetail_XCI_TCs Change-Id: Iec54ecca63ef9d83a2e6cceb4c8664a9d107eab4 Signed-off-by: xudan <xudan16@huawei.com>
2019-07-12Update OVP Support Email AddressTrevor Bramwell3-4/+4
As the portals have moved from the opnfv.org domain to lfnetworking.org, the support address has changed as well from 'verified' to 'ovp-support'. Change-Id: I341093992e456ac90580b3b96206a402f98bf2d2 Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
2019-07-10Update the Certification Workflow documentTrevor Bramwell1-49/+51
These changes are at the request of Brandon Wick. Change-Id: Ieb4ea0164656e349451820071dc0c3d3f9bf67ac Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
2019-07-10Add Swagger UI for dovetail APIxudan7-1/+195
Change-Id: If8f515b02f0372955739dd580967a3198930e98b Signed-off-by: xudan <xudan16@huawei.com>
2019-07-02Update osinterop check list with Hunter scopexudan1-217/+219
Change-Id: If8ca533772ad85a3789f5f448adbeb16491c23b0 Signed-off-by: xudan <xudan16@huawei.com>
2019-07-02Update trunk port list with Hunter and using blacklistxudan2-2/+32
1. Using blacklist to avoid running some sub test cases 2. Update the sub test cases according to Functest Hunter scope Change-Id: I61d6d796dcdfe0fef41ba7578d0f8f2221414686 Signed-off-by: xudan <xudan16@huawei.com>
2019-06-21Use Functest blacklist to exclude the sub test cases of patrolexudan2-6/+52
1. using blacklist to exclude some sub test cases 2. update the patrole scope according to Hunter using patrole 0.4.0 Change-Id: I32de419e482cf2f2d7289dba48155f3d09fdec91 Signed-off-by: xudan <xudan16@huawei.com>
2019-06-12Add RESTful APIs into Dovetail containerxudan8-5/+58
JIRA: DOVETAIL-774 Change-Id: Ifb934eaeda52c1d316d80d9bb9c1890ffb71049c Signed-off-by: xudan <xudan16@huawei.com>
2019-06-10Bugfix: Fix Dovetail 2 bugs found with CI resultsxudan7-26/+8
There are 2 bugs found with CI results of functest.tempest.neutron_tempest_plugin_api and all HA test cases. Yardtsick conf file does't mapping the keyfile to Yardstick container, so all HA test cases got SSHERROR. Also remove some useless debug msg to make dovetail.log more concise. Change-Id: I63c12d1b91ed5f8f8a893689384685ca9215f367 Signed-off-by: xudan <xudan16@huawei.com>
2019-05-23Change the OVP version to ovp.2019.0xPanagiotis Karalis6-12/+12
The OVP version is updated from ovp.2018.09 to ovp.2019.0x Change-Id: Ibbfe186f791c521155173fd405b0147844893cf8 Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2019-05-23Fix doc link errorxudan1-1/+1
Change-Id: I405acb86132eb4f430e6df63e890b404af052ec7 Signed-off-by: xudan <xudan16@huawei.com>
2019-05-22Change Functest version from gambia to hunterxudan2-2/+2
Change-Id: I9842467035572e869a92099f11288c6b9d773600 Signed-off-by: xudan <xudan16@huawei.com>
2019-05-22Remove sdnvpn test casesxudan20-1450/+8
Functest removed test case 'bgpvpn' from scope of Hunter release. Also remove them from Dovetail for next release. The original link of refstack is invalid now. Changing it to interop link. Change-Id: I5b686496e106d688b8b3768401be95ac58fa2122 Signed-off-by: xudan <xudan16@huawei.com>
2019-05-16Fix the verdict of sub test casesxudan2-3/+10
Error to group skipped sub test cases to passed one. Example: Skipped sub test case will be parsed as passed in following case. skipped sub test case: tempest.api.network.test_routers.RoutersTest.test_create_show_list_update_delete_router passed sub test case list includes this one: tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_subnet_id Then the skipped one will be checked as pass. JIRA: DOVETAIL-775 Change-Id: I449f4968c23a9def1177a7c9645e5dcb89b2ec30 Signed-off-by: xudan <xudan16@huawei.com>
2019-05-09Add Licensexudan106-0/+954
Many files in Dovetail repo are lacking of License after LF scanning. Add all of them except the following files: 1. docker/Dockerfile.aarch64.patch 2. 0001-Allow-additional-properties-in-API-responses.patch 3. docs/images/dovetail_offline_mode.xcf 4. docs/images/dovetail_onfline_mode.xcf 5. all png files 6. all empty files Change-Id: If1a0b7c2d35d6682c87c4eb9106e1998dc4ca3bc Signed-off-by: xudan <xudan16@huawei.com>
2019-04-26Change HEAT package from .tag.gz to be .zipxudan4-9/+10
As Trevor Lovett suggests, the Heat file is a zip file. So users need to package it different only for Dovetail testing. It will be better to change it to be .zip to keep it the same as what they do. JIRA: DOVETAIL-770 Change-Id: I5338720c5e1363081e1f8cdce33391a969f1cd97 Signed-off-by: xudan <xudan16@huawei.com>
2019-04-17Failed to upload results to ONAP portalPanagiotis Karalis7-1/+137
The results.json file generated by running 'dovetail run --testsuite xxx' is different from the one generated by running 'dovetail run --testcase xxx'. The former can be uploaded to ONAP portal successfully but the latter one can't. The main reason is that the latter one lacking of 'vnf_type'. Keep the results.json with the same format for both running one test suite or several test cases. JIRA: DOVETAIL-769 Change-Id: I46af84f9f229f9e601439f68c9ed9df9477d002b Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2019-04-12Update user guidexudan2-7/+29
Change-Id: I595975d13630c81c5d15ac0051225143d0394ba4 Signed-off-by: xudan <xudan16@huawei.com>
2019-04-03Push CI results to OPNFV test DBxudan6-12/+129
In order to use OPNFV test results page to do the ayalysis of all OVP test cases, it needs to push all results generated by Dovetail CI jobs to test DB and then using http://testresults.opnfv.org/test/#/results to check all results. The following data must contain in the POST body: 1. project_name 2. case_name 3. details 4. installer 5. scenario 6. pod_name 7. build_tag 8. criteria 9. start_date 10. stop_date 11. version JIRA: DOVETAIL-767 Change-Id: I925ae249e24efd7bfb1c68a69150e9c22f0cdf36 Signed-off-by: xudan <xudan16@huawei.com>
2019-03-29Stamatis stepping down.Aric Gardner1-7/+7
https://lists.opnfv.org/g/opnfv-tech-discuss/message/22993 Change-Id: Ibce504958dce88bb20d22b7be2e9ca2478a626fe Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
2019-03-28Fix VVP test caseStamatis Katsaounis5-9/+13
This patch adds check for no environment variables provided to container configuration. Furthermore, it changes the check results file to a list of check result files. Change-Id: I2112ee1fd4b3156608d8a27dd3b9c90dbe35a24c Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com> Co-Authored-By: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2019-03-20Improve DockerfileStamatis Katsaounis3-25/+19
This patch improves the Dockerfile which produces the Dovetail image. Change-Id: I65e65f155afe237b11c668e6486e48b55cc7e96c Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-03-18Replace subprocess commands with Docker SDKStamatis Katsaounis11-262/+282
JIRA: DOVETAIL-752 This patch replaces all python subprocess commands with the Docker SDK relevant commands. Change-Id: Iac7caffd80a863a8a022247d735b2a7f2792e49d Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-03-18Update tempest bgpvpn test casexudan1-18/+21
For functest.tempest.bgpvpn, it has been run by Functest networking-bgpvpn since Functest Gambia. Also the test list has been updated. Using Functest smoke image to run networking-bgpvpn test case. The test case list can be found here: http://testresults.opnfv.org/test/api/v1/results?case_name=networking-bgpvpn&version=gambia Change-Id: I18dd0fa24011c8805c2138d0efcb74474051c6c8 Signed-off-by: xudan <xudan16@huawei.com>
2019-03-13Calculate checksum for input VNFStamatis Katsaounis7-21/+86
This patch adds checksum information inside ONAP related test case run results. The checksum is produced by the VNF input which can either be a CSAR file or an archive of Heat templates. Change-Id: I0ed58bdc9cc4031da08fd2ac220ef294520ef447 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-03-13Add Neutron Tempest plugin API testsPanagiotis Karalis2-0/+566
Add in the Dovetail project the Neutron Tempest plugin API tests which are part of Functest project. Including the 547 sub-tests as first phase of this integration. JIRA: DOVETAIL-764 Change-Id: Idbebe8191fc788e8d62c4a4f95e159d9538c6e19 Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2019-03-12Update release note and user guide before tagging ovp-2.2.0xudan2-13/+34
Change-Id: I1b1a1c96db933dc54242a226a6b8ea4a58147a14 Signed-off-by: xudan <xudan16@huawei.com>
2019-03-12Update tempest compute listxudan1-1/+1
For Functest Gambia, it uses tempest with the commit ID https://github.com/opnfv/functest/blob/stable/gambia/docker/smoke/Dockerfile#L5 Then there is one sub test case in this compute list that needs to be updated. Change-Id: Ie38627853e1a69094764c72479c2b0f2dd796bbf Signed-off-by: xudan <xudan16@huawei.com>
2019-03-08Add Functest Rally test casesStamatis Katsaounis21-12/+633
This patch adds all Rally testcases of Functest to Dovetail. Change-Id: I8b20bce9966e924fd5aac41bde5a30442394db05 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-03-07Update python requirementsStamatis Katsaounis2-20/+6
JIRA: DOVETAIL-754 This patch removes unused python requirements and updates the versions of the remaining to be equal with upper constraints of OpenStack Queens version. Change-Id: Iefb9321b56c3e2004bf73a1b57919f898a5f8bdd Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-03-07Fix the verdict of TC in sub-testcase list resultPanagiotis Karalis2-0/+13
When the sub-test case list is defined, in the dovetail results, each test case is appeared with the its verdict next to its name. Once a sub-testcase is passed, the verdict PASS is appeared next to the name of sub testcase. Once a sub-testcase is skipped or failed, the verdict FAIL is appeared next to the name of sub-testcase. This fix updates the code so that the verdict of skipped and failed sub test cases to be appeared respectively next to its sub-tsetcase name. JIRA: DOVETAIL-763 Change-Id: Ib05b3793a57acbaa67b89321a840ae11251c2417 Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2019-03-07Provide a list of check results files instead of oneStamatis Katsaounis46-116/+168
This patchs gives the ability to provide more than one check results file. Furthermore, it does not alter the behavior of the existing tests but prepares dovetail to be able to fetch information from many sources. Change-Id: I23cd23d2908ad6bdddd94b977813d0e81dc0c05a Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-02-27Archive rally.log instead of tempest.log for tempest based testsxudan15-15/+15
Change-Id: Ibdacdd92c225aa3972eefc6112a118f2a4e6652c Signed-off-by: xudan <xudan16@huawei.com>
2019-02-22Require 100% CoverageStamatis Katsaounis1-0/+1
This patch applies strict requirement for full coverage of python code as an acceptance criterion. Change-Id: Ib6dad44eb4e270efba912ce482676c360da15d84 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-02-22Add a doc introducing dovetail frameworkxudan2-0/+382
This doc is used to introduce Dovetail framework and how to develop with this framework. JIRA: DOVETAIL-757 Change-Id: I3c56ce56151580d0e2aebf3485a55f4c7a23c8b6 Signed-off-by: xudan <xudan16@huawei.com>
2019-02-22Update the check list of osinterop sub test casesxudan1-1/+18
Functest Gambia transfer OpenStack interop version from 2017.09 to 2018.02. The test cases executed by Functest has increased to 217. Adding the other 17 new test cases into Dovetail check list when parsing the results. The detailed info of all sub test cases are recorded in rally.log instead of tempest.log. So change the archive log to be rally.log. Change-Id: I1f6fa5e1ff2ec47872836a0bd0ad19531135e41c Signed-off-by: xudan <xudan16@huawei.com>
2019-02-20Remove deprecated ONAP lifecycle test caseStamatis Katsaounis17-480/+9
This patch removes the ONAP lifecycle deprecated test case. Furthermore, it clears helper functions as well as unwanted configuration. Change-Id: Ib958457cf120ffb641a72be028d3a031741c44c6 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-02-20Add a quick start guide for ONAP VNF tests with Dovetailxudan2-0/+157
JIRA: DOVETAIL-759 Change-Id: Id1032d758b618803f09d036dfa9ef8fd54ada6ce Signed-off-by: xudan <xudan16@huawei.com>
2019-02-16Add ONAP Heat testsuiteStamatis Katsaounis2-0/+9
This patch adds a testsuite for running ONAP Heat test cases. Change-Id: I25a35a776dd58570379c9a493dffffc53677a396 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-02-14Dynamic results version and vnf type recognitionStamatis Katsaounis7-3/+26
This patch makes it able to fetch the version from the testsuite. Furthermore, in case of ONAP test cases it adds the vnf_type to the produced results.json file in order to be parsed from the results portal. Finally, it creates a testuite for ONAP TOSCA related tests. Change-Id: I613358c445cfe62730b98fee5e8c6fba7d322e90 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-02-01Add ONAP VVP Heat Template validation testsStamatis Katsaounis18-34/+398
Change-Id: I9a3d56932ce41191901381831013768d1c57a749 Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
2019-01-03Add ONAP VNF SDK verification testsxudan11-10/+395
Please refer to env_config.sh.onap.sample when testing it. The guide of preparing local env for onap package validation can be found here https://gerrit.onap.org/r/#/c/75119 Change-Id: I39d6b11544847756126623a7eb3b953dc800c470 Signed-off-by: xudan <xudan16@huawei.com>
2018-12-20Simplify project conf filesxudan33-289/+102
1. use copy commands instead of pre_copy 2. remove redundant items in project conf files 3. make the volume mapping clearer Change-Id: I6c6aa58fac65d7e40105e0a54f6544ee5c47db31 Signed-off-by: xudan <xudan16@huawei.com>
2018-12-17make item 'extra_container' available for every projectxudan2-4/+11
The item 'extra_container' used to be available for Bottlenecks only. Make it open for Dovetail framework. Change-Id: If4f2c032924447b97e401d497aca0309c0f5d847 Signed-off-by: xudan <xudan16@huawei.com>
2018-12-11Enable ShellRunnerxudan6-117/+49
1. fix the bug 'ShellRunner' object has no attribute 'archive_logs' 2. remove the testarea_supported because it's not very necessary for the test cases themselves and can make it a little more simple to add a test case with new test area JIRA: DOVETAIL-750 Change-Id: I71ee74615200376adca2a0db040753e5fce329bc Signed-off-by: xudan <xudan16@huawei.com>