summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-09-04Update test list for OS interop testsGeorg Kunz2-6/+2
The OpenStack Interop WG has recently deprecated [1] 4 cinder volume reservation tests. As a result, they are not part of Functest's set of interop tests anymore. This patch cleans up the test case validation list for the web portal. [1] https://review.openstack.org/#/c/583970/ Change-Id: I183602b099cb4d44c6cb3d21af6acaa4d3ee01ca Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-09-03Remove identity v2 TCs,not supported by FunctestPanagiotis Karalis9-96/+0
As Functest does not support identity v2 anymore, it does not make sense to try to run tempest identity v2 tests. They need to be removed correspondingly. JIRA: DOVETAIL-735 Change-Id: I09c06faf52893cc535bbed1fd406b81a7665e787 Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-08-31Reviewer guide update for OVP 2018.08 releasedimitris.tsiolakis23-113/+126
JIRA: DOVETAIL-701 Change-Id: I940bdd16762dd36ee338f41e5b2aab352c70600f Signed-off-by: dimitris.tsiolakis <dmts@intracom-telecom.com>
2018-08-31Updating OVP addendum doc for 2018.09 releaseGeorg Kunz1-202/+200
Updating the OVP addendum document for the next release. JIRA: DOVETAIL-733 Change-Id: I7b7679f96bff4837f69769bb71256bc30602dced Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-08-29Updating test case requirementsGeorg Kunz1-18/+15
Updating the test case requirements document to match the new test case naming convention which was introduced for the 2018.08 release. Additionally, minor syntactic fixes. Change-Id: I34b4b90eb4f8bab4f59df2415c98c27169e98332 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-08-29Release notes for OVP 2018.08 releasedimitris.tsiolakis1-175/+137
JIRA: DOVETAIL-700 Change-Id: I6835a6d6987d8e45c85fab851304a455db3ab0d6 Signed-off-by: dimitris.tsiolakis <dmts@intracom-telecom.com>
2018-08-28Change Bottlenecks volume mappingxudan2-1/+4
If mapping to host's /tmp, it won't clean all .log and .out files before run a new test. It causes the results file including all previous results. See https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-baremetal-default-mandatory-master/26/artifact/results/stress_logs/bottlenecks.stress.ping.out/*view*/ It will affect the results check. Mapping to DOVETAIL_HOME/results/bottlenecks, which will always be cleaned before running another test. Change-Id: I22bfc871d6f041ae38308bf8e8f685993a376973 Signed-off-by: xudan <xudan16@huawei.com>
2018-08-27Shortening headings and testing building of docsGeorg Kunz4-12/+12
This patch shortens the headings and is used to test if the docs tool chain builds the right version. Change-Id: Ibc162be1b46ff787a90510c360881234391c5ba7 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-08-22Add a volume mapping for Bottlenecksxudan1-0/+1
All results files including the .out files are stored in Yardstick /tmp Need to map that with host's /tmp to parse the results. JIRA: DOVETAIL-727 Change-Id: I5a81e6e6bdf79b2e0074df505deb425c37b93300 Signed-off-by: xudan <xudan16@huawei.com>
2018-08-20Bottlenecks SSH connectivity brokenxudan1-1/+1
For ping, it set the ssh login username to be ubuntu. So need to use ubuntu image rather than cirros. JIRA: DOVETAIL-727 Change-Id: Iaac8c8280a464efe3a431d4eb260c4a0a626e12d Signed-off-by: xudan <xudan16@huawei.com>
2018-08-14Rename ovp.next to be ovp.2018.08xudan5-10/+10
Change-Id: I32aaee370512c3d7c8d6e1fbb6ef25aebc5d483f Signed-off-by: xudan <xudan16@huawei.com>
2018-08-14Test case descriptions for Tempest smoke testsdimitris.tsiolakis7-1/+506
Adding a high-level description of the Tempest smoke tests. As agreed on the last Dovetail call, we are aiming for a relatively low level of detail and instead point to available documentation. As the tests themselves are quite simple of read, this is the best, most concise, and up-to-date version of documentation we can find. JIRA: DOVETAIL-599 Change-Id: I985592c55fd415e254e70db6c6480c7bf3670a29 Signed-off-by: dimitris.tsiolakis <dmts@intracom-telecom.com>
2018-08-14Update cli reference guidexudan1-47/+115
JIRA: DOVETAIL-713 Change-Id: Ie740c8bf24c297c2537b23e1d35e0b4d40cd3e35 Signed-off-by: xudan <xudan16@huawei.com>
2018-08-13Remove one sub test cases from neutron trunkxudan1-1/+0
There is one sub test case that can't work correctly on Mitaka, neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_update_trunk_with_description It will get a 'Bad request' error. Details: {u'message': u"Unrecognized attribute(s) 'description'", u'type': u'HTTPBadRequest', u'detail': u''}. That's mainly because of that, For Mitaka, there's only several extension resources that support attribute 'description', which is 'security_group_rules', 'security_groups', 'ports', 'subnets', 'networks', 'routers', 'floatingips' and 'subnetpools'. All other extension resources including trunk port can't create with description. See https://github.com/openstack/neutron/blob/mitaka-eol/neutron/extensions/standardattrdescription.py#L22 For Newton, all subclasses of class 'HasStandardAttributes' support the attribute 'description' including class Trunk. See https://github.com/openstack/neutron/blob/newton-eol/neutron/extensions/standardattrdescription.py#L49, https://github.com/openstack/neutron/blob/newton-eol/neutron/db/standard_attr.py#L158 and https://github.com/openstack/neutron/blob/newton-eol/neutron/services/trunk/models.py#L26 JIRA: DOVETAIL-723 Change-Id: Ifdfea7c54a1888d933c8ddd78a6feea1db93368c Signed-off-by: xudan <xudan16@huawei.com>
2018-08-14Update user guidexudan1-83/+73
JIRA: DOVETAIL-710 Change-Id: If6150e75202851e80b3b945743eca3362dba69c7 Signed-off-by: xudan <xudan16@huawei.com>
2018-08-14Remove 2 sub test cases from neutron_trunk check listxudan1-2/+0
1. neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_update_trunk This test case will get the value of key 'revision_number' [1], While this is a new extension for Newton and not support by Mitaka. Please refer to the history of neutron github and the differences between mitaka tag [3] (which has no file revisions.py) and newton [4]. 2. neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_show_trunk_has_project_id This test case requires extension 'project-id' [5], while this is also a new extension for Newton. Please refer to the history of neutron [6] and the difference between mitaka tag [3] (which has no file project_id.py) and newton [7]. Also it is a little unreasonable to check both tenant_id and project_id. For some products, they may only have either tenant_id or project_id (not both). Has sent an email to the neutron_trunk team but get no feedback until now. [1] https://github.com/openstack/neutron/blob/stable%2Fpike/neutron/tests/tempest/api/test_trunk.py#L123 [2] https://github.com/openstack/neutron/commits/newton-eol/neutron/extensions/revisions.py [3] https://github.com/openstack/neutron/tree/mitaka-eol/neutron/extensions [4] https://github.com/openstack/neutron/blob/newton-eol/neutron/extensions/revisions.py [5] https://github.com/openstack/neutron/blob/stable%2Fpike/neutron/tests/tempest/api/test_trunk.py#L110 [6] https://github.com/openstack/neutron/commits/newton-eol/neutron/extensions/project_id.py [7] https://github.com/openstack/neutron/blob/master/neutron/extensions/project_id.py JIRA: DOVETAIL-723 Change-Id: Idb4a56d47148b14ae4674549f98b1be65e9b8f15 Signed-off-by: xudan <xudan16@huawei.com>
2018-08-13Clean results directory to reduce the size of the package uploadedxudan1-1/+5
JIRA: DOVETAIL-726 Change-Id: I57cb14cd5fe03a51e1691d7b48359bdd9b472ae4 Signed-off-by: xudan <xudan16@huawei.com>
2018-08-13Create a test case descriptions for Patrole RBAC testsxudan2-0/+307
JIRA: DOVETAIL-682 Change-Id: I37bb5ea2ba5b68407541e3e9d90d760e1720d3e7 Signed-off-by: xudan <xudan16@huawei.com>
2018-08-13Test case description for Yardstick HA caseMika Rautakumpu1-12/+91
This is test case description for HA test neutron-l3-agent. Yardstick TC058. JIRA: DOVETAIL-637 Change-Id: I90e0225e6f086982da9f6ea686fbc8c179f45984 Signed-off-by: Mika Rautakumpu <mika.rautakumpu@nokia.com>
2018-08-09Test case descriptions for SNAPS smoke testsdimitris.tsiolakis2-0/+233
Adding a high-level description of the SNAPS smoke tests. As agreed on the last Dovetail call, we are aiming for a relatively low level of detail and instead point to available documentation. As the tests themselves are quite simple of read, this is the best, most concise, and up-to-date version of documentation we can find. JIRA: DOVETAIL-630 Change-Id: I38b072e29ec4cfe50baf0ac3350ea1a1d3e8d67a Signed-off-by: dimitris.tsiolakis <dmts@intracom-telecom.com>
2018-08-02Test case descriptions for Neutron Trunk Port testsGeorg Kunz1-0/+157
Adding a high-level description of the Neutron trunk port tempest tests. As agreed on the last Dovetail call, we are aiming for a relatively low level of detail and instead point to available documentation. As the tempest tests themselves are quite simple of read, this is the best, most concise, and up-to-date version of documentation we can find. JIRA: DOVETAIL-684 Change-Id: I76801b34f974bf7c77fded3ffa5588b7e5b2803c Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-08-01First test case description for HA testsMika Rautakumpu1-1/+101
This is one of three test case descriptions from jira ticket. Test case description is TC010 it is TC056 in yardstick. JIRA: DOVETAIL-605 Change-Id: I70afa882a667a6932aa7ec29bc5e0621b43fa0a4 Signed-off-by: Mika Rautakumpu <mika.rautakumpu@nokia.com>
2018-08-01Dovetail should update testcases.yaml filexudan1-2/+11
It failed to run trunk_port test case because of the wrong class name. See CI results https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-default-mandatory-fraser/7/artifact/results/tempest_logs/dovetail.tempest.neutron_trunk_ports.functest.log/*view*/ Update the testcases.yaml file based on the latest version of Functest Fraser. JIRA: DOVETAIL-715 Change-Id: Ib102d1489d1e5610a04d76aab0553d5171ec3d3e Signed-off-by: xudan <xudan16@huawei.com>
2018-07-31Renaming test cases to include test framework namesGeorg Kunz50-267/+265
Replacing the generic "dovetail" prefix of the testcase definitions with the name of the OPNFV test framework which hosts the test. JIRA: DOVETAIL-719 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com> Change-Id: I8f90423b8908197bdc7adddf82c61ff394893f52
2018-07-30Dovetail failed to archive tempest.log and tempest-report.html filesxudan18-36/+28
It failed to archive tempest.log and tempest-report.html files. See CI results https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-default-mandatory-fraser/7/artifact/results/ That's because the path of them have been changed 1. from tempest to tempest_custom 2. from refstack to refstack_defcore There seems to be no file bgpvpn.log, remove the copy operation. JIRA: DOVETAIL-714 Change-Id: I77d88124a44d3e5d869ada209760f69d1f6553bf Signed-off-by: xudan <xudan16@huawei.com>
2018-07-27Delete 2 sub test cases from patrolexudan1-2/+0
These 2 test cases failed on OSP10 with reasons 1. Provided object does not match schema 'image': u'shared' is not one of ['public', 'private'] 2. Provided object does not match schema 'image': u'community' is not one of ['public', 'private'] 'shared' and 'community' was added in the Image API v2.5 https://developer.openstack.org/api-ref/image/v2/index.html#images JIRA: DOVETAIL-704 Change-Id: Ie7ab06d5e2abe7bde25fdb7318abe341f0119bc7 Signed-off-by: xudan <xudan16@huawei.com>
2018-07-27Update bottlenecks confxudan1-23/+4
1. Copy a ubuntu image to Bottlenecks to support offline 2. Add an env DEPLOY_SCENARIO to bottlenecks to support DPDK 3. Set push results to DB False JIRA: DOVETAIL-706 JIRA: DOVETAIL-707 Change-Id: I81f293bb9fddc63fd8be455546ba4642ca670fc7 Signed-off-by: xudan <xudan16@huawei.com>
2018-07-27Create a test specification for vIMS test casePanagiotis Karalis1-3/+80
New test specification has been created for dovetail project. Test descriptions is related to cloudify_ims test case of functest project. JIRA: DOVETAIL-699 Change-Id: I033351ad7cf83256bd7a9e208a7ce406d5a4cd2b Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-07-25Cleaning up proposed-tests test suiteGeorg Kunz1-26/+1
We have moved all newly proposed tests to the next OVP test suite, so these tests do not need to be in here anymore - and do not need to be run in CI. Change-Id: I5f51b76a37b706c5b4abc75272ad799a72841f67 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-07-25Move OVP web portal code to a separate repoxudan531-120170/+0
The new repo for web portal is https://gerrit.opnfv.org/gerrit/dovetail-webportal JIRA: DOVETAIL-671 Change-Id: Iac085abc3d175b9a091d70d0448af56c7a6845e9 Signed-off-by: xudan <xudan16@huawei.com>
2018-07-24Making the results file a proper json fileGeorg Kunz5-37/+51
The results.json file is currently not a properly formated json file, but contains one json structure per line/testcase. Whit this change, all test results are being collected at runtime and then written to a properly formated json file at the end of a test run. Moreover, this patch introduces 'mandatory' and 'optional' sections to the test suites definition files in order to clearly indicate if a test is part of the corresponding test scope. The web portal needs to be adapted accordingly to be able to read the new results file. JIRA: DOVETAIL-672 Change-Id: Id0706c7e82622fc9c2c2fece26b2d6c07c1580be Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-07-23Delete skipped sub test cases from patrole and neutron_trunk check listxudan2-23/+0
According to Apex, Compass, Fuel CI results, delete the skipped sub test cases from patrole and trunk_port check list. JIRA: DOVETAIL-703 Change-Id: I86e3036a287d9cf56d78e43f92ce0d9ffed5941c Signed-off-by: xudan <xudan16@huawei.com>
2018-07-22Remove functest_testsuite and functest_testcase in dovetail_config.ymlxudan6-66/+194
The 2 sections 'functest_testsuite' and 'functest_testcase' are used by all Functest test cases. They aim to parse the results of Tempest sub test cases. Remove them, and check the sub test case list, 1. if it's empty, that means no need to check sub test cases 2. if it is not empty, check the sub test cases one by one. Additional, remove some useless keys in dovetail_config.yml JIRA: DOVETAIL-658 Change-Id: Ic829264b0c8eaedd8e2b3fa55b38a34644482c20 Signed-off-by: xudan <xudan16@huawei.com>
2018-07-18Create test case descriptions for Bottlenecks stress testxudan2-0/+160
This is coming from the former patch https://gerrit.opnfv.org/gerrit/#/c/37715/ JIRA: DOVETAIL-631 Change-Id: Ibe090fefaa92cb3fe6d7c3d96296724e8cfbcf1d Signed-off-by: xudan <xudan16@huawei.com>
2018-07-17Update test suite ovp.nextxudan10-113/+151
Update test suite ovp.next according to the latest OVP scope. Modify Dovetail daily jobs to run all the scope. There should be 2 jobs. One for all mandatory test cases and the other for all optional ones. That's mainly because of that the total executed time will be too large (more than 300 minutes). Then the job will always failed because of time out. It's hard to enlarge the time because it's already larger than common 3 hours. Split it into 2 jobs can avoid the time out issue and make the results clearer for reviewing. JIRA: DOVETAIL-694 Change-Id: Ie0ea6221868941781af1477f7c7719f7cb4351a4 Signed-off-by: xudan <xudan16@huawei.com>
2018-07-17Create a test specification for vEPC test casePanagiotis Karalis2-0/+150
New test specification has been created for dovetail project. Test descriptions is related to vEPC test case. JIRA: DOVETAIL-679 Change-Id: I72228f7f59a4aa704a620351ad8727c1b948dc66 Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-07-16Test case description for BGPVPN Tempest API testsGeorg Kunz1-0/+196
Adding test case descriptions for the BGPVPN API tests which are implemented as Tempest plugin in the networking-bgpvpn repository. These tests have already been integrated for Euphrates into Functest test and are hence a candidate for inclusion in the test scope of the OVP Euphrates release as extension of the existing SDNVPN test cases. JIRA: DOVETAIL-598 Change-Id: Id8578ed1e198ddbde967793737c69a5940c22475 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-07-16Update cirros image for bottlenecksGeorg Kunz1-5/+5
The fraser-based OPNFV tools use the newer cirros-0.4.0 image and we hence only ask users to download this version in the user guide. This updates the bottlenecks test to comply with this. Change-Id: I5a94d6a7196ae04e00150dc17a4a16add2d3fef6 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-07-13Remove the yardstick_tag and functest_tagxudan3-59/+0
These 2 commands can't work now. Also there is no need to support them, so remove them. JIRA: DOVETAIL-695 Change-Id: I6707c7fc1d19fe06690cd14ef39f9d421d00a2bc Signed-off-by: xudan <xudan16@huawei.com>
2018-07-12Enable OVP tests to run on a DPDK-enabled systemPanagiotis Karalis6-12/+55
DPDK-enabled NFVIs require support for hugepages which in turn is configured by means of extra_specs in flavors. Currently, Functest enables hugepage support if the name of the scenario contains the substring "ovs". Dovetail allows end users to run all tests on DPDK-enabled systems and the documentation has been updates accordingly. JIRA: DOVETAIL-665 Change-Id: If4898da168f8b4c69a045ef3b00b8a8862f29b0c Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-07-10CI ERROR - File contains violation on test_runnerPanagiotis Karalis1-1/+1
Proposed solution by CI server: "Avoid dangerous file parsing & serialization libs, use yaml.safe_load" Change-Id: Iff9764601bc135cec271e16444d0bbbb4da8d459 Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-07-09New test specs for ha database/controller_restartPanagiotis Karalis1-0/+150
New test specifications have been created for dovetail project. Test descriptions are related to test cases: - dovetail.ha.database - dovetail.ha.controller_restart JIRA: DOVETAIL-680 JIRA: DOVETAIL-681 Change-Id: I632cb69f9166a46e76f38a467f078fe5f31b63b3 Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-07-06Add dovetail-webportal repo to INFO file ListingTrevor Bramwell1-0/+1
Change-Id: I429e5bb1dffb1c63301ad5e9aba34d08d788ca27 Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
2018-07-04Update all existing test specificationsxudan19-1287/+1224
1. Update the names of 2 vping test cases 2. Update the names of 8 ha test cases 3. Update the sub test cases within osinterop 2017.09 4. Split IPv6 into ipv6_api and ipv6_scenario 5. Update the names of sdnvpn test cases 6. Update the names of 4 tempest scenario test cases 7. Remove forwardingpackets JIRA: DOVETAIL-685 Change-Id: I0d91b8a0477576e18581eb2788fdb666063c89b7 Signed-off-by: xudan <xudan16@huawei.com>
2018-07-03Adapt Yardstick fraserxudan1-3/+3
Yardstick fraser has changed from openstack python client to shade. OS_INSECURE is not a key for shade. It needs to be OS_VERIFY. Change-Id: I0973853d0527b04ec371ad4f6ce2f6c67ef50ccf Signed-off-by: xudan <xudan16@huawei.com>
2018-07-02Fix syntax error on clean_results_dir methodPanagiotis Karalis1-1/+1
Change-Id: I9f33bb320452efab79be5a22e625efac820183eb Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-07-02Splitting the networking tests in API and scenario testsGeorg Kunz3-12/+37
Split the networking tempest test case in Tempest API tests and scenario tests. The API tests will be promoted to mandatory while the scenario tests will remain optional. JIRA: DOVETAIL-678 Change-Id: I03022316076809536782e82fd531d98f45105caa Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-06-29Removing unused test suite and updating debug test suiteGeorg Kunz2-11/+6
The compliance_set test suite is not used. Updating the description of the debug test suite. Change-Id: Id2c3ca922ffb083f3443e00e5ca1a0648aec3e4a Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-06-29Update licence headers to comply with guidelinesGeorg Kunz12-11/+42
Updating the licence headers to comply with the community guidelines [1] and anteater checks. [1] https://wiki.opnfv.org/display/DEV/Contribution+Guidelines#ContributionGuidelines-License Change-Id: Idec787e32daf76c0b1bd6108977b78828125da5e Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-06-29Splitting IPv6 tests in API and scenario testsGeorg Kunz3-15/+40
Spliting the IPv6 test case in Tempest API tests and scenario tests. The API tests will be promoted to mandatory while the scenario tests will remain optional. JIRA: DOVETAIL-677 Change-Id: If3dca5ecf2ce5c9e02de5c6d45e674da029b609f Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>