summaryrefslogtreecommitdiffstats
path: root/docs/testing
AgeCommit message (Collapse)AuthorFilesLines
2018-03-20Exemption process for disabling strict API validationGeorg Kunz1-0/+183
This document describes an exemption process for disabling strict API response validation. JIRA: DOVETAIL-617 Change-Id: I690efdcae7dd15a6c0781a837a62677904836fca Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-03-06Separate valid docker tags for functest and yardstickGeorg Kunz1-3/+3
Specifying separate valid docker tags for functest and yardstick for input validation. DOVETAIL-592 Change-Id: I3365cdbd9ab521bf1368750c3f34b6863b3f7ec8 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-02-01Update OVP workflow documentEddie Arrage2-50/+53
- Fix typos and word changes - Remove editor's note - Adopt rewording suggestions - Add back toc that was inadvertently deleted - Modify to reflect process used during beta and current OVP web portal usage - Remove reference to application form and use online participation form instead. Add new fields required in participation form Change-Id: Ia51b7674315467e01081015b3aa780ec03652ebd Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-01-24Modify title and references in guidelines addendum docEddie Arrage1-3/+3
- Add 2018.01 to title to match reviewer guide, release notes, etc. as this document may change per release - Modify link for Governance Guidelines to reflect ratified doc on opnfv.org domain Change-Id: Ifdaa287b2919c44d408aa0ddd0b164b8ca4f406d Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-01-24Remove a sub test case under dovetail.tempest.tc004Eddie Arrage1-118/+23
- Fixed vping, ha, sdnvpn test case names to match test specs - Removed aliases from VIM operations and matched test case name changes in patch https://gerrit.opnfv.org/gerrit/#/c/47691/4 - CVP to OVP name change - A test case was removed from tempest tc004 because of the Tempest version from 15 to 14. A reference to it was still in this doc. Change-Id: If2e07e15b48ae6bca5ac0e262aead784cd67cd0f Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-01-18Transfer Danube reviewer guidelines from wikiEddie Arrage16-8/+190
The reviewer guidelines will be created per release, where the CVP/Dovetail releases are aligned to OPNFV releases. The guidelines outlined in this initial document are for the Danube release. Updated reviewer guidelines - Updated to account for issue where mandatory total always shows 215 in current version of web portal; removed one diagram and amended another - First section title change and wording for test area results - Removed white space - Added SUT info section and diagrams - Modified reference to cvp.opnfv.org with verified.opnfv.org - Modified reference to cvp@opnfv.org with verified@opnfv.org - Modified diagram filenames to reflect program name change - Changed CVP references to OVP and Danube to 2018.01 - Remove TODO for granting marks and escalation - this info is in other docs Change-Id: I01fc568ce6713d3c60daf339a2630e56e8dbcfcb Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-01-15Changing CVP to OVP in the documentationGeorg Kunz21-503/+128
Due to the name change from 'CVP' to 'OVP', the documentation has to be updated. Change-Id: Id934e8fc07f60802f7aa6b3d295d268c4c9b71f8 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-01-15Update test user guide to improve usabilityEddie Arrage1-222/+300
- Modify hosts.yaml syntax to match patch by Serena - Modify result file references for new timestamp patch by Georg - Revise online/offline diagrams for OVP name change - Add XCF GIMP source files for diagrams with layers for new elements - New changes for cvp.0.9.0 to ovp.1.0.0 - Remove reference to bottlenecks container for now - Modify section/heading structure - Incorporate comments: results.json vs. result files - Incorporate comments: online/offline install headings - Incorporate comments: remove IP from launch_db.sh - Incorporate comments: remove sudo for wget - Incorporate comments: pod.yaml for SUT HW info - Incorporate comments: test case to config file requirement - Incorporate comments: suggested changes to env_config.sh - Modify the sequence of the guide to reflect installation steps - Outline all required configuration files earlier in the guide - Highlight configuration file key entries and pitfalls - Improve requirements for pod.yaml file for HA - Improve online/offline install instructions - Call out Dovetail docker run command and explain options - Clarify when testapi and local db ports need to be changed - Explain the dovetail run --testsuite and --testarea arguments more clearly - Improve --report option and default result result behavior - Make CVP portal a separate section - Explain CVP portal workflow to reflect current state of interface - Update references to cvp.0.8.0 with cvp.0.9.0 Change-Id: I41857c1b07c211a95f19c99054ca414351c75c20 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2017-12-13Remove all alias tempest test cases in osinterop test specificaion docsxudan5-211/+8
1. Some tempest test cases have alias test cases because of the different versions of tempest. 2. However, Dovetail fixes the version of tempest to be 14.0.0 in its first release. So the number of test cases that can be executed in osinterop is always 205. And for different SUTs, the 205 test cases are all the same. 3. In this situation, there is no need to list the alias test cases. 4. Just list all the 205 test cases which will not confuse users when they reading dovetail docs. JIRA: DOVETAIL-566 Change-Id: I5e39dd82314833c8c0a6c1732c990f671ae64420 Signed-off-by: xudan <xudan16@huawei.com>
2017-11-28Remove one sub test case from dovetail.tempest.tc004xudan1-61/+10
1. Currently, dovetail.tempest.tc004 has 13 sub test cases defined in its config file and specification document of the test case. All these sub test cases are picked from tempest based on it's version 15.0.0. 2. For Dovetail 1st release, it fixes the version of Functest to be opnfv/dovetail:cvp.0.5.0, and Functest integretes tempest 14.0.0 in this version and it's E release. 3. There is one sub test case in dovetail.tempest.tc004 that is new in tempest 15.0.0, so it can not be recognized and executed by Functest D and E release. It is 'test_server_connectivity_cold_migration_revert'. 4. Remove this sub test case from dovetail.tempest.tc004. This means remove it from the test case config file and specification document. JIRA: DOVETAIL-561 Change-Id: Ia927af040794c7027a741a282ed75fe401f99ff1 Signed-off-by: xudan <xudan16@huawei.com>
2017-11-03Minor docs change to test docs integrationGeorg Kunz1-11/+0
This is minor patch which updates the documentation in the dovetail repo in order to test the toolchain integration in opnfvdocs. Change-Id: Id91c57d1a38cf02d8ed4653bc9eddb7081f025e7 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2017-11-02Modify reference to online modeEddie Arrage1-6/+7
Changed reference to online mode to refer to installation rather than uploading results from Test Host to CVP portal directly, as current text is misleading. Change-Id: I9439d340beabb76f04b7782243c050064753a8ec Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2017-11-01Add test case name into tempest testspecification docsxudan15-43/+78
1. All test cases' testspecification docs have no info about test case names. 2. User don't know which test case to run when they just want to run one special test case. 3. Add the info into testspecification docs. JIRA: DOVETAIL-530 Change-Id: I0ebe1b3b98be5609503814ea2a0cf71aefd06d35 Signed-off-by: xudan <xudan16@huawei.com>
2017-10-25Remove obsolete developer documentsEddie Arrage5-643/+0
Some developer documents are now old and suggested to be removed. Change-Id: Ifa26a1535d57a560a0525eafa300ae087feb3191 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2017-10-24Tag dovetail with cvp.0.8.0xudan1-7/+7
JIRA: DOVETAIL-536 Change-Id: I7a947b1aa57728e6add1d0ca881a6a62a46c0913 Signed-off-by: xudan <xudan16@huawei.com>
2017-10-24System preparation guideWenjing Chu1-66/+55
Change-Id: I4fc982d7037466b48424b7ce97d6207fbe44a09b Signed-off-by: Wenjing Chu <chu.wenjing@gmail.com>
2017-10-23Fixing references and layout in the docsGeorg Kunz7-81/+90
This patchset introduces labels for documentation-internal references and replaces footnote-style citations with inline hyperlinks. Additional minor changes for a nicer rendering of bullet lists. JIRA: DOVETAIL-521 Change-Id: Ic847fc05d546d673dadc7b8efee86fcb688133f5 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2017-10-23Add testsuite cvp.0.8.0 for Beta testsxudan1-11/+16
1. Now there is a testsuite proposed_tests for beta tests which may cause some confusions. 2. Copy all test cases to testsuite cvp.0.8.0 for beta testers to use. 3. Update userguide with testsuite cvp.0.8.0 JIRA: DOVETAIL-528 Change-Id: I35bc7dee6e07a9125a4c1991b0049f6e2d705cc5 Signed-off-by: xudan <xudan16@huawei.com>
2017-10-18Updating test case requirements documentGeorg Kunz2-59/+104
This patchset addresses review comments which were not incorporated in the previous version of the test case requirements description. Additionally, the text has been wrapped to 80 character lines. Change-Id: If817f6c44d94870d915791a1a11916ffe4871f06 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2017-10-18Term refstack is mentioned in dovetail Testing User Guidexudan1-3/+1
Remove refstack.log from the userguide because the refstack.log has been renamed dovetail.osinterop.tc00X.log JIRA: DOVETAIL-519 Change-Id: Icd0bd82ef78437ab96201e58b93e0a6997f9618a Signed-off-by: xudan <xudan16@huawei.com>
2017-10-18Add dovetail tool CLI documentxudan3-1/+205
According to the wiki page, add the CLI document. https://wiki.opnfv.org/display/dovetail/Dovetail+Command+Line JIRA: DOVETAIL-509 Change-Id: I6a32446cd4c1bcad83465b1f2e12d5c1487e954e Signed-off-by: xudan <xudan16@huawei.com>
2017-10-09Merge "Cvp application form"Wenjing Chu2-3/+92
2017-10-05Cvp application formWenjing Chu2-3/+92
Change-Id: I4bb0cea49179c7171ce766c52814f765e05ce9f8 Signed-off-by: Wenjing Chu <chu.wenjing@gmail.com>
2017-09-29Update user guide with the local DB operationsxudan1-62/+85
If users want to store results or upload results to CVP web portal, they need to build local DB adn testapi service. Should add more introduction about the local DB and testapi in the user guide. JIRA: DOVETAIL-504 Change-Id: I6493169b54104af8d7207d28fad93afedeae0b1b Signed-off-by: xudan <xudan16@huawei.com>
2017-09-25Merge "Replacing references to "DefCore""hongbo tian6-27/+19
2017-09-15Clearer structuring current and future test scope of CVP addendumGeorg Kunz1-101/+128
This patchset attempts to more clearly structure the outcome of the scope analysis by having dedicated "in scope" and "future scope" sections Change-Id: I6a42a8ac4495fa4e2e5e773a7f594704d9cc1e28 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2017-09-13Replacing references to "DefCore"Georg Kunz6-27/+19
This patch removes all references to the outdated term "DefCore" in the Dovetail documentation. The DefCore working group has been renamed OpenStack InterOp Working Group some time ago. This patch also proposes renaming the "defcore" test area to "osinterop". JIRA: DOVETAIL-499 Change-Id: I7bb2ca33cb1eaf17c3ca0fc79a4f923e5886ece4 Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2017-09-11Merge "Clarify criteria for testcase inclusion as running in Functest/CI And ↵hongbo tian1-16/+32
additional clarifications."
2017-09-08Merge "add more information for the Identity testcases in the refstack"hongbo tian1-8/+148
2017-09-08Merge "Doc: update user guide due to add new test cases and features"hongbo tian1-27/+118
2017-09-07Merge "Add test area 'forwarding packets through virtual networks in data path'"hongbo tian2-0/+151
2017-09-07Merge "Test case specification for VIM Network"hongbo tian2-18/+343
2017-09-06Doc: update user guide due to add new test cases and featuresxudan1-27/+118
There has been added some new test cases and features, so user guide should be updated according to the latest dovetail version. - Add introduction about https (with cacert and without cacert) - Add introduction about adding hosts' info - Add introduction about downloading cirros images - Add introduction about special configurations for running defcore and tempest test cases - Add introduction about how to check defcore and tempest failed test cases JIRA: DOVETAIL-493 Change-Id: I13aca4ee433056bd21590743a412d5d95f45fec4 Signed-off-by: xudan <xudan16@huawei.com>
2017-09-06Add test area 'forwarding packets through virtual networks in data path'xudan2-0/+151
JIRA: DOVETAIL-473 According to Dovetail addendum doc, add one test area about basic cloud capability 'forwarding packets through virtual networks in data path'. Add the test specification about this test area. Change-Id: I26416ecca6b89a94cf48ba0f55308df0fa6e1677 Signed-off-by: xudan <xudan16@huawei.com>
2017-09-05Add test area 'simple virtual machine resource scheduling on multiple nodes'xudan2-0/+381
JIRA: DOVETAIL-473 According to Dovetail addendum doc, add one test area about basic cloud capability 'simple virtual machine resource scheduling on multiple nodes'. Add the test specification about this test area. Change-Id: I7a5ff426442424dbc364a68296facb0bac6401ca Signed-off-by: xudan <xudan16@huawei.com>
2017-09-06Merge "Add test area 'correct behavior after common virtual machine life ↵hongbo tian2-0/+775
cycles events'"
2017-09-06Merge "Test case specification for VIM Image"hongbo tian2-17/+386
2017-09-05Add test area 'correct behavior after common virtual machine life cycles events'xudan2-0/+775
JIRA: DOVETAIL-473 According to Dovetail addendum doc, add one test area about basic cloud capability 'correct behavior after common virtual machine life cycles events'. Add the test specification about this test area. Change-Id: I051b3c5b501ed833a613ea6a6601125c4fa80c3e Signed-off-by: xudan <xudan16@huawei.com>
2017-09-05Merge "Add test area 'dynamic network runtime operations'"hongbo tian2-0/+422
2017-09-04Merge "Test case specification for VIM Compute"hongbo tian2-36/+627
2017-09-01Merge "Test case specification for VIM Volume"hongbo tian2-44/+1018
2017-09-01Add test area 'dynamic network runtime operations'xudan2-0/+422
JIRA: DOVETAIL-473 According to Dovetail addendum doc, add one test area about basic cloud capability 'dynamic network runtime operations through the life of a VNF'. Add the test specification about this test area. Change-Id: I59706bc4a5765cf779a03fc3c493e5a8d184c1dc Signed-off-by: xudan <xudan16@huawei.com>
2017-09-01Merge "Add test area 'filtering packets based on security rules and port ↵hongbo tian2-0/+461
security'"
2017-08-31Test case specification for VIM Computexudan2-36/+627
1. Add the alias VIM Compute test cases into testscope. 2. Add test case specification for VIM Compute. Change-Id: Icca09a58221511f607e43eb1d0415d9de4e452eb Signed-off-by: xudan <xudan16@huawei.com>
2017-08-25Added top-down scope analysis based on cvp objectives and end-user ↵Trevor Cooper1-0/+388
expectations. Also simplified to focus on scope only. Change-Id: Ibde9fdacfd026f7859bd6fd239d79fdbdeccad01 Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-08-25Merge "Draft a workflow that will work with the CVP web portal and test API. ↵hongbo tian2-0/+132
Also made simplifications for the first release."
2017-08-22Clarify criteria for testcase inclusion as running in Functest/CIBryan Sullivan1-16/+32
And additional clarifications. JIRA: DOVETAIL-477 Replace tabs. Further clarify testing programs and optionality criteria. Change-Id: I8eebdf215fb7cc75a6da9748629b1a73d0b71d19 Signed-off-by: Bryan Sullivan <bryan.sullivan@att.com>
2017-08-18Draft a workflow that will work with the CVP web portal and test API. Also ↵Trevor Cooper2-0/+132
made simplifications for the first release. Change-Id: Ief92a7e71c546883fdf472346646a43c9ae26c94 Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-08-18Test case specification for VIM Volumexudan2-44/+1018
1. Add the alias VIM Volume test cases into testscope. 2. Add test case specification for VIM Volume. Change-Id: I715a69aad380466659365cc554351efb4d6eb053 Signed-off-by: xudan <xudan16@huawei.com>
2017-08-16Test case specification for VIM Networkxudan2-18/+343
1. Add the alias VIM Network test cases into testscope. 2. Add test case specification for VIM Network. Change-Id: I133ad13b1f926e0093d92a9a39d88947648122ea Signed-off-by: xudan <xudan16@huawei.com>