summaryrefslogtreecommitdiffstats
path: root/docs/testing/developer/testcaserequirements
diff options
context:
space:
mode:
authorChristopherPrice <christopher.price@ericsson.com>2017-04-25 11:09:26 +0200
committerChristopher Price <christopher.price@ericsson.com>2017-04-26 09:59:20 +0000
commit7a9ad808527d29ceaeea584ae3fcef701a8cc58a (patch)
tree9e9623e1bd83613def0b4cd9f8f59fb7e523788e /docs/testing/developer/testcaserequirements
parent2fbc0733dbcebb9d3155775e7001977f909ffd50 (diff)
Committing the test case requirements specification to GIT for committer review.
This doucment needs to be +2'd by the majority of committers prior to merging to ensure we are all in agreement with regard to the test case requirements. This is a pre-requisite to test case selection for the CVP. Updated by dneary 2017-04-25: List formatting, and reorganising requirements for installer and stack composition. Change-Id: Iabaf11bc9479a20a58f985233d460013e4690d7c Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
Diffstat (limited to 'docs/testing/developer/testcaserequirements')
-rw-r--r--docs/testing/developer/testcaserequirements/index.rst92
1 files changed, 92 insertions, 0 deletions
diff --git a/docs/testing/developer/testcaserequirements/index.rst b/docs/testing/developer/testcaserequirements/index.rst
new file mode 100644
index 00000000..5dc87713
--- /dev/null
+++ b/docs/testing/developer/testcaserequirements/index.rst
@@ -0,0 +1,92 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Ericsson AB
+
+==========================================================
+Compliance and Verification program test case requirements
+==========================================================
+
+.. toctree::
+ :maxdepth: 2
+
+Version history
+===============
+
++------------+----------+------------------+----------------------------------+
+| **Date** | **Ver.** | **Author** | **Comment** |
+| | | | |
++------------+----------+------------------+----------------------------------+
+| 2017-03-15 | 0.0.1 | Chris Price | Draft version |
+| | | | |
++------------+----------+------------------+----------------------------------+
+
+CVP Test Suite Purpose and Goals
+================================
+
+The CVP test suite is intended to provide a method for validating the interfaces
+and behaviors of an NFVi platform according to the expected capabilities exposed in
+OPNFV. The behavioral foundation evaluated in these tests should serve to provide
+a functional baseline for VNF deployment and portability across NFVi instances.
+All CVP tests are available in open source and are executed in open source test frameworks.
+
+Test case requirements
+======================
+
+The following requirements are mandatory for test to be submitted for consideration in the
+ CVP test suite:
+
+- All test cases must be fully documented, in a common format
+
+ - Clearly identifying the test procedure and expected results / metrics to determine a “pass” or “fail” result.
+
+- Tests must be validated for purpose, tests should be run with both an expected positive and negative outcome.
+- Tests should focus on functionality, not performance.
+
+ - Performance test output could be built in as “for information only”, but must not carry pass/fail metrics.
+
+- Test cases should favor implementation of a published standard interface for validation
+
+ - Where no standard is available provide API support references
+ - If a standard exists and is not followed, an exemption is required
+
+- Test cases must pass on OPNFV reference deployments
+
+ - Tests must not require a specific NFVi platform composition or installation tool
+
+ - Tests and test tools must run independently of the method of platform installation and architecture.
+ - Tests and tool must run independent of specific OPNFV components allowing different components such as storage backends or SDN controllers.
+
+ - Tests must not require un-merged patches to the relevant upstream projects
+ - Tests must not require features or code which are out of scope for the latest release of the OPNFV project
+
+- Tests must run against a fully deployed and operational system under test.
+- Tests and test tools must support stand alone OPNFV and commercial derived OPNFV based solution
+
+ - There can be no dependency on OPNFV resources or infrastructure.
+
+- The following things must be documented for the test case:
+
+ - Use case specification
+ - Test preconditions
+ - Basic test flow execution descriptor
+ - Post conditions and pass fail criteria
+
+- The following things may be documented for the test case:
+
+ - Parameter border test cases descriptions
+ - Fault/Error test case descriptions
+
+New test case proposals should complete a CVP test case worksheet to ensure that all
+of these considerations are met before the test case is approved for inclusion in the
+CVP test suite.
+
+Dovetail Test Suite Naming Convention
+=====================================
+
+Test case naming and structure for dovetail needs to be done, it should consider:
+
+- Identifying the test area
+- Identifying the purpose of the test case
+
+To be discussed, agreed and amended to this document.
+