Start documenting CI Loops and Confidence Loops applied by XCI

Please note that the document you are about to review contains information
about the existing CI Loops and promotion criterias applied by XCI in order
to record the current state of affairs.

Once this change gets merged, a new change will be proposed and that change
will be the one where the existing criterias will be updated and the new ones
will be introduced.

This document is created to start the conversation around determining
the promotion criterias (test scopes) for each of the current and upcoming
CI Loops XCI uses.

The expectation with this document is to update it collaboratively with
scenario owners, projects, XCI Team, Test Projects, and Release Management
to find the necessary level of testing as much as possible.

The information this document contains and the criterias set in it will be
superseded by the CD-Based Release model once it becomes available.

Change-Id: I7b4afd87a4f7694eb78c996e7870b8dc1e833fbf
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>

1 files changed, 74 insertions, 0 deletions

diff --git a/docs/xci-criterias-cls.rst b/docs/xci-criterias-cls.rst
new file mode 100644
index 00000000..0a0f8f97
--- /dev/null
+++ b/docs/xci-criterias-cls.rst
@@ -0,0 +1,74 @@
+.. _xci-criterias-cls:
+
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. SPDX-License-Identifier: CC-BY-4.0
+.. (c) Fatih Degirmenci (fatih.degirmenci@ericsson.com)
+
+=============================================
+XCI Promotion Criterias and Confidence Levels
+=============================================
+
+This document is structured in a way to explain the current Promotion Criterias and Confidence
+Levels XCI uses to test and promote the scenarios. This is followed by other chapters to
+start the conversation around how these criterias can be improved depending on the features
+and scenarios that are onboarded to XCI or declared interest in participating.
+
+The expectation is to update this document collaboratively with the feature projects, scenario
+owners, XCI team, test projects and release management to find right/sufficient/necessary
+level of testing that are relevant to the features and scenarios.
+
+This document should be seen as guidance for the projects taking part in XCI until
+the OPNFV CD-Based Release Model and the criterias set for the CI Loops for that track
+become available. Until this happens, CI Loops will be constructed/updated by taking input
+from this document to provide feedback to the projects based on the test scope set by the
+projects themselves.
+
+The CD-Based Release Model will supersede the information and criterias set in this document.
+
+Existing CI Loops and Promotion Criterias
+=========================================
+
+XCI determined various CI Loops that run for the scenarios that take part in XCI.
+These loops are
+
+* verify
+* post-merge
+
+Currently, XCI uses verify and post-merge loops to verify the changes and promote
+the scenarios to the next loop in the CI Flow as candidates. The details of what
+is done by each loop currently are listed below.
+
+verify
+------
+
+The changes and subsequent patches enter this pipeline and get verified against
+the most basic criteria OPNFV has.
+
+* virtual noha deployment
+* functest healthcheck
+
+The checks done within this loop is common for all the scenarios and features no matter if
+they are OpenStack or Kubernetes scenarios.
+
+The changes that get Verified+1 from this pipeline is deemed to be good and
+can be merged to master if there is sufficient +2 votes from the XCI and/or project committers.
+
+post-merge
+----------
+
+The changes that are merged to master enter this pipeline and get verified
+against the same criteria as the verify pipeline.
+
+* virtual noha deployment
+* functest healthcheck
+
+The checks done within this loop is common for all the scenarios no matter if
+they are OpenStack or Kubernetes scenarios.
+
+The changes that are successfully verified get promoted for the next loop in
+the pipeline.
+
+Evolving CI Loops and Promotion Criterias
+=========================================
+
+TBD