Xtesting

From 051127be025624ea946438242d1936d54763e820 Mon Sep 17 00:00:00 2001 From: Cédric Ollivier Date: Fri, 23 Mar 2018 21:55:22 +0100 Subject: Publish xtesting presentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Change-Id: Ia33a17ed5868bf61f088feea1a608603fdeb28dc Signed-off-by: Cédric Ollivier --- doc/reveal.js | 1 + doc/xtesting/ONAP.png | Bin 0 -> 77673 bytes doc/xtesting/index.html | 52 ++++++++++++++++++ doc/xtesting/xtesting.md | 135 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 188 insertions(+) create mode 160000 doc/reveal.js create mode 100644 doc/xtesting/ONAP.png create mode 100644 doc/xtesting/index.html create mode 100644 doc/xtesting/xtesting.md (limited to 'doc') diff --git a/doc/reveal.js b/doc/reveal.js new file mode 160000 index 00000000..a2e69a4b --- /dev/null +++ b/doc/reveal.js @@ -0,0 +1 @@ +Subproject commit a2e69a4b42f9e968406f62073d1c4bf0ea2d3361 diff --git a/doc/xtesting/ONAP.png b/doc/xtesting/ONAP.png new file mode 100644 index 00000000..6543e91c Binary files /dev/null and b/doc/xtesting/ONAP.png differ diff --git a/doc/xtesting/index.html b/doc/xtesting/index.html new file mode 100644 index 00000000..71379408 --- /dev/null +++ b/doc/xtesting/index.html @@ -0,0 +1,52 @@ + + +Xtesting + + + + + + + + +

+ + + + + diff --git a/doc/xtesting/xtesting.md b/doc/xtesting/xtesting.md new file mode 100644 index 00000000..3bf7f599 --- /dev/null +++ b/doc/xtesting/xtesting.md @@ -0,0 +1,135 @@ +# Xtesting + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2018/03/23 + + + +## Why Xtesting? + + +### Internal needs + +- Functest has to verify Kubernetes deployment but its former Framework was +linked to OpenStack (e.g. credentials sourcing, rally verifiers, etc.) +- hosting both OpenStack and Kubernetes in the same python package would +increase dependencies and then complexify +[container slicing](http://testresults.opnfv.org/functest/dockerslicing/) + +**Why not refactoring the first Functest Framework?** + + +### External needs + +- Functest python and containers framework could be very useful out of OPNFV +(ease developing testcases, manage requirements and offer lighweight docker +images) +- a new Functest design could simplify test integration in a complete +[OPNFV-based CI/CD toolchain](http://docs.opnfv.org/en/stable-euphrates/testing/ecosystem/overview.html) +(e.g. Testing Containers, Test API and dashboard) + +**Let the developer only work on the test suites without diving into CI/CD +integration** + + + +## What's been done for Fraser? + + +### Framework + +- Functest Framework were moved to a new xtesting repository +(functest only hosts OpenStack testcases) +- it has been updated and improved to follow all Xtesting technical +guidelines: + - unlink to **OpenStack** and **OPNFV** + - support both python2 and python3 + - be fully covered by unit tests and well rated by pylint (10/10) + + +### Xtesting deliverables + +- Xtesting is released as [a python package](https://pypi.python.org/pypi/xtesting/) +and then is unlinked to OPNFV Milestones (Functest python package now depends +it) +- [opnfv/xtesting](https://hub.docker.com/r/opnfv/xtesting/) is proposed to +build third-parties containers (both amd64 and arm64 architectures). +- the API documentation is automatically built +[online](http://xtesting.readthedocs.io/en/latest/apidoc/xtesting.html) + + + +## Functest & Xtesting in ONAP OpenLab + + +### first verify the infrastructure via Functest + +``` ++----------------------------+------------------+---------------------+------------------+----------------+ +| TEST CASE | PROJECT | TIER | DURATION | RESULT | ++----------------------------+------------------+---------------------+------------------+----------------+ +| connection_check | functest | healthcheck | 00:07 | PASS | +| api_check | functest | healthcheck | 07:46 | PASS | +| snaps_health_check | functest | healthcheck | 00:36 | PASS | ++----------------------------+------------------+---------------------+------------------+----------------+ +``` + + + +``` ++------------------------------+------------------+---------------+------------------+----------------+ +| TEST CASE | PROJECT | TIER | DURATION | RESULT | ++------------------------------+------------------+---------------+------------------+----------------+ +| vping_ssh | functest | smoke | 00:57 | PASS | +| vping_userdata | functest | smoke | 00:33 | PASS | +| tempest_smoke_serial | functest | smoke | 13:22 | PASS | +| rally_sanity | functest | smoke | 24:07 | PASS | +| refstack_defcore | functest | smoke | 05:21 | PASS | +| patrole | functest | smoke | 04:29 | PASS | +| snaps_smoke | functest | smoke | 46:54 | PASS | +| odl | functest | smoke | 00:00 | SKIP | +| odl_netvirt | functest | smoke | 00:00 | SKIP | +| neutron_trunk | functest | smoke | 00:00 | SKIP | ++------------------------------+------------------+---------------+------------------+----------------+ +``` + + +``` ++----------------------+------------------+--------------+------------------+----------------+ +| TEST CASE | PROJECT | TIER | DURATION | RESULT | ++----------------------+------------------+--------------+------------------+----------------+ +| cloudify_ims | functest | vnf | 28:15 | PASS | +| vyos_vrouter | functest | vnf | 17:59 | PASS | +| juju_epc | functest | vnf | 46:44 | PASS | ++----------------------+------------------+--------------+------------------+----------------+ +``` + + + +### then run ONAP HealthCheck + +- All tests are run by a specialized docker container(**<100 MB**) instead of +the classical ONAP testing virtual machine (**> 1GB**). +- the container mainly inherits from opnfv/xtesting and is completed by: + - python dependencies + - all ONAP robotframework files retrieved from the original repositories + - testcases.yaml describing the testcases + +[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/) + + +![ONAP](ONAP.png) + + + + +## Conclusion + + +## Benefits + +- Xtesting allows a proper design inside OPNFV +- it helps other LFN projects: + - verifying the infrastructure on top of which the components are deployed + - ease verifying the components as well in the same CI/CD toolchain -- cgit 1.2.3-korg