diff options
Diffstat (limited to 'doc/xtesting')
| -rw-r--r-- | doc/xtesting/ONAP.png | bin | 0 -> 77673 bytes | |||
| -rw-r--r-- | doc/xtesting/index.html | 52 | ||||
| -rw-r--r-- | doc/xtesting/xtesting.md | 135 |
3 files changed, 187 insertions, 0 deletions
diff --git a/doc/xtesting/ONAP.png b/doc/xtesting/ONAP.png Binary files differnew file mode 100644 index 00000000..6543e91c --- /dev/null +++ b/doc/xtesting/ONAP.png diff --git a/doc/xtesting/index.html b/doc/xtesting/index.html new file mode 100644 index 00000000..71379408 --- /dev/null +++ b/doc/xtesting/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Xtesting</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="xtesting.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/doc/xtesting/xtesting.md b/doc/xtesting/xtesting.md new file mode 100644 index 00000000..3bf7f599 --- /dev/null +++ b/doc/xtesting/xtesting.md @@ -0,0 +1,135 @@ +# Xtesting + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2018/03/23 + + + +## Why Xtesting? + + +### Internal needs + +- Functest has to verify Kubernetes deployment but its former Framework was +linked to OpenStack (e.g. credentials sourcing, rally verifiers, etc.) +- hosting both OpenStack and Kubernetes in the same python package would +increase dependencies and then complexify +[container slicing](http://testresults.opnfv.org/functest/dockerslicing/) + +**Why not refactoring the first Functest Framework?** + + +### External needs + +- Functest python and containers framework could be very useful out of OPNFV +(ease developing testcases, manage requirements and offer lighweight docker +images) +- a new Functest design could simplify test integration in a complete +[OPNFV-based CI/CD toolchain](http://docs.opnfv.org/en/stable-euphrates/testing/ecosystem/overview.html) +(e.g. Testing Containers, Test API and dashboard) + +**Let the developer only work on the test suites without diving into CI/CD +integration** + + + +## What's been done for Fraser? + + +### Framework + +- Functest Framework were moved to a new xtesting repository +(functest only hosts OpenStack testcases) +- it has been updated and improved to follow all Xtesting technical +guidelines: + - unlink to **OpenStack** and **OPNFV** + - support both python2 and python3 + - be fully covered by unit tests and well rated by pylint (10/10) + + +### Xtesting deliverables + +- Xtesting is released as [a python package](https://pypi.python.org/pypi/xtesting/) +and then is unlinked to OPNFV Milestones (Functest python package now depends +it) +- [opnfv/xtesting](https://hub.docker.com/r/opnfv/xtesting/) is proposed to +build third-parties containers (both amd64 and arm64 architectures). +- the API documentation is automatically built +[online](http://xtesting.readthedocs.io/en/latest/apidoc/xtesting.html) + + + +## Functest & Xtesting in ONAP OpenLab + + +### first verify the infrastructure via Functest + +``` ++----------------------------+------------------+---------------------+------------------+----------------+ +| TEST CASE | PROJECT | TIER | DURATION | RESULT | ++----------------------------+------------------+---------------------+------------------+----------------+ +| connection_check | functest | healthcheck | 00:07 | PASS | +| api_check | functest | healthcheck | 07:46 | PASS | +| snaps_health_check | functest | healthcheck | 00:36 | PASS | ++----------------------------+------------------+---------------------+------------------+----------------+ +``` +<!-- .element: style="font-size: 0.34em" --> + + +``` ++------------------------------+------------------+---------------+------------------+----------------+ +| TEST CASE | PROJECT | TIER | DURATION | RESULT | ++------------------------------+------------------+---------------+------------------+----------------+ +| vping_ssh | functest | smoke | 00:57 | PASS | +| vping_userdata | functest | smoke | 00:33 | PASS | +| tempest_smoke_serial | functest | smoke | 13:22 | PASS | +| rally_sanity | functest | smoke | 24:07 | PASS | +| refstack_defcore | functest | smoke | 05:21 | PASS | +| patrole | functest | smoke | 04:29 | PASS | +| snaps_smoke | functest | smoke | 46:54 | PASS | +| odl | functest | smoke | 00:00 | SKIP | +| odl_netvirt | functest | smoke | 00:00 | SKIP | +| neutron_trunk | functest | smoke | 00:00 | SKIP | ++------------------------------+------------------+---------------+------------------+----------------+ +``` +<!-- .element: style="font-size: 0.34em" --> + +``` ++----------------------+------------------+--------------+------------------+----------------+ +| TEST CASE | PROJECT | TIER | DURATION | RESULT | ++----------------------+------------------+--------------+------------------+----------------+ +| cloudify_ims | functest | vnf | 28:15 | PASS | +| vyos_vrouter | functest | vnf | 17:59 | PASS | +| juju_epc | functest | vnf | 46:44 | PASS | ++----------------------+------------------+--------------+------------------+----------------+ +``` +<!-- .element: style="font-size: 0.34em" --> + + +### then run ONAP HealthCheck + +- All tests are run by a specialized docker container(**<100 MB**) instead of +the classical ONAP testing virtual machine (**> 1GB**). +- the container mainly inherits from opnfv/xtesting and is completed by: + - python dependencies + - all ONAP robotframework files retrieved from the original repositories + - testcases.yaml describing the testcases + +[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/) + + + +<!-- .element: style="border: 0; width: 70%" --> + + + +## Conclusion + + +## Benefits + +- Xtesting allows a proper design inside OPNFV +- it helps other LFN projects: + - verifying the infrastructure on top of which the components are deployed + - ease verifying the components as well in the same CI/CD toolchain |
