diff options
Diffstat (limited to 'docs/com')
34 files changed, 1663 insertions, 29 deletions
diff --git a/docs/com/css/theme/OPNFV-Berlin.css b/docs/com/css/theme/OPNFV-Berlin.css index 9f957d6f1..34f73b4bd 100644 --- a/docs/com/css/theme/OPNFV-Berlin.css +++ b/docs/com/css/theme/OPNFV-Berlin.css @@ -31,13 +31,13 @@ body { bottom: 1em; left: 1em; font-size: 0.5em; - + } .reveal .slides > section, .reveal .slides > section > section { line-height: 1.3; font-weight: inherit; } - + .reveal .opnfv-title { background-image: url("../../img/title-bg-berlin.png" no-repeat); } @@ -87,7 +87,7 @@ body { .reveal h1 { text-shadow: none; } - + /********************************************* * OTHER diff --git a/docs/com/css/theme/OPNFV.css b/docs/com/css/theme/OPNFV.css index 745a2a8de..c59c59f8d 100644 --- a/docs/com/css/theme/OPNFV.css +++ b/docs/com/css/theme/OPNFV.css @@ -30,13 +30,13 @@ body { bottom: 1em; left: 1em; font-size: 0.5em; - + } .reveal .slides > section, .reveal .slides > section > section { line-height: 1.3; font-weight: inherit; } - + .reveal .opnfv-title { background-image: url("../../img/title-bg.png" no-repeat); } @@ -83,7 +83,7 @@ body { .reveal h1 { text-shadow: none; } - + /********************************************* * OTHER diff --git a/docs/com/pres/Summit/Berlin-2016/conversation.html b/docs/com/pres/Summit/Berlin-2016/conversation.html index 356c2ad1e..5c1e18450 100644 --- a/docs/com/pres/Summit/Berlin-2016/conversation.html +++ b/docs/com/pres/Summit/Berlin-2016/conversation.html @@ -215,7 +215,7 @@ </div> <div class='footer'> - <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> + <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> </div> </div> diff --git a/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html b/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html index 97fa66c18..2b3b3b4ef 100644 --- a/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html +++ b/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html @@ -81,7 +81,7 @@ </section> <section data-markdown> ## so concretely... - * Validate scenarios for the release + * Validate scenarios for the release * Give confidence on OPNFV fresh releases * Ensure consistancy towards installers * Ensure End to End interoperability @@ -135,7 +135,7 @@ * 2 installers: Foreman/Fuel * Lots of manual operations (reporting, documentation) </section> - </section> + </section> <section> <section data-markdown> # Brahmaputra @@ -146,8 +146,8 @@ * 3 feature projects: doctor, promise, sdnvpn * 13 Scenarios * 4 Installers (Apex, Compass, Fuel, Joid) - * Test result collection - * Automatic dashboard + * Test result collection + * Automatic dashboard </section> <section> <h3>Jiras</h3> @@ -196,7 +196,7 @@ </tr> </tbody> </table> - * difference of duration due to POD and test suites + * difference of duration due to POD and test suites </section> <section data-markdown> @@ -204,7 +204,7 @@ * Tempest/Rally * harmonizing installer related OpenStack configuration differences * creating customized test lists - * main challenge was resolving SDN controller interworking problems + * main challenge was resolving SDN controller interworking problems * vIMS * complete but complex test case * very interesting to automate (pre MANO, most of Telco needs met there) @@ -212,7 +212,7 @@ </section> - <section> + <section> <section data-markdown> # Colorado </section> @@ -221,7 +221,7 @@ ## What's new? * New internal test cases: healthcheck, security * New controler: OpenContrail? - * New feature projects (domino, multisites, movie, parser, moon, copper, models, onos-sfc...) + * New feature projects (domino, multisites, movie, parser, moon, copper, models, onos-sfc...) * ARM Support (congratulations to ENEA team!) * Simplified feature project integration journey </section> @@ -255,19 +255,19 @@ * New VNFs (vEPC, vCDN, vWhatever...) * Multi sites (e.g bgpvpn with different back ends) * Better test coverage - </section> + </section> <section data-markdown> ## A Functional testing As a Service framework * Scenario owner can select their relevant test cases * Test duration estimation * Agile dashboarding * Analytics - </section> + </section> <section data-markdown> ## Upstream * Rally: time to give back more - * Functest description for ETSI? - </section> + * Functest description for ETSI? + </section> </section> <section> diff --git a/docs/com/pres/Summit/Berlin-2016/testapi.html b/docs/com/pres/Summit/Berlin-2016/testapi.html index c40637cbf..5e56d5052 100644 --- a/docs/com/pres/Summit/Berlin-2016/testapi.html +++ b/docs/com/pres/Summit/Berlin-2016/testapi.html @@ -69,7 +69,7 @@ * Test cases * Results </section> - + <section data-markdown> ## Status Currently * From many projects @@ -87,7 +87,7 @@ </aside> </script> </section> - + <section data-markdown> ## Help building * Dashboards @@ -135,8 +135,8 @@ <li>Scripts to support databse backup/restore/update</li> <li>Support installation</li> <aside class='notes'> - So you can discover and understand the capabilities of the service without - access to source code, documentation, or through network traffic inspection, + So you can discover and understand the capabilities of the service without + access to source code, documentation, or through network traffic inspection, and also you can interact with the TestAPI directly through swagger website. </aside> </ul> @@ -148,12 +148,12 @@ <section data-markdown> # API Evoluation </section> - + <section> <h2> URI changes...</h2> - <div style="text-align:left""> + <div style="text-align:left""> <p> testresults.opnfv.org/<span style="color:lightblue">TestAPI</span> => <br>testresults.opnfv.org/<span style="color:yellow">test/api/v1</span> </p> - + <p> /test/api/v1/<b>pods</b></p> <p> /test/api/v1/<b>projects</b></p> <p> /test/api/v1/projects/qtip/<b>cases</b></p> @@ -200,7 +200,7 @@ <section> <h2>unit tests</h2> <pre><code class="hljs" data-trim contenteditable> -umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh +umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh Tests running... WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms diff --git a/docs/com/pres/compliance/compliance.md b/docs/com/pres/compliance/compliance.md new file mode 100644 index 000000000..cbe0dbe17 --- /dev/null +++ b/docs/com/pres/compliance/compliance.md @@ -0,0 +1,97 @@ +# From Verification to CNTT Compliance + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2019/12/04 + + + +## Infrastructure Verification + + +### Functest in a nutshell + +- verify any kind of OpenStack and Kubernetes deployments (OPNFV model) + including production environments +- conform with upstream rules (OpenStack gate jobs and Kubernetes conformance + tests) +- ensure that the platforms meet Network Functions Virtualization requirements + + +### Functest suites + +- all functional tests (3000+) as defined by the upstream communities + (e.g. Tempest, neutron-tempest-api, Barbican, Patrole...) +- 3 hours upstream API and dataplane benchmarking tests (Rally, VMTP and + Shaker) +- Virtual Network Function deployments and testing (vIMS, vRouter and vEPC) + + + +## CNTT Compliance + + +### Verification vs Compliance + +- verification allows skipping test if optional services (Gnocchi, Barbican, + etc.) or capabilities (remote console access, Neutron BGPVPN or SFC, etc.) + are missing +- compliance forces here the full API descriptions as currently proposed by + CNTT (please see + [Interfaces and APIs](https://github.com/cntt-n/CNTT/blob/master/doc/ref_arch/openstack/chapters/chapter05.md)) +- then the testcase descriptions should forbid skipping any test and cover only + the mandatory services (and their mandatory capabilities) + + +### Changelog + +- all the logics were already in Functest and the underlying frameworks (Rally, + Tempest, etc.) +- 2 new CNTT-related containers including the new testcase descriptions were + published to easily verify the compliance +- all 3 Functest SUTs (Rocky, Stein/Train and Master) are now compliant with + CNTT API to ensure the continuous integration +- the benchmarking testcases doesn't validate any KPI as nothing is written in + CNTT documentation + + +### RI verification and Compliance + +- CNTT Reference Implementation 1 is already continuously verified + ([continuous integration model](https://build.opnfv.org/ci/view/cntt/job/cntt-latest-daily/)) +- the conformance is currently failing due to a few bugs in deployments and + missing features + +**Be free to [deploy your own CNTT Compliance CI/CD toolchain](https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004) + in a few commands** + + + +## Conclusion + + +### Next steps + +- fix reference implementation 1 deployments and then achieve the compliance +- port existing OPNFV testcases to Xtesting and then add them in the continuous + integration loop +- update the testcase descritions according to the CNTT progress (KPI, API + changes) + + +### Takeaways + +- Functest allows verifying any production Infrastructure and now checking the + CNTT API Compliance +- all containers can be already consumed +- any third-party certification should reuse the Functest CNTT-related + containers as they are +- [CNTT RI continuous integration](https://build.opnfv.org/ci/view/cntt/job/cntt-latest-daily/) + is in place and any testcase can be smoothly added if they leverage on + Xtesting + +**Try it, and you will love it!** + + + +## Thank you diff --git a/docs/com/pres/compliance/index.html b/docs/com/pres/compliance/index.html new file mode 100644 index 000000000..94fc121d6 --- /dev/null +++ b/docs/com/pres/compliance/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>From Verification to Compliance</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="compliance.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/dockerslicing/dockerslicing.md b/docs/com/pres/dockerslicing/dockerslicing.md index a66453248..0d1ce5e21 100644 --- a/docs/com/pres/dockerslicing/dockerslicing.md +++ b/docs/com/pres/dockerslicing/dockerslicing.md @@ -63,7 +63,7 @@ ### 8 Functest containers -```bash +```shell $ sudo docker search opnfv |grep functest- opnfv/functest-core OPNFV Functest core image opnfv/functest-restapi OPNFV Functest restapi image diff --git a/docs/com/pres/euphrates_functest_evolution/euphrates.md b/docs/com/pres/euphrates_functest_evolution/euphrates.md index 49aab16cd..2c3105539 100644 --- a/docs/com/pres/euphrates_functest_evolution/euphrates.md +++ b/docs/com/pres/euphrates_functest_evolution/euphrates.md @@ -120,7 +120,7 @@ export OS_IDENTITY_API_VERSION=3 #### Retrieve all the images ``` -mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* +mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | sh -s -- images && ls -1 images/* images/CentOS-7-aarch64-GenericCloud.qcow2 images/CentOS-7-aarch64-GenericCloud.qcow2.xz images/CentOS-7-x86_64-GenericCloud.qcow2 diff --git a/docs/com/pres/functest2019/functest2019.md b/docs/com/pres/functest2019/functest2019.md new file mode 100644 index 000000000..48f01cb0c --- /dev/null +++ b/docs/com/pres/functest2019/functest2019.md @@ -0,0 +1,105 @@ +# Functest 2019 + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2019/01/08 + + + +## Gates + + +### Today's Verify -1 + +- one error is detected by yamllint, pep8, pylint, ansible-lint, bashate or + doc8 +- one unit test fails (py27 and py35) +- specific modules are not rated 10/10 by pylint +- our Framework is not fully covered +- unix permissions are wrong +- one doc cannot be generated without warning + +Please see [tox.ini](https://git.opnfv.org/functest/tree/tox.ini) for details + + +### Great! But + +- the current gate checks all coding rules, our Framework and all interfaces +to third-parties but **not our testcases** +- it doesn't break **the circular dependencies** between Installers and +Functest (Installers ask for a trustable healthcheck but we need results of +"Installer runs" as prerequisites) + +Our testcases have still **to be verified by hand** what differs from +OpenStack Workflow. + + +### Functional gating + +- we need to run all Functest patches before merge against reference +plateforms (OpenStack and Kubernetes) +- a voting job must forbid the merge if one test fails (as the workflow +process implemented by OpenStack) + + +### Improve the current daily model + +- all Functest jobs are linked to the OPNFV installers and can hardly be + reused by endusers +- all test suites are run sequentially +- rally_full is excluded due to its duration (~3 hours) +- all possible remaining resources are cleant when installing the scenarios + +**It's fine for gating installers but not for verifying that Functest supports parallel and live testing** + + +### New Xtesting Ansible role + +- deploy anywhere the full OPNFV CI/CD toolchain in few commands +- easily add external bots voting in reviews +- produce all Xtesting-based fonctional jobs (Xtesting, Functest and Functest + Kubernetes) in Releng +- could be instanciated for testing services out of the infrastructure domain + +**It's already in a [good shape](https://lists.opnfv.org/g/opnfv-tech-discuss/message/22552). [Try it!](https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004)** + + + +## Quality Assurance + +**Functest is matching the OpenStack Quality Golden rules** + + +### Pylint and coverage (started from E release) +- only need to refactor several vnf testcases and to remove duplicated code +- document well and cover Functest utils: they are reused by +OPNFV third-parties (SFC, SDNVPN) + +** Functest will be rated 10/10 very soon** + + +### releasing + +- Functest could be also released as a classical Python packages like Xtesting( +[PyPI](https://pypi.python.org/pypi)) +- Xtesting and Functest could be released as GNU/Linux distribution packages + + + +## other challenges + + +### proposals + +- integrate Rally and Tempest as core modules in Functest (we do merge tempest.py and + conf_utils.py) +- deploy cloud-native VNF (Clearwater IMS?) +- why not leveraging on OpenStack middlewares such as + [oslo.config](https://docs.openstack.org/oslo.config/latest/)? +- promote our VNF testcases in the upstream communities + +**Any new testcase is more than welcome!** + + + +## Thank you diff --git a/docs/com/pres/functest2019/index.html b/docs/com/pres/functest2019/index.html new file mode 100644 index 000000000..33dc9a9f2 --- /dev/null +++ b/docs/com/pres/functest2019/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Functest 2019</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="functest2019.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/functest2020/functest2020.md b/docs/com/pres/functest2020/functest2020.md new file mode 100644 index 000000000..a81667939 --- /dev/null +++ b/docs/com/pres/functest2020/functest2020.md @@ -0,0 +1,61 @@ +# Functest 2020 + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/01/16 + + + +## What's new in Jerma? + + +### Better test case verification + +- switch to Ceph in our Functest SUTs +- add Neutron features in our SUTs to improve the case verification +- test all capabilities possible (e.g. vnc_console) +- verify automatically the CNTT-related containers into additional to the + classical ones +- harden xrally_kubernetes + +**Functest SUTs are compliant to CNTT RC** + + +### New test cases + +- tempest_horizon +- tempest_keystone +- tempest_cinder +- refstack_platform +- refstack_object +- octavia +- xrally_kubernetes + + +### New usage + +- **support CNTT RC (API testing, API and dataplane benchmarking, VNF onboarding and testing)** +- verify ONAP WindRiver OpenLab via Functest CI in a VM ("Inception model") +- allow minimal l2-only testing via Rally + +**still pushing the limit!** + + + +## Kali (K-release) + + +### Ideas: + +- finish updating to Alpine 3.11 and Python 3.8 +- finish KloudBuster integration (it has to be updated to Python3 first) +- add tempest-stress +- update and possibly add heat-tempest-plugin +- add cyborg-tempest-plugin? ironic-tempest-plugin ? xxx-tempest-plugin? +- add CNF into Kubernetes testing + +**continuously hardening the gates and improving code quality** + + + +## Thank you diff --git a/docs/com/pres/functest2020/index.html b/docs/com/pres/functest2020/index.html new file mode 100644 index 000000000..88195becb --- /dev/null +++ b/docs/com/pres/functest2020/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Functest 2020</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="functest2020.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/gambia/gambia.md b/docs/com/pres/gambia/gambia.md new file mode 100644 index 000000000..4a65e9905 --- /dev/null +++ b/docs/com/pres/gambia/gambia.md @@ -0,0 +1,178 @@ +# Functest on steroids + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2019/01/08 + + + +## OPNFV + + +<!-- .element: style="border: 0; width: 90%" --> + + + +## Verify OpenStack and Kubernetes + + +### Functest in a nutshell + +- verify any kind of OpenStack and Kubernetes deployments +- conform with upstream rules (OpenStack gate jobs and Kubernetes conformance + tests) +- ensure that the platforms meet Network Functions Virtualization requirements + + +### Functest suites + +- as many upstream functional tests as possible (e.g. Tempest, + neutron-tempest-api, Barbican, Patrole...) +- upstream API and dataplane benchmarking tools (Rally, Vmtp and Shaker) +- additional VNF deployments and testing (vIMS, vRouter and vEPC) + +**Dovetail only runs few Functest functional tests and then we have to verify +any OPNFV-certified scenarios via Functest anyway** + + +### What's new in Functest? + +- new testcases were quickly integrated in Functest: Patrole, Barbican, Shaker, + ... +- all testcases can run in parallel to decrease the overall duration +- the resources cleaning has been improved +- **our testcases may be run vs VIM in production** +- **it includes most of the OpenStack gate jobs** + + +### Support of OS and K8s master + +| Functest | OpenStack | Kubernetes | +| :------: | :---------: | :--------: | +| master | master | master | +| hunter | rocky | v1.11.3 | +| gambia | queens | v1.11.3 | + + +### < 50 Euros + + +<!-- .element: style="border: 0; width: 70%" --> + + + +## Reuse of OPNFV + + +### Xtesting in a nutshell + +- allow the developer to work only on the test suites without diving into CI/CD + integration +- simplify test integration in a complete LFN-based CI/CD toolchain (e.g. + Jenkins, Testing Containers, Test API and dashboard) +- allow a proper design and verify multiple components in the same CI/CD + toolchain (OpenStack, Kubernetes, ONAP, etc.) + +**Easy to use and very useful for any CI/CD toochain (unlinked to Infrastrure)** + + +### A user story ONAP + +- all tests are run by a specialized Docker container(**<100 MB**) instead of +the classical ONAP testing virtual machine (**> 1GB**). +- the container mainly inherits from opnfv/xtesting and is completed by: + - Python dependencies + - all ONAP Robot Framework files retrieved from the original repositories + - testcases.yaml describing the testcases + +[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/) + + +### What's new in Xtesting? + +- new ansible roles and playbooks have been developed to allow **deploying your + full CI/CD toolchains in few minutes** (Jenkins, Minio, TestAPI, MongoDB and + Docker registry) + +```shell +virtualenv xtesting +. xtesting/bin/activate +pip install ansible docker +ansible-galaxy install collivier.xtesting +git clone https://gerrit.opnfv.org/gerrit/functest-xtesting functest-xtesting-src +ansible-playbook functest-xtesting-src/ansible/site.yml +deactivate +``` + +**They are already reused in Functest and by Orange out of the +Infrastrure domain** + + + +## Collect results + + +### OPNFV Test Database in a nutshell + +- it's a fair comparison of Neutron implementations (Agents vs SDN + controller) +- it stores all verification results and all performance data from different + hardware over the world which could be easily postprocessed +- it could be very useful to select the adequate opensource solutions + regarding metrics and capabilities + + +### which Neutron backend? + +- most Neutron standalone and OVN scenarios pass Functest decently +- no ODL scenarios pass the advanced testcases (benchmarking tools and + VNFs). It's still unclear if it's due to the Installers, ODL or POD + misconfigurations. +- no Tungsten Fabric is released in Gambia + +**We expected that ODL results would have improved before the first Gambia + corrective** + + +### Contrail testing (out of OPNFV) + +- **3.X 4.X**: mostly verified except some functional tests about + visibility which fail due to the falsy admin role (they can be easily + blacklisted) +- **5.X**: a limited set of bugs in Contrail mostly forbid running few + functional tests and benchmarking tools: + - wrong external network listing + - Contrail doesn't allow booting a VM without network (and elects the wrong + network) + + +### And performance? + +- OVS DPDK is not fully integrated by an OPNFV installer (Fuel is in a good + shape to support it) +- the only scenario including VPP is not part of Gambia + +**From the time being, we can't evaluate the benefits of OVS DPDK or VPP thanks + to OPNFV** + + + +## Conclusion + + +### Gambia + +- Functest and Xtesting are powerful and easy to (re)use (containers, jenkins + jobs, ansible playbooks, Raspberry PI, etc.). +- the number of installers and scenarios decreased in Gambia (it's still + unclear regarding the overall quality). **What about OPNFV Test Database if + it decreases again in 2019?** + + +### And beyond + +- test frameworks are now considered as crucial for OPNFV (see [Last OPNFV Marketing update](https://wiki.opnfv.org/download/attachments/2925933/OPNFV%20Marketing%20Update%20091818.pptx?version=1&modificationDate=1537228648000&api=v2)) and Functest and Xtesting could be +already widely reused out of OPNFV +- the new test-driven approach as proposed by the [OPNFV Strategic Plan](https://wiki.opnfv.org/download/attachments/2925933/OPNFV%20Strategy%20and%20Plan%20v0.5.pptx?version=1&modificationDate=1540961098000&api=v2) could increase **the quality of all +scenarios**. But we are also suggesting to let the installers decide their +test cases. diff --git a/docs/com/pres/gambia/index.html b/docs/com/pres/gambia/index.html new file mode 100644 index 000000000..db3a6aaa2 --- /dev/null +++ b/docs/com/pres/gambia/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Functest on steroids</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="gambia.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/gambia/raspberrypi.jpg b/docs/com/pres/gambia/raspberrypi.jpg Binary files differnew file mode 100644 index 000000000..60a2638f3 --- /dev/null +++ b/docs/com/pres/gambia/raspberrypi.jpg diff --git a/docs/com/pres/gambiachallenges/gambia.md b/docs/com/pres/gambiachallenges/gambia.md new file mode 100644 index 000000000..078ec61ce --- /dev/null +++ b/docs/com/pres/gambiachallenges/gambia.md @@ -0,0 +1,79 @@ +# Functest Gambia, new challenges + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2018/03/22 + + + +## Gating + + +### Our gating is very simple + +- today all test suites are run sequentially +- all resources are not necessarily cleant until the next OpenStack deployment + +**It's fine for gating installers but not for verifying a production +environment (e.g. multi users running the same testcase)** + + +### Parallel and Live testing + +- all resources should be allocated in their own OpenStack projects (already + mostly the case) +- no testcase should create singletons on resource attributes (name, ip, etc.) +- all resources have to be cleant at the end + +**According to [Orange testing results](https://wiki.opnfv.org/pages/viewpage.action?pageId=13211751), +Functest is already in a good shape to meet this goal** + + + +## Quality Assurance + +**Gambia release will match the OpenStack Quality Golden rules** + + +### Pylint and coverage (started from E release) +- need to refactor several vnf testcases and to remove duplicated code +- document well and cover Functest utils: they are reused by +OPNFV third-parties (SFC, SDNVPN) + +** Functest will be rated 10/10** + + +### releasing + +- Functest could be also released as a classical Python package ( +[PyPI](https://pypi.python.org/pypi)) +- its API docs could be published in [Read the Docs](https://readthedocs.org/) + + +### Current technical debt +- former python modules (e.g. cli) or containers (e.g. restapi) have to be +removed +- functest shouldn't contain any OPNFV Installer logic. **All Jenkins Jobs +must set the right env vars as all endusers** +- vnf descriptors should be hosted by Functest instead of using external github +repositories + + + +## other challenges + + +### proposals + +- ease integrating tempest plugins with a minimum effort (new tempest driver in +Functest Framework and new opnfv/functest-tempest container) +- add opnfv/functest-components in verify jobs +- why not leveraging on OpenStack middlewares such as + [oslo.config](https://docs.openstack.org/oslo.config/latest/) or + [stevedore](https://docs.openstack.org/stevedore/latest/)? + +**Any new testcase is more than welcome!** + + + +## Thank you diff --git a/docs/com/pres/gambiachallenges/index.html b/docs/com/pres/gambiachallenges/index.html new file mode 100644 index 000000000..592ccf2f5 --- /dev/null +++ b/docs/com/pres/gambiachallenges/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Gambia challenges</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="gambia.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/gates/gates.md b/docs/com/pres/gates/gates.md new file mode 100644 index 000000000..aaf087743 --- /dev/null +++ b/docs/com/pres/gates/gates.md @@ -0,0 +1,82 @@ +# Functest Gates + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2018/04/19 + + + +### Why gating? + +- maintain an overall high quality code whatever the skills involved in our +open community +- detect the errors as soon as possible (before merge operations) +- verify automatically most of Functest milestones + + +### Today's Verify -1 + +- one error is detected by yamllint, pylint or doc8 +- one unit test fails (py27 and py35) +- our Framework is not fully covered +- specific modules are not rated 10/10 by pylint +- unix permissions are wrong +- one doc cannot be generated without warning + +Please see [tox.ini](https://git.opnfv.org/functest/tree/tox.ini) for details + + +### Gambia's Verify -1 + +- enforce an overall 10/10 by pylint +- check the full coverage of all third-party interfaces +- integrate other linters like ShellScript (if they meet tox basics) + +Please see +[Gambia Challenges](http://testresults.opnfv.org/functest/gambiachallenges/) +for details + + +### Great! But + +- the current gate checks all coding rules, our Framework and all interfaces +to third-parties but **not our testcases** +- it doesn't break **the circular dependencies** between Installers and +Functest (Installers ask for a trustable healthcheck but we need results of +"Installer runs" as prerequisites) + +Our testcases have still **to be verified by hand** what differs from +OpenStack Workflow + + + +### functional gating + +- we need to run all Functest patches before merge against reference +plateforms (OpenStack and Kubernetes) +- a voting job must forbid the merge if one test fails (as the workflow +process implemented by OpenStack) + + +### reference platform + +- it could be baremetal or virtual if all testcases (including all VNFs) can +be tested successfully +- any compliant Installer could be selected as long as the appropriate target +VIM version is proposed at the beginning of the release: + - devstack (OpenStack gates) + - kolla-ansible (Orange ONAP OpenLab) + - XCI + - APEX + + +### Verify +2 + +Only the functional check will take hours once the patchset is accepted +by core reviewers (each patchset will be verified in ~10 minutes). + +**Always stable!** + + + +## Thank you! diff --git a/docs/com/pres/gates/index.html b/docs/com/pres/gates/index.html new file mode 100644 index 000000000..b64e371f7 --- /dev/null +++ b/docs/com/pres/gates/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Functest Gates</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="gates.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/oran/ftth.png b/docs/com/pres/oran/ftth.png Binary files differnew file mode 100644 index 000000000..5b441d1de --- /dev/null +++ b/docs/com/pres/oran/ftth.png diff --git a/docs/com/pres/oran/oran.md b/docs/com/pres/oran/oran.md new file mode 100644 index 000000000..87fba1306 --- /dev/null +++ b/docs/com/pres/oran/oran.md @@ -0,0 +1,90 @@ +--- +author: Cédric Ollivier +title: OPNFV/CNTT CI assets +date: 2020/12/01 +--- +## Continuous integration + + +### A few CNTT reqs + +- OPNFV has built a complete CI/CD toolchain for continuously deploying and + testing cloud infrastructure +- Reference conformance only requires for the local deployment of the + same components +- all test cases must be delivered as **Docker containers** including the + common test case execution and the unified way to manage all the interactions + +**Integrate smoothly and deploy fastly** + + +### How OPNFV/CNTT helps? + +- **Functest** offers a collection of state-of-the-art virtual infrastructure + test suites +- **Xtesting** helps assembling sparse test cases and accelerating the adoption + of CI/CD best practices +- **XtestingCI** eases deploying anywhere plug-and-play CI/CD toolchains in a + few commands +- **CNTT** defines conformance suites and its playbooks leveraging this 3 + testing frameworks + +**Leverage best opensource tools and practices** + + +### Testing [FTTH](https://en.wikipedia.org/wiki/Fiber_to_the_x) equipment + + + + +### Deploy your CI toolchain + +``` +virtualenv xtesting +. xtesting/bin/activate +pip install ansible +ansible-galaxy install collivier.xtesting +git clone https://gerrit.opnfv.org/gerrit/functest-xtesting functest-xtesting-src +ansible-playbook functest-xtesting-src/ansible/site.yml +deactivate +rm -rf functest-xtesting-src xtesting +``` + +**More scenarios in [[1]](https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004)** + + + +## Testing + + +### Verif', Compliance and Interop' + +- verification allows skipping test if optional services or capabilities are + missing +- compliance forces the full API descriptions as currently proposed by + CNTT and then skips all optional capability testing +- the opensource certifications are driven by simple interoperability testing + (RefStack and OVP are about less than 10% of Functest. **trustability?**) + +**Fine tune the same test frameworks** + + +### [CNTT RC1](https://build.opnfv.org/ci/view/functest/job/functest-leguer-daily/12/) + + + + + +## Conclusion + + +### Key takeaways + +- **test integration requirements** are crucial to smoothly assemble all + test cases +- leverage **existing** OPNFV testing knowledge (projects) and experience + (history) by utilising the OPNFV toolchain design already in-place +- **3000+** functional tests, **3 hours** upstream API and dataplane benchmarks + and VNFs automatically onboarded and tested **can be reused asis** + +**Keep it simple and do not reivent the wheel** diff --git a/docs/com/pres/oran/rc1.png b/docs/com/pres/oran/rc1.png Binary files differnew file mode 100644 index 000000000..9179f3bb6 --- /dev/null +++ b/docs/com/pres/oran/rc1.png diff --git a/docs/com/pres/reveal.js b/docs/com/pres/reveal.js -Subproject a349ff43c58c23f9c837b8ea9b5fc7d4761b8de +Subproject 2c5396b7d347f8ee1344016f15b93d4f7840156 diff --git a/docs/com/pres/rfp/index.html b/docs/com/pres/rfp/index.html new file mode 100644 index 000000000..538c0c8c8 --- /dev/null +++ b/docs/com/pres/rfp/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>OPNFV and CNTT in Orange RFP</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="rfp.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/rfp/rfp.md b/docs/com/pres/rfp/rfp.md new file mode 100644 index 000000000..66f7ba9e7 --- /dev/null +++ b/docs/com/pres/rfp/rfp.md @@ -0,0 +1,97 @@ +# OPNFV and CNTT in Orange RFP + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/09/19 + + + +## Vision and contributions + + +### Our guidelines + +- **automate** to bring determinism and to meet the new + software release rate +- test all software layers **independently** (OpenStack, Kubernetes and VNFs) +- run all deployment and verification jobs in **our** continous integration + chains +- leverage best **opensource** tools and practices + +**Integrate smoothly and deploy everywhere fastly** + + +### How OpenSource helps? + +- **Functest** offers a collection of state-of-the-art virtual infrastructure + test suites +- **Xtesting** helps assembling sparse test cases and accelerating the adoption + of CI/CD best practices +- **XtestingCI** eases deploying anywhere plug-and-play CI/CD toolchains in a + few commands +- **CNTT** defines conformance suites and its playbooks leveraging this 3 + softwares + +**Any contribution is more than welcome! +[[1]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + + +## CNTT/OPNFV in Orange + + +### A couple of RFP requirements + +- the **full** CNTT reference conformance for OpenStack results and outputs + (Orange CNTT Field Trial is in a very good shape + [[1]](http://testresults.opnfv.org/functest/field_trial/) + [[2]](https://www.linkedin.com/pulse/cntt-field-trials-c%C3%A9dric-ollivier/)) +- the **success** of the Functest Kubernetes test suites (now released as + part of CNTT RC2 Baraque) +- **first** VNF test cases running in **our** continuous integration chain + thanks to Xtesting and XtestingCI + +**It's implementing Orange and CNTT targets** + + +### Orange CNTT RC1 Field Trial + +- helped detecting a couple of issues in CNTT RC1 +- integrated cinder backup and nova instance_password in Orange IaaS +- to fix 10 remaining single test failures (out 2000+ functional tests, 3 hours + benchmarking and 3 VNFs automatically onboarded and tested) +- to enhance Functest juju_epc to pass proxies + +**99,999%** + + +### Wish list + +- to integrate **more benchmarks** in CNTT conformance (e.g. disk benchmarking) +- to switch from the current Kubernetes interoperability testing to a **true** + CNTT conformance suite +- to build the first **VNF and CNF** conformance suites (**high priority**) + +**We need your contribution helps! +[[1]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + + +## Conclusion + + +### Take aways + +- Orange leverages OPNFV and CNTT in RFP +- we keep contributing in both specification and implementation streams for + the success of Network Function Virtualization +- we expect more OPNFV and CNTT contributions especially for VNF and CNF + conformance suites, our initial CNTT target + +**Try CNTT reference suites, you will love them!** + + + +## Thank you! diff --git a/docs/com/pres/stockholm/chainedci.png b/docs/com/pres/stockholm/chainedci.png Binary files differnew file mode 100644 index 000000000..df594cd3a --- /dev/null +++ b/docs/com/pres/stockholm/chainedci.png diff --git a/docs/com/pres/stockholm/ftth.png b/docs/com/pres/stockholm/ftth.png Binary files differnew file mode 100644 index 000000000..5b441d1de --- /dev/null +++ b/docs/com/pres/stockholm/ftth.png diff --git a/docs/com/pres/stockholm/index.html b/docs/com/pres/stockholm/index.html new file mode 100644 index 000000000..65fa37a3a --- /dev/null +++ b/docs/com/pres/stockholm/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>From Paris to Stockholm</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="stockholm.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/stockholm/stockholm.md b/docs/com/pres/stockholm/stockholm.md new file mode 100644 index 000000000..2fe99cd6e --- /dev/null +++ b/docs/com/pres/stockholm/stockholm.md @@ -0,0 +1,164 @@ +# From Paris to Stockholm + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2019/06/09 + + + +## Agenda + + +### Agenda + +- Functest and Xtesting in a nutshell +- what's new from Paris (Iruya and J-release)? + - code improvement + - functional gates + - switch to python3 + - Raspberry PI +- what's next? GSMA, CNF, ETSI... + + + +## Infrastructure verification + + +### Functest in a nutshell + +- verify any kind of OpenStack and Kubernetes deployments (OPNFV model) + including production environments +- conform with upstream rules (OpenStack gate jobs and Kubernetes conformance + tests) +- ensure that the platforms meet Network Functions Virtualization requirements + + +### Functest suites + +- all functional tests (2000+) as defined by the upstream communities + (e.g. Tempest, neutron-tempest-api, Barbican, Patrole...) +- upstream API and dataplane benchmarking tools (Rally, VMTP and Shaker) +- Virtual Network Function deployments and testing (vIMS, vRouter and vEPC) + + + +### Network automation journey + + +### Xtesting in a nutshell + +- allow the developer to work only on the test suites without diving into + CI/CD integration +- simplify test integration in a complete LFN-based CI/CD toolchain (e.g. + Jenkins, Testing Containers, Test API and dashboard) +- allow a proper design and verify multiple components in the same CI/CD + toolchain (OpenStack, Kubernetes, ONAP, etc.) + +**Easy to use and very useful for any CI/CD toolchain** + + +### Chained CI + + +<!-- .element: style="border: 0" --> + + +### Testing [FTTH](https://en.wikipedia.org/wiki/Fiber_to_the_x) equipment + + +<!-- .element: style="border: 0; width: 90%" --> + + +### Demo + +- describe the testcase execution +- build and publish your container +- list your test in the toolchain description +- deploy your own toolchain in a few commands + +https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004 + + + +## What's new from Paris (Iruya and J-release) + + +### Code improvement + +- all patches are now verified by [bandit](https://github.com/PyCQA/bandit) (in + addition to yamllint, pep8, pylint, ansible-lint and bashate. Please see + [tox.ini](https://git.opnfv.org/functest/tree/tox.ini) for details). +- 10/10 by pylint is now enforced for all + Functest packages + +**we still need python3.6 in build servers to run our unit tests vs python3** + + +### Functional Gates + +- live-migration, VMTP and Shaker are now verified in all gates (pre and post + merge) +- Functest is continuously executed up to 4 tests in parallel verifying our + reference OpenStack/Kubernetes +- we continuously check the remaining resources after multiple runs (they only + remain a few region and volume leaks which are being fixed) + + +### Switch to python3 + +- all latest Functest containers switched to python3 as default (Functest has + supported both python2 and python3 via six for a while) +- all missing decode/encode calls are backported to Hunter and Iruya +- VMTP has been updated to keep the same testcase list +- our patch about Cloudify REST client will be published upstream if the + project is still active + + +### Raspberry PI + +- travis-ci.org allows us to continuously cross-compile containers (Hunter, + Iruya and latest) for arm (Raspberry PI) and arm64 +- no additional configuration is required (Docker manifest allows an unify way + to run the containers whatever the architectures) +- only testcases leveraging on go have not yet been ported (juju_epc and + Kubernetes conformance tests) + +**Try it, and you will love it!** + + +### New testcases + +- refstack_platform and refstack_object (already part of tempest_full) are now + dedicated testcases in Functest +- tempest_slow is on our agenda to keep conforming with OpenStack gates + + +### New opportunities + +- a few indentified opportunities about +[CNF](https://www.cncf.io/announcement/2019/02/25/cncf-launches-cloud-native-network-functions-cnf-testbed/): +[Clearwater Ims via Kubernetes](https://github.com/Metaswitch/clearwater-docker/) +and [xrally-kubernetes](https://github.com/xrally/xrally-kubernetes) +- GSMA should come with new testcases needs (Functest can already be executed + vs the current profiles) +- ETSI could leverage on Xtesting to package its testcases + + + +## Conclusion + + +### Takeaways + +- Xtesting is a key helper from first tests to full end2end service testing +- Functest allows verifying production Infrastructure and is breaking the + classical preproduction/production model +- And we are still pushing the limits: + - python packages + - GNU/Linux packages + +**Upstream first! Try them, and you will love them! ** + + + +## Thank you diff --git a/docs/com/pres/vevent202004/index.html b/docs/com/pres/vevent202004/index.html new file mode 100644 index 000000000..7c1118242 --- /dev/null +++ b/docs/com/pres/vevent202004/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>2020 April Virtual Technical Event</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="vevent202004.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/vevent202004/vevent202004.md b/docs/com/pres/vevent202004/vevent202004.md new file mode 100644 index 000000000..a7f8b5dc4 --- /dev/null +++ b/docs/com/pres/vevent202004/vevent202004.md @@ -0,0 +1,99 @@ +# CNTT RC April 2020 and beyond + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/04/21 + + + +## CNTT RC status + + +### Functest SUTs + +- ease verifying automatically the CNTT-related containers (5 SUTs in total) +- mostly conformed with OpenStack reference implementations except: + - all use Ceph and Ceph RadosGW + - one SUT leverages on OVN to check RA1 Chapter5 vs SDN Controllers +- OpenStack and Kubernetes are not redeployed between 2 runs as opposed to + [CNTT RI](https://build.opnfv.org/ci/view/cntt/job/cntt-latest-daily/102/) + +**Functest SUTs are compliant to CNTT RC** + + +### RC verification vs Neutron + +- all branches are successful vs Neutron reference implementation: + - [Functest Hunter (Rocky)](https://build.opnfv.org/ci/view/functest/job/functest-hunter-daily/485/) + - [Functest Iruya (Stein)](https://build.opnfv.org/ci/view/functest/job/functest-iruya-daily/452/) + - [Functest Jerma (Train)](https://build.opnfv.org/ci/view/functest/job/functest-jerma-daily/249/) + - [Functest Master (next Ussuri)](https://build.opnfv.org/ci/view/functest/job/functest-latest-daily/597/) + + +### RC verification vs OVN + +- the overall results are in a good shape as Functest master includes all latest software: + - [CNTT Conformance passed sucessfully](https://build.opnfv.org/ci/view/functest/job/functest-ovn-latest-daily/17/) + - [IaaS verification](https://build.opnfv.org/ci/view/functest/job/functest-ovn-latest-daily/17/) + failed due to one Rally task hanging iteration (under analysis) + +**RA1 Chapter 5 is compatible with external SDN controllers** + + + +## Field trial results + + +### Orange IaaS + +- Orange IaaS verification has leveraged on the Functest containers before CNTT +- Functest containers are also executed to verify the deployment before any VNF + onboarding +- a few mandatory features are missing to fully pass CNTT Compliance + ([cinder backup](https://cntt-n.github.io/CNTT/doc/ref_arch/openstack/chapters/chapter05.html) + and [keystone security_compliance](https://cntt-n.github.io/CNTT/doc/ref_arch/openstack/chapters/chapter05.html)) +- a few extra RadosGW configurations are needed + +**the CNTT Conformance is in a good shape** + + + +## What's next? + + +### last minute's changes: + +- tempest_heat was recently added in Functest IaaS verification and will be + tuned for CNTT compliance +- tempest_horizon should be added into CNTT RC document (mandatory features) + +**they would be added into CNTT RC very soon** + + +### and beyond + +- NFVBench and VSperf are being xtesting-ready and should be added in RC +- a few CNTT issues are opened for developping Functest and then RC: + - [Run fio or dd in Functest](https://github.com/cntt-n/CNTT/issues/1477) + - [Add ceph bench and rbd bench into Functest](https://github.com/cntt-n/CNTT/issues/1476) + - [Add swift-bench in Functest](https://github.com/cntt-n/CNTT/issues/1474) + +**[help is needed!](https://github.com/cntt-n/CNTT/issues?q=is%3Aopen+is%3Aissue+project%3Acntt-n%2FCNTT%2F4)** + + + +## Conclusion + + +### Take aways + +- CNTT RC1 1.0 seems good and stable +- [RA1 Chapter 5]((https://cntt-n.github.io/CNTT/doc/ref_arch/openstack/chapters/chapter05.html) + is compatible with external SDN controllers +- Orange IaaS is closed to be compliant + +**any contribution and RC results are more than welcome!** + + + +## Thank you diff --git a/docs/com/pres/vevent202010/index.html b/docs/com/pres/vevent202010/index.html new file mode 100644 index 000000000..a24c721d2 --- /dev/null +++ b/docs/com/pres/vevent202010/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Orange CNTT RC1 Field Trial feedbacks</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="vevent202010.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/vevent202010/vevent202010.md b/docs/com/pres/vevent202010/vevent202010.md new file mode 100644 index 000000000..b128210ac --- /dev/null +++ b/docs/com/pres/vevent202010/vevent202010.md @@ -0,0 +1,62 @@ +# Orange CNTT RC1 Field Trial + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/10/14 + + + +## CNTT RC1 Field Trial + + +### Orange CNTT RC1 Field Trial + +- helped detecting a couple of issues in CNTT RC1 +- integrated cinder backup and nova instance_password in Orange IaaS +- to fix 10 remaining single test failures (out 2000+ functional tests, 3 hours + benchmarking and 3 VNFs automatically onboarded and tested) +- to enhance Functest juju_epc to pass proxies + +**99,999%** + + +### Orange CNTT RC1 feedbacks + +- we easily executed the CNTT RC1 playbook asis and no clear bug was found in + CNTT RC1 Baldy +- CNTT RC1 is now used in our Orange IaaS verification in addition to the + classical Functest containers +- CNTT RC1 is also executed to verify all deployments before onboarding any VNF +- CNTT RC1 Baldy and CNTT RC2 Baraque are listed in **Orange RFP requirements** + (see last ONES + [[1]](http://testresults.opnfv.org/functest/ONES2020NA_OPNFV-CNTT_RFP.1080p.mp4)) + + + +## What else? + + +### Wish list + +- to integrate **more benchmarks** in CNTT conformance (e.g. disk benchmarking) +- to switch from the current Kubernetes interoperability testing to a **true** + CNTT conformance suite +- to build the first **VNF and CNF** conformance suites (**high priority**) + +**We need your contribution helps! +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[3]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + +### New Functest opportunities + +- to implement live monitoring of your OpenStack and Kubernetes deployments via + a subset of Functest (healthcheck? new Rally tasks?) +- to implement new functest testcases to validate and verify OpenStack upgrades + (new Rally tasks)? + +**Try Functest Leguer, you will love it [[4]](https://www.linkedin.com/pulse/opnfv-functest-leguer-out-c%25C3%25A9dric-ollivier/)** + + + +## Thank you! |
