diff options
Diffstat (limited to 'docs')
40 files changed, 1618 insertions, 697 deletions
diff --git a/docs/com/css/theme/OPNFV-Berlin.css b/docs/com/css/theme/OPNFV-Berlin.css index 9f957d6f1..34f73b4bd 100644 --- a/docs/com/css/theme/OPNFV-Berlin.css +++ b/docs/com/css/theme/OPNFV-Berlin.css @@ -31,13 +31,13 @@ body { bottom: 1em; left: 1em; font-size: 0.5em; - + } .reveal .slides > section, .reveal .slides > section > section { line-height: 1.3; font-weight: inherit; } - + .reveal .opnfv-title { background-image: url("../../img/title-bg-berlin.png" no-repeat); } @@ -87,7 +87,7 @@ body { .reveal h1 { text-shadow: none; } - + /********************************************* * OTHER diff --git a/docs/com/css/theme/OPNFV.css b/docs/com/css/theme/OPNFV.css index 745a2a8de..c59c59f8d 100644 --- a/docs/com/css/theme/OPNFV.css +++ b/docs/com/css/theme/OPNFV.css @@ -30,13 +30,13 @@ body { bottom: 1em; left: 1em; font-size: 0.5em; - + } .reveal .slides > section, .reveal .slides > section > section { line-height: 1.3; font-weight: inherit; } - + .reveal .opnfv-title { background-image: url("../../img/title-bg.png" no-repeat); } @@ -83,7 +83,7 @@ body { .reveal h1 { text-shadow: none; } - + /********************************************* * OTHER diff --git a/docs/com/pres/Summit/Berlin-2016/conversation.html b/docs/com/pres/Summit/Berlin-2016/conversation.html index 356c2ad1e..5c1e18450 100644 --- a/docs/com/pres/Summit/Berlin-2016/conversation.html +++ b/docs/com/pres/Summit/Berlin-2016/conversation.html @@ -215,7 +215,7 @@ </div> <div class='footer'> - <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> + <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> </div> </div> diff --git a/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html b/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html index 97fa66c18..2b3b3b4ef 100644 --- a/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html +++ b/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html @@ -81,7 +81,7 @@ </section> <section data-markdown> ## so concretely... - * Validate scenarios for the release + * Validate scenarios for the release * Give confidence on OPNFV fresh releases * Ensure consistancy towards installers * Ensure End to End interoperability @@ -135,7 +135,7 @@ * 2 installers: Foreman/Fuel * Lots of manual operations (reporting, documentation) </section> - </section> + </section> <section> <section data-markdown> # Brahmaputra @@ -146,8 +146,8 @@ * 3 feature projects: doctor, promise, sdnvpn * 13 Scenarios * 4 Installers (Apex, Compass, Fuel, Joid) - * Test result collection - * Automatic dashboard + * Test result collection + * Automatic dashboard </section> <section> <h3>Jiras</h3> @@ -196,7 +196,7 @@ </tr> </tbody> </table> - * difference of duration due to POD and test suites + * difference of duration due to POD and test suites </section> <section data-markdown> @@ -204,7 +204,7 @@ * Tempest/Rally * harmonizing installer related OpenStack configuration differences * creating customized test lists - * main challenge was resolving SDN controller interworking problems + * main challenge was resolving SDN controller interworking problems * vIMS * complete but complex test case * very interesting to automate (pre MANO, most of Telco needs met there) @@ -212,7 +212,7 @@ </section> - <section> + <section> <section data-markdown> # Colorado </section> @@ -221,7 +221,7 @@ ## What's new? * New internal test cases: healthcheck, security * New controler: OpenContrail? - * New feature projects (domino, multisites, movie, parser, moon, copper, models, onos-sfc...) + * New feature projects (domino, multisites, movie, parser, moon, copper, models, onos-sfc...) * ARM Support (congratulations to ENEA team!) * Simplified feature project integration journey </section> @@ -255,19 +255,19 @@ * New VNFs (vEPC, vCDN, vWhatever...) * Multi sites (e.g bgpvpn with different back ends) * Better test coverage - </section> + </section> <section data-markdown> ## A Functional testing As a Service framework * Scenario owner can select their relevant test cases * Test duration estimation * Agile dashboarding * Analytics - </section> + </section> <section data-markdown> ## Upstream * Rally: time to give back more - * Functest description for ETSI? - </section> + * Functest description for ETSI? + </section> </section> <section> diff --git a/docs/com/pres/Summit/Berlin-2016/testapi.html b/docs/com/pres/Summit/Berlin-2016/testapi.html index c40637cbf..5e56d5052 100644 --- a/docs/com/pres/Summit/Berlin-2016/testapi.html +++ b/docs/com/pres/Summit/Berlin-2016/testapi.html @@ -69,7 +69,7 @@ * Test cases * Results </section> - + <section data-markdown> ## Status Currently * From many projects @@ -87,7 +87,7 @@ </aside> </script> </section> - + <section data-markdown> ## Help building * Dashboards @@ -135,8 +135,8 @@ <li>Scripts to support databse backup/restore/update</li> <li>Support installation</li> <aside class='notes'> - So you can discover and understand the capabilities of the service without - access to source code, documentation, or through network traffic inspection, + So you can discover and understand the capabilities of the service without + access to source code, documentation, or through network traffic inspection, and also you can interact with the TestAPI directly through swagger website. </aside> </ul> @@ -148,12 +148,12 @@ <section data-markdown> # API Evoluation </section> - + <section> <h2> URI changes...</h2> - <div style="text-align:left""> + <div style="text-align:left""> <p> testresults.opnfv.org/<span style="color:lightblue">TestAPI</span> => <br>testresults.opnfv.org/<span style="color:yellow">test/api/v1</span> </p> - + <p> /test/api/v1/<b>pods</b></p> <p> /test/api/v1/<b>projects</b></p> <p> /test/api/v1/projects/qtip/<b>cases</b></p> @@ -200,7 +200,7 @@ <section> <h2>unit tests</h2> <pre><code class="hljs" data-trim contenteditable> -umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh +umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh Tests running... WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms diff --git a/docs/com/pres/compliance/compliance.md b/docs/com/pres/compliance/compliance.md new file mode 100644 index 000000000..cbe0dbe17 --- /dev/null +++ b/docs/com/pres/compliance/compliance.md @@ -0,0 +1,97 @@ +# From Verification to CNTT Compliance + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2019/12/04 + + + +## Infrastructure Verification + + +### Functest in a nutshell + +- verify any kind of OpenStack and Kubernetes deployments (OPNFV model) + including production environments +- conform with upstream rules (OpenStack gate jobs and Kubernetes conformance + tests) +- ensure that the platforms meet Network Functions Virtualization requirements + + +### Functest suites + +- all functional tests (3000+) as defined by the upstream communities + (e.g. Tempest, neutron-tempest-api, Barbican, Patrole...) +- 3 hours upstream API and dataplane benchmarking tests (Rally, VMTP and + Shaker) +- Virtual Network Function deployments and testing (vIMS, vRouter and vEPC) + + + +## CNTT Compliance + + +### Verification vs Compliance + +- verification allows skipping test if optional services (Gnocchi, Barbican, + etc.) or capabilities (remote console access, Neutron BGPVPN or SFC, etc.) + are missing +- compliance forces here the full API descriptions as currently proposed by + CNTT (please see + [Interfaces and APIs](https://github.com/cntt-n/CNTT/blob/master/doc/ref_arch/openstack/chapters/chapter05.md)) +- then the testcase descriptions should forbid skipping any test and cover only + the mandatory services (and their mandatory capabilities) + + +### Changelog + +- all the logics were already in Functest and the underlying frameworks (Rally, + Tempest, etc.) +- 2 new CNTT-related containers including the new testcase descriptions were + published to easily verify the compliance +- all 3 Functest SUTs (Rocky, Stein/Train and Master) are now compliant with + CNTT API to ensure the continuous integration +- the benchmarking testcases doesn't validate any KPI as nothing is written in + CNTT documentation + + +### RI verification and Compliance + +- CNTT Reference Implementation 1 is already continuously verified + ([continuous integration model](https://build.opnfv.org/ci/view/cntt/job/cntt-latest-daily/)) +- the conformance is currently failing due to a few bugs in deployments and + missing features + +**Be free to [deploy your own CNTT Compliance CI/CD toolchain](https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004) + in a few commands** + + + +## Conclusion + + +### Next steps + +- fix reference implementation 1 deployments and then achieve the compliance +- port existing OPNFV testcases to Xtesting and then add them in the continuous + integration loop +- update the testcase descritions according to the CNTT progress (KPI, API + changes) + + +### Takeaways + +- Functest allows verifying any production Infrastructure and now checking the + CNTT API Compliance +- all containers can be already consumed +- any third-party certification should reuse the Functest CNTT-related + containers as they are +- [CNTT RI continuous integration](https://build.opnfv.org/ci/view/cntt/job/cntt-latest-daily/) + is in place and any testcase can be smoothly added if they leverage on + Xtesting + +**Try it, and you will love it!** + + + +## Thank you diff --git a/docs/com/pres/compliance/index.html b/docs/com/pres/compliance/index.html new file mode 100644 index 000000000..94fc121d6 --- /dev/null +++ b/docs/com/pres/compliance/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>From Verification to Compliance</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="compliance.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/dockerslicing/dockerslicing.md b/docs/com/pres/dockerslicing/dockerslicing.md index a66453248..0d1ce5e21 100644 --- a/docs/com/pres/dockerslicing/dockerslicing.md +++ b/docs/com/pres/dockerslicing/dockerslicing.md @@ -63,7 +63,7 @@ ### 8 Functest containers -```bash +```shell $ sudo docker search opnfv |grep functest- opnfv/functest-core OPNFV Functest core image opnfv/functest-restapi OPNFV Functest restapi image diff --git a/docs/com/pres/euphrates_functest_evolution/euphrates.md b/docs/com/pres/euphrates_functest_evolution/euphrates.md index 49aab16cd..2c3105539 100644 --- a/docs/com/pres/euphrates_functest_evolution/euphrates.md +++ b/docs/com/pres/euphrates_functest_evolution/euphrates.md @@ -120,7 +120,7 @@ export OS_IDENTITY_API_VERSION=3 #### Retrieve all the images ``` -mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* +mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | sh -s -- images && ls -1 images/* images/CentOS-7-aarch64-GenericCloud.qcow2 images/CentOS-7-aarch64-GenericCloud.qcow2.xz images/CentOS-7-x86_64-GenericCloud.qcow2 diff --git a/docs/com/pres/functest2020/functest2020.md b/docs/com/pres/functest2020/functest2020.md new file mode 100644 index 000000000..a81667939 --- /dev/null +++ b/docs/com/pres/functest2020/functest2020.md @@ -0,0 +1,61 @@ +# Functest 2020 + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/01/16 + + + +## What's new in Jerma? + + +### Better test case verification + +- switch to Ceph in our Functest SUTs +- add Neutron features in our SUTs to improve the case verification +- test all capabilities possible (e.g. vnc_console) +- verify automatically the CNTT-related containers into additional to the + classical ones +- harden xrally_kubernetes + +**Functest SUTs are compliant to CNTT RC** + + +### New test cases + +- tempest_horizon +- tempest_keystone +- tempest_cinder +- refstack_platform +- refstack_object +- octavia +- xrally_kubernetes + + +### New usage + +- **support CNTT RC (API testing, API and dataplane benchmarking, VNF onboarding and testing)** +- verify ONAP WindRiver OpenLab via Functest CI in a VM ("Inception model") +- allow minimal l2-only testing via Rally + +**still pushing the limit!** + + + +## Kali (K-release) + + +### Ideas: + +- finish updating to Alpine 3.11 and Python 3.8 +- finish KloudBuster integration (it has to be updated to Python3 first) +- add tempest-stress +- update and possibly add heat-tempest-plugin +- add cyborg-tempest-plugin? ironic-tempest-plugin ? xxx-tempest-plugin? +- add CNF into Kubernetes testing + +**continuously hardening the gates and improving code quality** + + + +## Thank you diff --git a/docs/com/pres/functest2020/index.html b/docs/com/pres/functest2020/index.html new file mode 100644 index 000000000..88195becb --- /dev/null +++ b/docs/com/pres/functest2020/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Functest 2020</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="functest2020.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/gambia/gambia.md b/docs/com/pres/gambia/gambia.md index cf5f0f1bf..4a65e9905 100644 --- a/docs/com/pres/gambia/gambia.md +++ b/docs/com/pres/gambia/gambia.md @@ -85,7 +85,7 @@ the classical ONAP testing virtual machine (**> 1GB**). - all ONAP Robot Framework files retrieved from the original repositories - testcases.yaml describing the testcases -[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/) +[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/) ### What's new in Xtesting? @@ -94,7 +94,7 @@ the classical ONAP testing virtual machine (**> 1GB**). full CI/CD toolchains in few minutes** (Jenkins, Minio, TestAPI, MongoDB and Docker registry) -```bash +```shell virtualenv xtesting . xtesting/bin/activate pip install ansible docker diff --git a/docs/com/pres/oran/ftth.png b/docs/com/pres/oran/ftth.png Binary files differnew file mode 100644 index 000000000..5b441d1de --- /dev/null +++ b/docs/com/pres/oran/ftth.png diff --git a/docs/com/pres/oran/oran.md b/docs/com/pres/oran/oran.md new file mode 100644 index 000000000..87fba1306 --- /dev/null +++ b/docs/com/pres/oran/oran.md @@ -0,0 +1,90 @@ +--- +author: Cédric Ollivier +title: OPNFV/CNTT CI assets +date: 2020/12/01 +--- +## Continuous integration + + +### A few CNTT reqs + +- OPNFV has built a complete CI/CD toolchain for continuously deploying and + testing cloud infrastructure +- Reference conformance only requires for the local deployment of the + same components +- all test cases must be delivered as **Docker containers** including the + common test case execution and the unified way to manage all the interactions + +**Integrate smoothly and deploy fastly** + + +### How OPNFV/CNTT helps? + +- **Functest** offers a collection of state-of-the-art virtual infrastructure + test suites +- **Xtesting** helps assembling sparse test cases and accelerating the adoption + of CI/CD best practices +- **XtestingCI** eases deploying anywhere plug-and-play CI/CD toolchains in a + few commands +- **CNTT** defines conformance suites and its playbooks leveraging this 3 + testing frameworks + +**Leverage best opensource tools and practices** + + +### Testing [FTTH](https://en.wikipedia.org/wiki/Fiber_to_the_x) equipment + + + + +### Deploy your CI toolchain + +``` +virtualenv xtesting +. xtesting/bin/activate +pip install ansible +ansible-galaxy install collivier.xtesting +git clone https://gerrit.opnfv.org/gerrit/functest-xtesting functest-xtesting-src +ansible-playbook functest-xtesting-src/ansible/site.yml +deactivate +rm -rf functest-xtesting-src xtesting +``` + +**More scenarios in [[1]](https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004)** + + + +## Testing + + +### Verif', Compliance and Interop' + +- verification allows skipping test if optional services or capabilities are + missing +- compliance forces the full API descriptions as currently proposed by + CNTT and then skips all optional capability testing +- the opensource certifications are driven by simple interoperability testing + (RefStack and OVP are about less than 10% of Functest. **trustability?**) + +**Fine tune the same test frameworks** + + +### [CNTT RC1](https://build.opnfv.org/ci/view/functest/job/functest-leguer-daily/12/) + + + + + +## Conclusion + + +### Key takeaways + +- **test integration requirements** are crucial to smoothly assemble all + test cases +- leverage **existing** OPNFV testing knowledge (projects) and experience + (history) by utilising the OPNFV toolchain design already in-place +- **3000+** functional tests, **3 hours** upstream API and dataplane benchmarks + and VNFs automatically onboarded and tested **can be reused asis** + +**Keep it simple and do not reivent the wheel** diff --git a/docs/com/pres/oran/rc1.png b/docs/com/pres/oran/rc1.png Binary files differnew file mode 100644 index 000000000..9179f3bb6 --- /dev/null +++ b/docs/com/pres/oran/rc1.png diff --git a/docs/com/pres/rfp/index.html b/docs/com/pres/rfp/index.html new file mode 100644 index 000000000..538c0c8c8 --- /dev/null +++ b/docs/com/pres/rfp/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>OPNFV and CNTT in Orange RFP</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="rfp.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/rfp/rfp.md b/docs/com/pres/rfp/rfp.md new file mode 100644 index 000000000..66f7ba9e7 --- /dev/null +++ b/docs/com/pres/rfp/rfp.md @@ -0,0 +1,97 @@ +# OPNFV and CNTT in Orange RFP + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/09/19 + + + +## Vision and contributions + + +### Our guidelines + +- **automate** to bring determinism and to meet the new + software release rate +- test all software layers **independently** (OpenStack, Kubernetes and VNFs) +- run all deployment and verification jobs in **our** continous integration + chains +- leverage best **opensource** tools and practices + +**Integrate smoothly and deploy everywhere fastly** + + +### How OpenSource helps? + +- **Functest** offers a collection of state-of-the-art virtual infrastructure + test suites +- **Xtesting** helps assembling sparse test cases and accelerating the adoption + of CI/CD best practices +- **XtestingCI** eases deploying anywhere plug-and-play CI/CD toolchains in a + few commands +- **CNTT** defines conformance suites and its playbooks leveraging this 3 + softwares + +**Any contribution is more than welcome! +[[1]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + + +## CNTT/OPNFV in Orange + + +### A couple of RFP requirements + +- the **full** CNTT reference conformance for OpenStack results and outputs + (Orange CNTT Field Trial is in a very good shape + [[1]](http://testresults.opnfv.org/functest/field_trial/) + [[2]](https://www.linkedin.com/pulse/cntt-field-trials-c%C3%A9dric-ollivier/)) +- the **success** of the Functest Kubernetes test suites (now released as + part of CNTT RC2 Baraque) +- **first** VNF test cases running in **our** continuous integration chain + thanks to Xtesting and XtestingCI + +**It's implementing Orange and CNTT targets** + + +### Orange CNTT RC1 Field Trial + +- helped detecting a couple of issues in CNTT RC1 +- integrated cinder backup and nova instance_password in Orange IaaS +- to fix 10 remaining single test failures (out 2000+ functional tests, 3 hours + benchmarking and 3 VNFs automatically onboarded and tested) +- to enhance Functest juju_epc to pass proxies + +**99,999%** + + +### Wish list + +- to integrate **more benchmarks** in CNTT conformance (e.g. disk benchmarking) +- to switch from the current Kubernetes interoperability testing to a **true** + CNTT conformance suite +- to build the first **VNF and CNF** conformance suites (**high priority**) + +**We need your contribution helps! +[[1]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + + +## Conclusion + + +### Take aways + +- Orange leverages OPNFV and CNTT in RFP +- we keep contributing in both specification and implementation streams for + the success of Network Function Virtualization +- we expect more OPNFV and CNTT contributions especially for VNF and CNF + conformance suites, our initial CNTT target + +**Try CNTT reference suites, you will love them!** + + + +## Thank you! diff --git a/docs/com/pres/vevent202004/index.html b/docs/com/pres/vevent202004/index.html new file mode 100644 index 000000000..7c1118242 --- /dev/null +++ b/docs/com/pres/vevent202004/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>2020 April Virtual Technical Event</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="vevent202004.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/vevent202004/vevent202004.md b/docs/com/pres/vevent202004/vevent202004.md new file mode 100644 index 000000000..a7f8b5dc4 --- /dev/null +++ b/docs/com/pres/vevent202004/vevent202004.md @@ -0,0 +1,99 @@ +# CNTT RC April 2020 and beyond + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/04/21 + + + +## CNTT RC status + + +### Functest SUTs + +- ease verifying automatically the CNTT-related containers (5 SUTs in total) +- mostly conformed with OpenStack reference implementations except: + - all use Ceph and Ceph RadosGW + - one SUT leverages on OVN to check RA1 Chapter5 vs SDN Controllers +- OpenStack and Kubernetes are not redeployed between 2 runs as opposed to + [CNTT RI](https://build.opnfv.org/ci/view/cntt/job/cntt-latest-daily/102/) + +**Functest SUTs are compliant to CNTT RC** + + +### RC verification vs Neutron + +- all branches are successful vs Neutron reference implementation: + - [Functest Hunter (Rocky)](https://build.opnfv.org/ci/view/functest/job/functest-hunter-daily/485/) + - [Functest Iruya (Stein)](https://build.opnfv.org/ci/view/functest/job/functest-iruya-daily/452/) + - [Functest Jerma (Train)](https://build.opnfv.org/ci/view/functest/job/functest-jerma-daily/249/) + - [Functest Master (next Ussuri)](https://build.opnfv.org/ci/view/functest/job/functest-latest-daily/597/) + + +### RC verification vs OVN + +- the overall results are in a good shape as Functest master includes all latest software: + - [CNTT Conformance passed sucessfully](https://build.opnfv.org/ci/view/functest/job/functest-ovn-latest-daily/17/) + - [IaaS verification](https://build.opnfv.org/ci/view/functest/job/functest-ovn-latest-daily/17/) + failed due to one Rally task hanging iteration (under analysis) + +**RA1 Chapter 5 is compatible with external SDN controllers** + + + +## Field trial results + + +### Orange IaaS + +- Orange IaaS verification has leveraged on the Functest containers before CNTT +- Functest containers are also executed to verify the deployment before any VNF + onboarding +- a few mandatory features are missing to fully pass CNTT Compliance + ([cinder backup](https://cntt-n.github.io/CNTT/doc/ref_arch/openstack/chapters/chapter05.html) + and [keystone security_compliance](https://cntt-n.github.io/CNTT/doc/ref_arch/openstack/chapters/chapter05.html)) +- a few extra RadosGW configurations are needed + +**the CNTT Conformance is in a good shape** + + + +## What's next? + + +### last minute's changes: + +- tempest_heat was recently added in Functest IaaS verification and will be + tuned for CNTT compliance +- tempest_horizon should be added into CNTT RC document (mandatory features) + +**they would be added into CNTT RC very soon** + + +### and beyond + +- NFVBench and VSperf are being xtesting-ready and should be added in RC +- a few CNTT issues are opened for developping Functest and then RC: + - [Run fio or dd in Functest](https://github.com/cntt-n/CNTT/issues/1477) + - [Add ceph bench and rbd bench into Functest](https://github.com/cntt-n/CNTT/issues/1476) + - [Add swift-bench in Functest](https://github.com/cntt-n/CNTT/issues/1474) + +**[help is needed!](https://github.com/cntt-n/CNTT/issues?q=is%3Aopen+is%3Aissue+project%3Acntt-n%2FCNTT%2F4)** + + + +## Conclusion + + +### Take aways + +- CNTT RC1 1.0 seems good and stable +- [RA1 Chapter 5]((https://cntt-n.github.io/CNTT/doc/ref_arch/openstack/chapters/chapter05.html) + is compatible with external SDN controllers +- Orange IaaS is closed to be compliant + +**any contribution and RC results are more than welcome!** + + + +## Thank you diff --git a/docs/com/pres/vevent202010/index.html b/docs/com/pres/vevent202010/index.html new file mode 100644 index 000000000..a24c721d2 --- /dev/null +++ b/docs/com/pres/vevent202010/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Orange CNTT RC1 Field Trial feedbacks</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="vevent202010.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/vevent202010/vevent202010.md b/docs/com/pres/vevent202010/vevent202010.md new file mode 100644 index 000000000..b128210ac --- /dev/null +++ b/docs/com/pres/vevent202010/vevent202010.md @@ -0,0 +1,62 @@ +# Orange CNTT RC1 Field Trial + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/10/14 + + + +## CNTT RC1 Field Trial + + +### Orange CNTT RC1 Field Trial + +- helped detecting a couple of issues in CNTT RC1 +- integrated cinder backup and nova instance_password in Orange IaaS +- to fix 10 remaining single test failures (out 2000+ functional tests, 3 hours + benchmarking and 3 VNFs automatically onboarded and tested) +- to enhance Functest juju_epc to pass proxies + +**99,999%** + + +### Orange CNTT RC1 feedbacks + +- we easily executed the CNTT RC1 playbook asis and no clear bug was found in + CNTT RC1 Baldy +- CNTT RC1 is now used in our Orange IaaS verification in addition to the + classical Functest containers +- CNTT RC1 is also executed to verify all deployments before onboarding any VNF +- CNTT RC1 Baldy and CNTT RC2 Baraque are listed in **Orange RFP requirements** + (see last ONES + [[1]](http://testresults.opnfv.org/functest/ONES2020NA_OPNFV-CNTT_RFP.1080p.mp4)) + + + +## What else? + + +### Wish list + +- to integrate **more benchmarks** in CNTT conformance (e.g. disk benchmarking) +- to switch from the current Kubernetes interoperability testing to a **true** + CNTT conformance suite +- to build the first **VNF and CNF** conformance suites (**high priority**) + +**We need your contribution helps! +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[3]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + +### New Functest opportunities + +- to implement live monitoring of your OpenStack and Kubernetes deployments via + a subset of Functest (healthcheck? new Rally tasks?) +- to implement new functest testcases to validate and verify OpenStack upgrades + (new Rally tasks)? + +**Try Functest Leguer, you will love it [[4]](https://www.linkedin.com/pulse/opnfv-functest-leguer-out-c%25C3%25A9dric-ollivier/)** + + + +## Thank you! diff --git a/docs/release/release-notes/functest-release.rst b/docs/release/release-notes/functest-release.rst index 1f5d80d81..b5e228caa 100644 --- a/docs/release/release-notes/functest-release.rst +++ b/docs/release/release-notes/functest-release.rst @@ -33,8 +33,13 @@ The internal test cases are: * vping_userdata * cinder_test * tempest_smoke + * tempest_horizon * odl - * neutron-tempest-plugin-api + * tempest_neutron + * tempest_cinder + * tempest_keystone + * tempest_heat + * tempest_telemetry * rally_sanity * refstack_compute * refstack_object @@ -42,12 +47,12 @@ The internal test cases are: * tempest_full * tempest_scenario * tempest_slow - * patrole - * barbican - * neutron_trunk - * networking-bgpvpn - * networking-sfc - * octavia + * patrole_admin + * patrole_user + * patrole_reader + * tempest_barbican + * tempest_octavia + * tempest_cyborg * rally_full * rally_jobs * vmtp @@ -63,9 +68,17 @@ Kubernetes The internal test cases are: + * k8s_quick * k8s_smoke - * xrally_kubernetes * k8s_conformance + * xrally_kubernetes + * kube_hunter + * kube_bench_master + * kube_bench_node + * xrally_kubernetes_full + * k8s_vims + * helm_vims + * cnf_conformance Release Data ============ @@ -88,11 +101,16 @@ Software * https://hub.docker.com/r/opnfv/functest-smoke * https://hub.docker.com/r/opnfv/functest-benchmarking * https://hub.docker.com/r/opnfv/functest-vnf + * https://hub.docker.com/r/opnfv/functest-smoke-cntt + * https://hub.docker.com/r/opnfv/functest-benchmarking-cntt Functest Docker images (Kubernetes): * https://hub.docker.com/r/opnfv/functest-kubernetes-healthcheck * https://hub.docker.com/r/opnfv/functest-kubernetes-smoke + * https://hub.docker.com/r/opnfv/functest-kubernetes-security + * https://hub.docker.com/r/opnfv/functest-kubernetes-benchmarking + * https://hub.docker.com/r/opnfv/functest-kubernetes-cnf Docker tag for master: latest @@ -123,8 +141,8 @@ Key benefits * all testcases can run in parallel (tested with 4 executors in our gates) * no remaining resources detected in our gates after multiple runs -.. _`Functest toolchains`: https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004 -.. _`Raspberry PI`: https://wiki.opnfv.org/display/functest/Run+Functest+containers+on+Raspberry+PI +.. _`Functest toolchains`: https://github.com/collivier/ansible-role-xtesting +.. _`Raspberry PI`: https://github.com/opnfv/functest/blob/master/PI.md Code quality ------------ @@ -135,9 +153,8 @@ Code quality Useful links ============ - * wiki project page: https://wiki.opnfv.org/display/functest/Opnfv+Functional+Testing * Functest git repository: https://github.com/opnfv/functest - * Functest CI dashboard: https://build.opnfv.org/ci/view/functest/ + * Functest CI dashboard: http://104.154.71.112:8080/view/functest/ * JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611 * Functest IRC channel: #opnfv-functest * Reporting page: http://testresults.opnfv.org/reporting/master/functest/functest.html diff --git a/docs/release/release-notes/index.rst b/docs/release/release-notes/index.rst index c19d55610..25c2cbdfb 100644 --- a/docs/release/release-notes/index.rst +++ b/docs/release/release-notes/index.rst @@ -10,5 +10,3 @@ Functest Release Notes :maxdepth: 4 functest-release.rst - -Build date: |today| diff --git a/docs/results/euphrates/5.0/apex.html b/docs/results/euphrates/5.0/apex.html index 2a75054c3..34f0c1503 100644 --- a/docs/results/euphrates/5.0/apex.html +++ b/docs/results/euphrates/5.0/apex.html @@ -307,13 +307,13 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=http://testresultS.opnfv.org/reporting>os-odl-fdio-ha</a></td> <td><div id="gaugeScenario5"></div></td> @@ -327,7 +327,7 @@ $(document).ready(function (){ <td>4/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/108/console>os-nosdn-ovs_dpdk-ha</a></td> <td><div id="gaugeScenario8"></div></td> @@ -341,9 +341,9 @@ $(document).ready(function (){ <td>16/18</td> <td>2</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/139/console>os-nosdn-fdio-ha</a></td> <td><div id="gaugeScenario12"></div></td> @@ -357,11 +357,11 @@ $(document).ready(function (){ <td>18/20</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/121/console>os-nosdn-bar-ha</a></td> <td><div id="gaugeScenario17"></div></td> @@ -369,7 +369,7 @@ $(document).ready(function (){ <td>26/27</td> <td>4</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/134/console>os-nosdn-nofeature-ha</a></td> <td><div id="gaugeScenario19"></div></td> @@ -413,9 +413,9 @@ $(document).ready(function (){ <td>8/30</td> <td>4</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/114/console>os-ovn-nofeature-noha</a></td> <td><div id="gaugeScenario7"></div></td> @@ -423,9 +423,9 @@ $(document).ready(function (){ <td>6/18</td> <td>2</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/136/console>os-odl-nofeature-noha</a></td> <td><div id="gaugeScenario10"></div></td> @@ -439,9 +439,9 @@ $(document).ready(function (){ <td>4/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=http://testresultS.opnfv.org/reporting>os-odl-fdio_dvr-noha</a></td> <td><div id="gaugeScenario14"></div></td> @@ -461,7 +461,7 @@ $(document).ready(function (){ <td>18/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/128/console>os-nosdn-calipso-noha</a></td> <td><div id="gaugeScenario18"></div></td> @@ -469,7 +469,7 @@ $(document).ready(function (){ <td>18/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr> </table> @@ -489,31 +489,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -534,31 +534,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -582,37 +582,37 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> FDS - + </th><th> SNAPS - + </th><th> Domino * @@ -636,34 +636,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Doctor * @@ -693,37 +693,37 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> FDS - + </th><th> SNAPS - + </th><th> Domino * @@ -747,31 +747,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -792,31 +792,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -843,31 +843,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -888,31 +888,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Doctor * @@ -942,34 +942,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Doctor * @@ -996,31 +996,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -1041,31 +1041,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -1089,34 +1089,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Doctor * @@ -1143,37 +1143,37 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> FDS - + </th><th> SNAPS - + </th><th> Domino * @@ -1197,31 +1197,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -1251,31 +1251,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -1302,31 +1302,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -1356,31 +1356,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -1407,31 +1407,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * diff --git a/docs/results/euphrates/5.0/compass.html b/docs/results/euphrates/5.0/compass.html index d62ba4d2e..b7f7a20ea 100644 --- a/docs/results/euphrates/5.0/compass.html +++ b/docs/results/euphrates/5.0/compass.html @@ -223,11 +223,11 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-euphrates/81/console>os-odl_l3-nofeature-ha</a></td> <td><div id="gaugeScenario4"></div></td> @@ -247,7 +247,7 @@ $(document).ready(function (){ <td>26/30</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-euphrates/72/console>os-nosdn-ovs_dpdk-ha</a></td> <td><div id="gaugeScenario8"></div></td> @@ -255,9 +255,9 @@ $(document).ready(function (){ <td>2/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-baremetal-daily-euphrates/24/console>os-odl_l2-moon-ha</a></td> <td><div id="gaugeScenario11"></div></td> @@ -301,11 +301,11 @@ $(document).ready(function (){ <td>24/30</td> <td>4</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-euphrates/86/console>os-nosdn-nofeature-noha</a></td> <td><div id="gaugeScenario7"></div></td> @@ -313,7 +313,7 @@ $(document).ready(function (){ <td>26/27</td> <td>6</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-euphrates/82/console>os-nosdn-ovs_dpdk-noha</a></td> <td><div id="gaugeScenario9"></div></td> @@ -327,9 +327,9 @@ $(document).ready(function (){ <td>14/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr> </table> @@ -349,31 +349,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -397,34 +397,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> SFC * @@ -451,34 +451,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * @@ -502,31 +502,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * @@ -550,31 +550,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -598,34 +598,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> SFC * @@ -652,31 +652,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -700,31 +700,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -745,31 +745,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -790,31 +790,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * @@ -838,34 +838,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * @@ -889,31 +889,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * diff --git a/docs/results/euphrates/5.0/daisy.html b/docs/results/euphrates/5.0/daisy.html index 6de8d0895..09580fc19 100644 --- a/docs/results/euphrates/5.0/daisy.html +++ b/docs/results/euphrates/5.0/daisy.html @@ -127,9 +127,9 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr> </table> @@ -149,31 +149,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -206,34 +206,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * diff --git a/docs/results/euphrates/5.0/fuel@aarch64.html b/docs/results/euphrates/5.0/fuel@aarch64.html index c8a2f2db0..aa9f1eb66 100644 --- a/docs/results/euphrates/5.0/fuel@aarch64.html +++ b/docs/results/euphrates/5.0/fuel@aarch64.html @@ -115,7 +115,7 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-fuel-armband-baremetal-arm-daily-euphrates/31/console>os-nosdn-nofeature-ha</a></td> <td><div id="gaugeScenario2"></div></td> @@ -147,9 +147,9 @@ $(document).ready(function (){ <td>1/9</td> <td>1</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr> </table> @@ -169,31 +169,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -220,31 +220,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -283,34 +283,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Promise * diff --git a/docs/results/euphrates/5.0/fuel@x86.html b/docs/results/euphrates/5.0/fuel@x86.html index 3b918dd74..52b67b37f 100644 --- a/docs/results/euphrates/5.0/fuel@x86.html +++ b/docs/results/euphrates/5.0/fuel@x86.html @@ -151,7 +151,7 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-fuel-baremetal-daily-euphrates/40/console>os-odl-nofeature-ha</a></td> <td><div id="gaugeScenario2"></div></td> @@ -159,9 +159,9 @@ $(document).ready(function (){ <td>29/30</td> <td>6</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-fuel-baremetal-daily-euphrates/38/console>os-nosdn-ovs-ha</a></td> <td><div id="gaugeScenario5"></div></td> @@ -193,7 +193,7 @@ $(document).ready(function (){ <td>25/27</td> <td>10</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-fuel-virtual-daily-euphrates/91/console>os-odl-nofeature-noha</a></td> <td><div id="gaugeScenario3"></div></td> @@ -207,9 +207,9 @@ $(document).ready(function (){ <td>27/27</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr> </table> @@ -229,31 +229,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -280,34 +280,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Promise * @@ -334,34 +334,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Promise * @@ -388,31 +388,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -439,31 +439,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -490,31 +490,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * diff --git a/docs/results/euphrates/5.0/joid.html b/docs/results/euphrates/5.0/joid.html index d58efcca1..9b6d0f4d0 100644 --- a/docs/results/euphrates/5.0/joid.html +++ b/docs/results/euphrates/5.0/joid.html @@ -163,7 +163,7 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/84/console>os-ocl-nofeature-ha</a></td> <td><div id="gaugeScenario2"></div></td> @@ -171,7 +171,7 @@ $(document).ready(function (){ <td>3/27</td> <td>10</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/67/console>os-nosdn-openbaton-ha</a></td> <td><div id="gaugeScenario4"></div></td> @@ -185,7 +185,7 @@ $(document).ready(function (){ <td>2/15</td> <td>4</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/81/console>os-nosdn-nofeature-ha</a></td> <td><div id="gaugeScenario7"></div></td> @@ -211,7 +211,7 @@ $(document).ready(function (){ <td>3/27</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/90/console>os-nosdn-nofeature-noha</a></td> <td><div id="gaugeScenario3"></div></td> @@ -219,9 +219,9 @@ $(document).ready(function (){ <td>15/27</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/88/console>os-nosdn-lxd-noha</a></td> <td><div id="gaugeScenario6"></div></td> @@ -229,7 +229,7 @@ $(document).ready(function (){ <td>3/15</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr> </table> @@ -249,31 +249,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -294,31 +294,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -339,31 +339,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -390,31 +390,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -441,19 +441,19 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> vPing (ssh) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> Promise * @@ -480,19 +480,19 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> vPing (ssh) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> Promise * @@ -519,31 +519,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * diff --git a/docs/results/js/default.css b/docs/results/js/default.css index e32fa5fba..cbb564326 100644 --- a/docs/results/js/default.css +++ b/docs/results/js/default.css @@ -55,7 +55,7 @@ td{ background-color: #0095a2; } -h1 { +h1 { display: block; font-size: 2em; margin-top: 0.67em; diff --git a/docs/results/js/trend.js b/docs/results/js/trend.js index f24213382..47ee4d906 100644 --- a/docs/results/js/trend.js +++ b/docs/results/js/trend.js @@ -63,13 +63,13 @@ var trend = function(container, trend_data) { .attr("d", valueline(trend_data)) .attr("stroke", "steelblue") .attr("fill", "none"); - + trend_svg.selectAll(".dot") .data(trend_data) .enter().append("circle") .attr("r", 2.5) .attr("cx", function(d) { return trend_x(d.date); }) - .attr("cy", function(d) { return trend_y(d.score); }); + .attr("cy", function(d) { return trend_y(d.score); }); return trend; } diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index 32881e65a..f4dcea507 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -14,6 +14,8 @@ ci Clearwater cloudify Cloudify +cnf +cntt conf config dashboarding @@ -30,6 +32,7 @@ fds frontend functest Functest +functionalities guestbook Guestbook healthcheck @@ -55,6 +58,7 @@ kubernetes Kubernetes kvm linux +login metadata Metadata middleware @@ -84,6 +88,7 @@ Reachability reconnection redis refactored +refactoring refstack Refstack repo @@ -124,6 +129,7 @@ vEPC verifier versioned vIMS +vims Virtualised Virtualized vm diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst index a0faf42c1..da5485ef0 100644 --- a/docs/testing/developer/devguide/index.rst +++ b/docs/testing/developer/devguide/index.rst @@ -208,13 +208,6 @@ Additional dashboarding is managed at the testing group level, see `[5]`_ for details. -======= -How TOs -======= - -See How to section on Functest wiki `[6]`_ - - ========== References ========== @@ -227,8 +220,6 @@ _`[3]`: https://github.com/opnfv/releng-testresults/tree/master/reporting _`[4]`: https://wiki.opnfv.org/display/functest/2017+Beijing?preview=%2F11699623%2F11700523%2FTestAPI+-+test+results+collection+service.pptx -_`[5]`: https://opnfv.biterg.io/login?nextUrl=%2Fgoto%2F283dba93ca18e95964f852c63af1d1ba - -_`[6]`: https://wiki.opnfv.org/pages/viewpage.action?pageId=7768932 +_`[5]`: https://lfanalytics.io/projects/lfn%2Fopnfv/dashboard IRC support chan: #opnfv-functest diff --git a/docs/testing/user/configguide/ci.rst b/docs/testing/user/configguide/ci.rst deleted file mode 100644 index ee815dbe1..000000000 --- a/docs/testing/user/configguide/ci.rst +++ /dev/null @@ -1,16 +0,0 @@ -.. SPDX-License-Identifier: CC-BY-4.0 - -Integration in CI -================= -In CI we use the Docker images and execute the appropriate commands within the -container from Jenkins. - -4 steps have been defined:: - * functest-cleanup: clean existing functest dockers on the jumphost - * functest-daily: run dockers opnfv/functest-* (healthcheck, smoke, - benchmarking, vnf) - * functest-store-results: push logs to artifacts - -See `[1]`_ for details. - -.. _`[1]`: https://github.com/opnfv/releng/blob/master/jjb/functest/functest-daily-jobs.yaml diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index f0583f245..47c25adb1 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -13,10 +13,11 @@ Functest Dockers for OpenStack deployment ----------------------------------------- Docker images are available on the dockerhub: - * opnfv/functest-core * opnfv/functest-healthcheck * opnfv/functest-smoke + * opnfv/functest-smoke-cntt * opnfv/functest-benchmarking + * opnfv/functest-benchmarking-cntt * opnfv/functest-vnf @@ -28,6 +29,7 @@ cat env:: DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario NAMESERVER=XXX # if not 8.8.8.8 EXTERNAL_NETWORK=XXX # if not first network with router:external=True + DASHBOARD_URL=XXX # else tempest_horizon will be skipped NEW_USER_ROLE=XXX # if not member SDN_CONTROLLER_IP=XXX # if odl scenario VOLUME_DEVICE_NAME=XXX # if not vdb @@ -50,11 +52,12 @@ See section on OpenStack credentials for details. Create a directory for the different images (attached as a Docker volume):: - mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* + mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | sh -s -- images && ls -1 images/* - images/cirros-0.4.0-aarch64-disk.img - images/cirros-0.4.0-x86_64-disk.img + images/cirros-0.5.1-aarch64-disk.img + images/cirros-0.5.1-x86_64-disk.img images/cloudify-docker-manager-community-19.01.24.tar + images/Fedora-Cloud-Base-30-1.2.x86_64.qcow2 images/shaker-image-1.3.0+stretch.qcow2 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img @@ -76,17 +79,18 @@ Results shall be displayed as follows:: +--------------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +--------------------------+------------------+---------------------+------------------+----------------+ - | connection_check | functest | healthcheck | 00:02 | PASS | - | tenantnetwork1 | functest | healthcheck | 00:06 | PASS | + | connection_check | functest | healthcheck | 00:03 | PASS | + | tenantnetwork1 | functest | healthcheck | 00:05 | PASS | | tenantnetwork2 | functest | healthcheck | 00:06 | PASS | | vmready1 | functest | healthcheck | 00:06 | PASS | | vmready2 | functest | healthcheck | 00:08 | PASS | - | singlevm1 | functest | healthcheck | 00:28 | PASS | - | singlevm2 | functest | healthcheck | 00:25 | PASS | - | vping_ssh | functest | healthcheck | 00:36 | PASS | - | vping_userdata | functest | healthcheck | 00:34 | PASS | - | cinder_test | functest | healthcheck | 01:03 | PASS | - | tempest_smoke | functest | healthcheck | 05:13 | PASS | + | singlevm1 | functest | healthcheck | 00:32 | PASS | + | singlevm2 | functest | healthcheck | 00:37 | PASS | + | vping_ssh | functest | healthcheck | 00:46 | PASS | + | vping_userdata | functest | healthcheck | 00:39 | PASS | + | cinder_test | functest | healthcheck | 01:05 | PASS | + | tempest_smoke | functest | healthcheck | 05:39 | PASS | + | tempest_horizon | functest | healthcheck | 01:05 | PASS | | odl | functest | healthcheck | 00:00 | SKIP | +--------------------------+------------------+---------------------+------------------+----------------+ @@ -104,28 +108,57 @@ Run smoke suite:: Results shall be displayed as follows:: - +------------------------------------+------------------+---------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +------------------------------------+------------------+---------------+------------------+----------------+ - | neutron-tempest-plugin-api | functest | smoke | 09:12 | PASS | - | rally_sanity | functest | smoke | 16:29 | PASS | - | refstack_compute | functest | smoke | 06:25 | PASS | - | refstack_object | functest | smoke | 01:54 | PASS | - | refstack_platform | functest | smoke | 06:52 | PASS | - | tempest_full | functest | smoke | 30:26 | PASS | - | tempest_scenario | functest | smoke | 09:23 | PASS | - | tempest_slow | functest | smoke | 24:42 | PASS | - | patrole | functest | smoke | 02:36 | PASS | - | barbican | functest | smoke | 02:13 | PASS | - | neutron_trunk | functest | smoke | 00:00 | SKIP | - | networking-bgpvpn | functest | smoke | 00:00 | SKIP | - | networking-sfc | functest | smoke | 00:00 | SKIP | - | octavia | functest | smoke | 00:00 | SKIP | - +------------------------------------+------------------+---------------+------------------+----------------+ + +---------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+---------------+------------------+----------------+ + | tempest_neutron | functest | smoke | 15:30 | PASS | + | tempest_cinder | functest | smoke | 02:01 | PASS | + | tempest_keystone | functest | smoke | 01:17 | PASS | + | tempest_heat | functest | smoke | 22:14 | PASS | + | tempest_telemetry | functest | smoke | 00:00 | SKIP | + | rally_sanity | functest | smoke | 17:24 | PASS | + | refstack_compute | functest | smoke | 07:03 | PASS | + | refstack_object | functest | smoke | 02:09 | PASS | + | refstack_platform | functest | smoke | 07:31 | PASS | + | tempest_full | functest | smoke | 41:52 | PASS | + | tempest_scenario | functest | smoke | 08:42 | PASS | + | tempest_slow | functest | smoke | 43:42 | PASS | + | patrole_admin | functest | smoke | 21:06 | PASS | + | patrole_member | functest | smoke | 21:23 | PASS | + | patrole_reader | functest | smoke | 21:56 | PASS | + | tempest_barbican | functest | smoke | 02:30 | PASS | + | tempest_octavia | functest | smoke | 00:00 | SKIP | + | tempest_cyborg | functest | smoke | 00:00 | SKIP | + +---------------------------+------------------+---------------+------------------+----------------+ Note: if the scenario does not support some tests, they are indicated as SKIP. See User guide for details. +Testing smoke CNTT suite +^^^^^^^^^^^^^^^^^^^^^^^^ + +Run smoke-cntt suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-smoke-cntt + +Results shall be displayed as follows:: + + +-------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------------+------------------+---------------+------------------+----------------+ + | tempest_neutron_cntt | functest | smoke | 11:35 | PASS | + | tempest_cinder_cntt | functest | smoke | 01:58 | PASS | + | tempest_keystone_cntt | functest | smoke | 01:13 | PASS | + | tempest_heat_cntt | functest | smoke | 22:32 | PASS | + | rally_sanity_cntt | functest | smoke | 17:16 | PASS | + | tempest_full_cntt | functest | smoke | 41:13 | PASS | + | tempest_scenario_cntt | functest | smoke | 08:57 | PASS | + | tempest_slow_cntt | functest | smoke | 35:58 | PASS | + +-------------------------------+------------------+---------------+------------------+----------------+ + Testing benchmarking suite ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -141,15 +174,36 @@ Results shall be displayed as follows:: +--------------------+------------------+----------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +--------------------+------------------+----------------------+------------------+----------------+ - | rally_full | functest | benchmarking | 92:16 | PASS | - | rally_jobs | functest | benchmarking | 18:49 | PASS | - | vmtp | functest | benchmarking | 15:28 | PASS | - | shaker | functest | benchmarking | 24:04 | PASS | + | rally_full | functest | benchmarking | 93:03 | PASS | + | rally_jobs | functest | benchmarking | 27:05 | PASS | + | vmtp | functest | benchmarking | 17:56 | PASS | + | shaker | functest | benchmarking | 24:02 | PASS | +--------------------+------------------+----------------------+------------------+----------------+ Note: if the scenario does not support some tests, they are indicated as SKIP. See User guide for details. +Testing benchmarking CNTT suite +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run benchmarking-cntt suite:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-benchmarking-cntt + +Results shall be displayed as follows:: + + +-------------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+----------------------+------------------+----------------+ + | rally_full_cntt | functest | benchmarking | 89:52 | PASS | + | rally_jobs_cntt | functest | benchmarking | 19:39 | PASS | + | vmtp | functest | benchmarking | 16:59 | PASS | + | shaker | functest | benchmarking | 23:43 | PASS | + +-------------------------+------------------+----------------------+------------------+----------------+ + Testing vnf suite ^^^^^^^^^^^^^^^^^ @@ -165,20 +219,22 @@ Results shall be displayed as follows:: +----------------------+------------------+--------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +----------------------+------------------+--------------+------------------+----------------+ - | cloudify | functest | vnf | 03:49 | PASS | - | cloudify_ims | functest | vnf | 24:20 | PASS | - | heat_ims | functest | vnf | 32:13 | PASS | - | vyos_vrouter | functest | vnf | 14:55 | PASS | - | juju_epc | functest | vnf | 41:24 | PASS | + | cloudify | functest | vnf | 05:08 | PASS | + | cloudify_ims | functest | vnf | 24:46 | PASS | + | heat_ims | functest | vnf | 33:12 | PASS | + | vyos_vrouter | functest | vnf | 15:53 | PASS | + | juju_epc | functest | vnf | 27:52 | PASS | +----------------------+------------------+--------------+------------------+----------------+ Functest Dockers for Kubernetes deployment ------------------------------------------ Docker images are available on the dockerhub: - * opnfv/functest-kubernetes-core * opnfv/functest-kubernetes-healthcheck * opnfv/functest-kubernetes-smoke + * opnfv/functest-kubernetes-security + * opnfv/functest-kubernetes-benchmarking + * opnfv/functest-kubernetes-cnf Preparing your environment ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -204,7 +260,8 @@ Results shall be displayed as follows:: +-------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +-------------------+------------------+---------------------+------------------+----------------+ - | k8s_smoke | functest | healthcheck | 01:09 | PASS | + | k8s_quick | functest | healthcheck | 00:18 | PASS | + | k8s_smoke | functest | healthcheck | 01:14 | PASS | +-------------------+------------------+---------------------+------------------+----------------+ Testing smoke suite @@ -221,10 +278,65 @@ Results shall be displayed as follows:: +---------------------------+------------------+---------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +---------------------------+------------------+---------------+------------------+----------------+ - | xrally_kubernetes | functest | smoke | 22:04 | PASS | - | k8s_conformance | functest | smoke | 173:48 | PASS | + | k8s_conformance | functest | smoke | 94:26 | PASS | + | xrally_kubernetes | functest | smoke | 13:05 | PASS | +---------------------------+------------------+---------------+------------------+----------------+ +Testing security suite +^^^^^^^^^^^^^^^^^^^^^^ + +Run smoke suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-security + +Results shall be displayed as follows:: + + +---------------------------+------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+------------------+------------------+----------------+ + | kube_hunter | functest | security | 00:19 | PASS | + | kube_bench_master | functest | security | 00:02 | PASS | + | kube_bench_node | functest | security | 00:01 | PASS | + +---------------------------+------------------+------------------+------------------+----------------+ + +Testing benchmarking suite +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run benchmarking suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-benchmarking + +Results shall be displayed as follows:: + + +--------------------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------------+------------------+----------------------+------------------+----------------+ + | xrally_kubernetes_full | functest | benchmarking | 34:16 | PASS | + +--------------------------------+------------------+----------------------+------------------+----------------+ + +Testing cnf suite +^^^^^^^^^^^^^^^^^ + +Run cnf suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-cnf + +Results shall be displayed as follows:: + + +-------------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+--------------+------------------+----------------+ + | k8s_vims | functest | cnf | 09:06 | PASS | + | helm_vims | functest | cnf | 08:54 | PASS | + | cnf_conformance | functest | cnf | 02:00 | PASS | + +-------------------------+------------------+--------------+------------------+----------------+ + Environment variables ===================== @@ -353,7 +465,7 @@ By default all the logs are put un /home/opnfv/functest/results/functest.log. If you want to have more logs in console, you may edit the logging.ini file manually. Connect on the docker then edit the file located in -/usr/lib/python3.7/site-packages/xtesting/ci/logging.ini +/usr/lib/python3.8/site-packages/xtesting/ci/logging.ini Change wconsole to console in the desired module to get more traces. @@ -363,8 +475,8 @@ Configuration You may also directly modify the python code or the configuration file (e.g. testcases.yaml used to declare test constraints) under -/usr/lib/python3.7/site-packages/xtesting and -/usr/lib/python3.7/site-packages/functest +/usr/lib/python3.8/site-packages/xtesting and +/usr/lib/python3.8/site-packages/functest Tips @@ -576,5 +688,5 @@ work with the above pre-requisite actions. .. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/ .. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html .. _`images/CentOS-7-x86_64-GenericCloud.qcow2`: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 -.. _`images/cirros-0.4.0-x86_64-disk.img`: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img +.. _`images/cirros-0.5.1-x86_64-disk.img`: http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img .. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img diff --git a/docs/testing/user/configguide/index.rst b/docs/testing/user/configguide/index.rst index e7967adb3..e5963435e 100644 --- a/docs/testing/user/configguide/index.rst +++ b/docs/testing/user/configguide/index.rst @@ -11,7 +11,6 @@ Functest Installation Guide intro.rst prerequisites.rst configguide.rst - ci.rst References ========== @@ -20,4 +19,4 @@ References IRC support channel: #opnfv-functest -.. _`[1]`: https://github.com/opnfv/releng/blob/master/jjb/functest/functest-daily-jobs.yaml +.. _`[1]`: https://github.com/opnfv/releng/blob/master/jjb/functest/functest.yaml diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index 07ab0e62d..1e73cd622 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -40,7 +40,7 @@ References `[11]`_: Robot Framework web site -`[13]`_: SNAPS wiki +`[13]`_: SNAPS `[14]`_: vRouter @@ -64,12 +64,12 @@ IRC support chan: #opnfv-functest .. _`[9]`: https://github.com/openstack/interop .. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst .. _`[11]`: https://robotframework.org/ -.. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO +.. _`[13]`: https://git.opnfv.org/snaps/ .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter .. _`[15]`: https://aptira.com/testing-openstack-tempest-part-1/ .. _`[16]`: http://testresults.opnfv.org/test/ .. _`OPNFV main site`: https://www.opnfv.org/ -.. _`Functest page`: https://wiki.opnfv.org/display/functest/Opnfv+Functional+Testing +.. _`Functest page`: https://github.com/opnfv/functest/ .. _`OpenRC`: http://docs.openstack.org/user-guide/common/cli_set_environment_variables_using_openstack_rc.html .. _`Rally installation procedure`: https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html .. _`config_functest.yaml` : https://github.com/opnfv/functest/blob/master/functest/ci/config_functest.yaml diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst index 6061ce9ef..98247d488 100644 --- a/docs/testing/user/userguide/test_details.rst +++ b/docs/testing/user/userguide/test_details.rst @@ -405,8 +405,8 @@ The kubernetes testcases are distributed across various Tiers: .. _`[9]`: https://github.com/openstack/interop .. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst .. _`[11]`: https://robotframework.org/ -.. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO +.. _`[13]`: https://git.opnfv.org/snaps/ .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter -.. _`[15]`: https://www.rebaca.com/what-we-do/abot-5g-network-simulator/ +.. _`[15]`: https://github.com/RebacaInc/abot_charm .. _`[16]`: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md .. _`[17]`: https://github.com/Metaswitch/clearwater-heat/blob/release-129/clearwater.yaml diff --git a/docs/testing/user/userguide/test_results.rst b/docs/testing/user/userguide/test_results.rst index a1d4dfb59..10f87d8ec 100644 --- a/docs/testing/user/userguide/test_results.rst +++ b/docs/testing/user/userguide/test_results.rst @@ -10,7 +10,7 @@ In manual mode test results are displayed in the console and result files are put in /home/opnfv/functest/results. If you want additional logs, you may configure the logging.ini under -/usr/lib/python3.7/site-packages/xtesting/ci. +/usr/lib/python3.8/site-packages/xtesting/ci. Automated testing ----------------- @@ -24,62 +24,93 @@ Healthcheck suite:: +--------------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +--------------------------+------------------+---------------------+------------------+----------------+ - | connection_check | functest | healthcheck | 00:02 | PASS | - | tenantnetwork1 | functest | healthcheck | 00:06 | PASS | + | connection_check | functest | healthcheck | 00:03 | PASS | + | tenantnetwork1 | functest | healthcheck | 00:05 | PASS | | tenantnetwork2 | functest | healthcheck | 00:06 | PASS | | vmready1 | functest | healthcheck | 00:06 | PASS | | vmready2 | functest | healthcheck | 00:08 | PASS | - | singlevm1 | functest | healthcheck | 00:28 | PASS | - | singlevm2 | functest | healthcheck | 00:25 | PASS | - | vping_ssh | functest | healthcheck | 00:36 | PASS | - | vping_userdata | functest | healthcheck | 00:34 | PASS | - | cinder_test | functest | healthcheck | 01:03 | PASS | - | tempest_smoke | functest | healthcheck | 05:13 | PASS | + | singlevm1 | functest | healthcheck | 00:32 | PASS | + | singlevm2 | functest | healthcheck | 00:37 | PASS | + | vping_ssh | functest | healthcheck | 00:46 | PASS | + | vping_userdata | functest | healthcheck | 00:39 | PASS | + | cinder_test | functest | healthcheck | 01:05 | PASS | + | tempest_smoke | functest | healthcheck | 05:39 | PASS | + | tempest_horizon | functest | healthcheck | 01:05 | PASS | | odl | functest | healthcheck | 00:00 | SKIP | +--------------------------+------------------+---------------------+------------------+----------------+ Smoke suite:: - +------------------------------------+------------------+---------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +------------------------------------+------------------+---------------+------------------+----------------+ - | neutron-tempest-plugin-api | functest | smoke | 09:12 | PASS | - | rally_sanity | functest | smoke | 16:29 | PASS | - | refstack_compute | functest | smoke | 06:25 | PASS | - | refstack_object | functest | smoke | 01:54 | PASS | - | refstack_platform | functest | smoke | 06:52 | PASS | - | tempest_full | functest | smoke | 30:26 | PASS | - | tempest_scenario | functest | smoke | 09:23 | PASS | - | tempest_slow | functest | smoke | 24:42 | PASS | - | patrole | functest | smoke | 02:36 | PASS | - | barbican | functest | smoke | 02:13 | PASS | - | neutron_trunk | functest | smoke | 00:00 | SKIP | - | networking-bgpvpn | functest | smoke | 00:00 | SKIP | - | networking-sfc | functest | smoke | 00:00 | SKIP | - | octavia | functest | smoke | 00:00 | SKIP | - +------------------------------------+------------------+---------------+------------------+----------------+ + +---------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+---------------+------------------+----------------+ + | tempest_neutron | functest | smoke | 15:30 | PASS | + | tempest_cinder | functest | smoke | 02:01 | PASS | + | tempest_keystone | functest | smoke | 01:17 | PASS | + | tempest_heat | functest | smoke | 22:14 | PASS | + | tempest_telemetry | functest | smoke | 00:00 | SKIP | + | rally_sanity | functest | smoke | 17:24 | PASS | + | refstack_compute | functest | smoke | 07:03 | PASS | + | refstack_object | functest | smoke | 02:09 | PASS | + | refstack_platform | functest | smoke | 07:31 | PASS | + | tempest_full | functest | smoke | 41:52 | PASS | + | tempest_scenario | functest | smoke | 08:42 | PASS | + | tempest_slow | functest | smoke | 43:42 | PASS | + | patrole_admin | functest | smoke | 21:06 | PASS | + | patrole_member | functest | smoke | 21:23 | PASS | + | patrole_reader | functest | smoke | 21:56 | PASS | + | tempest_barbican | functest | smoke | 02:30 | PASS | + | tempest_octavia | functest | smoke | 00:00 | SKIP | + | tempest_cyborg | functest | smoke | 00:00 | SKIP | + +---------------------------+------------------+---------------+------------------+----------------+ + +Smoke CNTT suite:: + + +-------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------------+------------------+---------------+------------------+----------------+ + | tempest_neutron_cntt | functest | smoke | 11:35 | PASS | + | tempest_cinder_cntt | functest | smoke | 01:58 | PASS | + | tempest_keystone_cntt | functest | smoke | 01:13 | PASS | + | tempest_heat_cntt | functest | smoke | 22:32 | PASS | + | rally_sanity_cntt | functest | smoke | 17:16 | PASS | + | tempest_full_cntt | functest | smoke | 41:13 | PASS | + | tempest_scenario_cntt | functest | smoke | 08:57 | PASS | + | tempest_slow_cntt | functest | smoke | 35:58 | PASS | + +-------------------------------+------------------+---------------+------------------+----------------+ Benchmarking suite:: +--------------------+------------------+----------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +--------------------+------------------+----------------------+------------------+----------------+ - | rally_full | functest | benchmarking | 92:16 | PASS | - | rally_jobs | functest | benchmarking | 18:49 | PASS | - | vmtp | functest | benchmarking | 15:28 | PASS | - | shaker | functest | benchmarking | 24:04 | PASS | + | rally_full | functest | benchmarking | 93:03 | PASS | + | rally_jobs | functest | benchmarking | 27:05 | PASS | + | vmtp | functest | benchmarking | 17:56 | PASS | + | shaker | functest | benchmarking | 24:02 | PASS | +--------------------+------------------+----------------------+------------------+----------------+ +Benchmarking CNTT suite:: + + +-------------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+----------------------+------------------+----------------+ + | rally_full_cntt | functest | benchmarking | 89:52 | PASS | + | rally_jobs_cntt | functest | benchmarking | 19:39 | PASS | + | vmtp | functest | benchmarking | 16:59 | PASS | + | shaker | functest | benchmarking | 23:43 | PASS | + +-------------------------+------------------+----------------------+------------------+----------------+ + Vnf suite:: +----------------------+------------------+--------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +----------------------+------------------+--------------+------------------+----------------+ - | cloudify | functest | vnf | 03:49 | PASS | - | cloudify_ims | functest | vnf | 24:20 | PASS | - | heat_ims | functest | vnf | 32:13 | PASS | - | vyos_vrouter | functest | vnf | 14:55 | PASS | - | juju_epc | functest | vnf | 41:24 | PASS | + | cloudify | functest | vnf | 05:08 | PASS | + | cloudify_ims | functest | vnf | 24:46 | PASS | + | heat_ims | functest | vnf | 33:12 | PASS | + | vyos_vrouter | functest | vnf | 15:53 | PASS | + | juju_epc | functest | vnf | 27:52 | PASS | +----------------------+------------------+--------------+------------------+----------------+ Kubernetes healthcheck suite:: @@ -87,7 +118,8 @@ Kubernetes healthcheck suite:: +-------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +-------------------+------------------+---------------------+------------------+----------------+ - | k8s_smoke | functest | healthcheck | 01:09 | PASS | + | k8s_quick | functest | healthcheck | 00:18 | PASS | + | k8s_smoke | functest | healthcheck | 01:14 | PASS | +-------------------+------------------+---------------------+------------------+----------------+ Kubernetes smoke suite:: @@ -95,10 +127,38 @@ Kubernetes smoke suite:: +---------------------------+------------------+---------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +---------------------------+------------------+---------------+------------------+----------------+ - | xrally_kubernetes | functest | smoke | 22:04 | PASS | - | k8s_conformance | functest | smoke | 173:48 | PASS | + | k8s_conformance | functest | smoke | 94:26 | PASS | + | xrally_kubernetes | functest | smoke | 13:05 | PASS | +---------------------------+------------------+---------------+------------------+----------------+ +Kubernetes security suite:: + + +---------------------------+------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+------------------+------------------+----------------+ + | kube_hunter | functest | security | 00:19 | PASS | + | kube_bench_master | functest | security | 00:02 | PASS | + | kube_bench_node | functest | security | 00:01 | PASS | + +---------------------------+------------------+------------------+------------------+----------------+ + +Kubernetes benchmarking suite:: + + +--------------------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------------+------------------+----------------------+------------------+----------------+ + | xrally_kubernetes_full | functest | benchmarking | 34:16 | PASS | + +--------------------------------+------------------+----------------------+------------------+----------------+ + +Kubernetes cnf suite:: + + +-------------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+--------------+------------------+----------------+ + | k8s_vims | functest | cnf | 09:06 | PASS | + | helm_vims | functest | cnf | 08:54 | PASS | + | cnf_conformance | functest | cnf | 02:00 | PASS | + +-------------------------+------------------+--------------+------------------+----------------+ + Results are automatically pushed to the test results database, some additional result files are pushed to OPNFV artifact web sites. diff --git a/docs/testing/user/userguide/troubleshooting.rst b/docs/testing/user/userguide/troubleshooting.rst index 66e2433f6..d857ed4c4 100644 --- a/docs/testing/user/userguide/troubleshooting.rst +++ b/docs/testing/user/userguide/troubleshooting.rst @@ -44,8 +44,8 @@ follows:: The Functest CLI is designed to route a call to the corresponding internal python scripts, located in paths:: - /usr/lib/python3.7/site-packages/functest/opnfv_tests/openstack/vping/vping_ssh.py - /usr/lib/python3.7/site-packages/functest/opnfv_tests/openstack/vping/vping_userdata.py + /usr/lib/python3.8/site-packages/functest/opnfv_tests/openstack/vping/vping_ssh.py + /usr/lib/python3.8/site-packages/functest/opnfv_tests/openstack/vping/vping_userdata.py Notes: @@ -102,7 +102,7 @@ In this case, proceed to create it manually. These are some hints:: --protocol tcp --port-range-min 80 --port-range-max 80 --remote-ip-prefix 0.0.0.0/0 The next step is to create the instances. The image used is located in -*/home/opnfv/functest/data/cirros-0.4.0-x86_64-disk.img* and a Glance image is +*/home/opnfv/functest/data/cirros-0.5.1-x86_64-disk.img* and a Glance image is created with the name **functest-vping**. If booting the instances fails (i.e. the status is not **ACTIVE**), you can check why it failed by doing:: @@ -125,7 +125,7 @@ This test case creates a floating IP on the external network and assigns it to the second instance **opnfv-vping-2**. The purpose of this is to establish a SSH connection to that instance and SCP a script that will ping the first instance. This script is located in the repository under -/usr/lib/python3.7/site-packages/functest/opnfv_tests/openstack/vping/ping.sh +/usr/lib/python3.8/site-packages/functest/opnfv_tests/openstack/vping/ping.sh and takes an IP as a parameter. When the SCP is completed, the test will do a SSH call to that script inside the second instance. Some problems can happen here:: @@ -261,40 +261,28 @@ Functest offers a possibility to test a customized list of Tempest test cases. To enable that, add a new entry in docker/smoke/testcases.yaml on the "smoke" container with the following content:: - - - case_name: tempest_custom - project_name: functest - criteria: 100 - blocking: false - description: >- - The test case allows running a customized list of tempest - test cases - dependencies: - installer: '' - scenario: '' - run: - module: 'functest.opnfv_tests.openstack.tempest.tempest' - class: 'TempestCustom' - -Also, a list of the Tempest test cases must be provided to the container or -modify the existing one in -/usr/lib/python3.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt - -Example of custom list of tests 'my-custom-tempest-tests.txt':: - - tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke] - tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke] + - + case_name: tempest_custom + project_name: functest + criteria: 100 + blocking: false + description: >- + The test case allows running a customized list of tempest + test cases + run: + name: tempest_common + args: + mode: "tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops|\ + tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops" This is an example of running a customized list of Tempest tests in Functest:: sudo docker run --env-file env \ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ -v $(pwd)/images:/home/opnfv/functest/images \ - -v $(pwd)/my-custom-testcases.yaml:/usr/lib/python3.7/site-packages/functest/ci/testcases.yaml \ - -v $(pwd)/my-custom-tempest-tests.txt:/usr/lib/python3.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt \ + -v $(pwd)/my-custom-testcases.yaml:/usr/lib/python3.8/site-packages/xtesting/ci/testcases.yaml \ opnfv/functest-smoke run_tests -t tempest_custom - Rally ^^^^^ @@ -314,7 +302,7 @@ Possible scenarios are: To know more about what those scenarios are doing, they are defined in directory: -/usr/lib/python3.7/site-packages/functest/opnfv_tests/openstack/rally/scenario +/usr/lib/python3.8/site-packages/functest/opnfv_tests/openstack/rally/scenario For more info about Rally scenario definition please refer to the Rally official documentation. `[3]`_ |
