diff options
Diffstat (limited to 'docs')
32 files changed, 1024 insertions, 590 deletions
diff --git a/docs/com/css/theme/OPNFV-Berlin.css b/docs/com/css/theme/OPNFV-Berlin.css index 9f957d6f1..34f73b4bd 100644 --- a/docs/com/css/theme/OPNFV-Berlin.css +++ b/docs/com/css/theme/OPNFV-Berlin.css @@ -31,13 +31,13 @@ body { bottom: 1em; left: 1em; font-size: 0.5em; - + } .reveal .slides > section, .reveal .slides > section > section { line-height: 1.3; font-weight: inherit; } - + .reveal .opnfv-title { background-image: url("../../img/title-bg-berlin.png" no-repeat); } @@ -87,7 +87,7 @@ body { .reveal h1 { text-shadow: none; } - + /********************************************* * OTHER diff --git a/docs/com/css/theme/OPNFV.css b/docs/com/css/theme/OPNFV.css index 745a2a8de..c59c59f8d 100644 --- a/docs/com/css/theme/OPNFV.css +++ b/docs/com/css/theme/OPNFV.css @@ -30,13 +30,13 @@ body { bottom: 1em; left: 1em; font-size: 0.5em; - + } .reveal .slides > section, .reveal .slides > section > section { line-height: 1.3; font-weight: inherit; } - + .reveal .opnfv-title { background-image: url("../../img/title-bg.png" no-repeat); } @@ -83,7 +83,7 @@ body { .reveal h1 { text-shadow: none; } - + /********************************************* * OTHER diff --git a/docs/com/pres/Summit/Berlin-2016/conversation.html b/docs/com/pres/Summit/Berlin-2016/conversation.html index 356c2ad1e..5c1e18450 100644 --- a/docs/com/pres/Summit/Berlin-2016/conversation.html +++ b/docs/com/pres/Summit/Berlin-2016/conversation.html @@ -215,7 +215,7 @@ </div> <div class='footer'> - <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> + <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> </div> </div> diff --git a/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html b/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html index 97fa66c18..2b3b3b4ef 100644 --- a/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html +++ b/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html @@ -81,7 +81,7 @@ </section> <section data-markdown> ## so concretely... - * Validate scenarios for the release + * Validate scenarios for the release * Give confidence on OPNFV fresh releases * Ensure consistancy towards installers * Ensure End to End interoperability @@ -135,7 +135,7 @@ * 2 installers: Foreman/Fuel * Lots of manual operations (reporting, documentation) </section> - </section> + </section> <section> <section data-markdown> # Brahmaputra @@ -146,8 +146,8 @@ * 3 feature projects: doctor, promise, sdnvpn * 13 Scenarios * 4 Installers (Apex, Compass, Fuel, Joid) - * Test result collection - * Automatic dashboard + * Test result collection + * Automatic dashboard </section> <section> <h3>Jiras</h3> @@ -196,7 +196,7 @@ </tr> </tbody> </table> - * difference of duration due to POD and test suites + * difference of duration due to POD and test suites </section> <section data-markdown> @@ -204,7 +204,7 @@ * Tempest/Rally * harmonizing installer related OpenStack configuration differences * creating customized test lists - * main challenge was resolving SDN controller interworking problems + * main challenge was resolving SDN controller interworking problems * vIMS * complete but complex test case * very interesting to automate (pre MANO, most of Telco needs met there) @@ -212,7 +212,7 @@ </section> - <section> + <section> <section data-markdown> # Colorado </section> @@ -221,7 +221,7 @@ ## What's new? * New internal test cases: healthcheck, security * New controler: OpenContrail? - * New feature projects (domino, multisites, movie, parser, moon, copper, models, onos-sfc...) + * New feature projects (domino, multisites, movie, parser, moon, copper, models, onos-sfc...) * ARM Support (congratulations to ENEA team!) * Simplified feature project integration journey </section> @@ -255,19 +255,19 @@ * New VNFs (vEPC, vCDN, vWhatever...) * Multi sites (e.g bgpvpn with different back ends) * Better test coverage - </section> + </section> <section data-markdown> ## A Functional testing As a Service framework * Scenario owner can select their relevant test cases * Test duration estimation * Agile dashboarding * Analytics - </section> + </section> <section data-markdown> ## Upstream * Rally: time to give back more - * Functest description for ETSI? - </section> + * Functest description for ETSI? + </section> </section> <section> diff --git a/docs/com/pres/Summit/Berlin-2016/testapi.html b/docs/com/pres/Summit/Berlin-2016/testapi.html index c40637cbf..5e56d5052 100644 --- a/docs/com/pres/Summit/Berlin-2016/testapi.html +++ b/docs/com/pres/Summit/Berlin-2016/testapi.html @@ -69,7 +69,7 @@ * Test cases * Results </section> - + <section data-markdown> ## Status Currently * From many projects @@ -87,7 +87,7 @@ </aside> </script> </section> - + <section data-markdown> ## Help building * Dashboards @@ -135,8 +135,8 @@ <li>Scripts to support databse backup/restore/update</li> <li>Support installation</li> <aside class='notes'> - So you can discover and understand the capabilities of the service without - access to source code, documentation, or through network traffic inspection, + So you can discover and understand the capabilities of the service without + access to source code, documentation, or through network traffic inspection, and also you can interact with the TestAPI directly through swagger website. </aside> </ul> @@ -148,12 +148,12 @@ <section data-markdown> # API Evoluation </section> - + <section> <h2> URI changes...</h2> - <div style="text-align:left""> + <div style="text-align:left""> <p> testresults.opnfv.org/<span style="color:lightblue">TestAPI</span> => <br>testresults.opnfv.org/<span style="color:yellow">test/api/v1</span> </p> - + <p> /test/api/v1/<b>pods</b></p> <p> /test/api/v1/<b>projects</b></p> <p> /test/api/v1/projects/qtip/<b>cases</b></p> @@ -200,7 +200,7 @@ <section> <h2>unit tests</h2> <pre><code class="hljs" data-trim contenteditable> -umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh +umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh Tests running... WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms diff --git a/docs/com/pres/dockerslicing/dockerslicing.md b/docs/com/pres/dockerslicing/dockerslicing.md index a66453248..0d1ce5e21 100644 --- a/docs/com/pres/dockerslicing/dockerslicing.md +++ b/docs/com/pres/dockerslicing/dockerslicing.md @@ -63,7 +63,7 @@ ### 8 Functest containers -```bash +```shell $ sudo docker search opnfv |grep functest- opnfv/functest-core OPNFV Functest core image opnfv/functest-restapi OPNFV Functest restapi image diff --git a/docs/com/pres/euphrates_functest_evolution/euphrates.md b/docs/com/pres/euphrates_functest_evolution/euphrates.md index 49aab16cd..2c3105539 100644 --- a/docs/com/pres/euphrates_functest_evolution/euphrates.md +++ b/docs/com/pres/euphrates_functest_evolution/euphrates.md @@ -120,7 +120,7 @@ export OS_IDENTITY_API_VERSION=3 #### Retrieve all the images ``` -mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* +mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | sh -s -- images && ls -1 images/* images/CentOS-7-aarch64-GenericCloud.qcow2 images/CentOS-7-aarch64-GenericCloud.qcow2.xz images/CentOS-7-x86_64-GenericCloud.qcow2 diff --git a/docs/com/pres/gambia/gambia.md b/docs/com/pres/gambia/gambia.md index cf5f0f1bf..4a65e9905 100644 --- a/docs/com/pres/gambia/gambia.md +++ b/docs/com/pres/gambia/gambia.md @@ -85,7 +85,7 @@ the classical ONAP testing virtual machine (**> 1GB**). - all ONAP Robot Framework files retrieved from the original repositories - testcases.yaml describing the testcases -[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/) +[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/) ### What's new in Xtesting? @@ -94,7 +94,7 @@ the classical ONAP testing virtual machine (**> 1GB**). full CI/CD toolchains in few minutes** (Jenkins, Minio, TestAPI, MongoDB and Docker registry) -```bash +```shell virtualenv xtesting . xtesting/bin/activate pip install ansible docker diff --git a/docs/com/pres/oran/ftth.png b/docs/com/pres/oran/ftth.png Binary files differnew file mode 100644 index 000000000..5b441d1de --- /dev/null +++ b/docs/com/pres/oran/ftth.png diff --git a/docs/com/pres/oran/oran.md b/docs/com/pres/oran/oran.md new file mode 100644 index 000000000..87fba1306 --- /dev/null +++ b/docs/com/pres/oran/oran.md @@ -0,0 +1,90 @@ +--- +author: Cédric Ollivier +title: OPNFV/CNTT CI assets +date: 2020/12/01 +--- +## Continuous integration + + +### A few CNTT reqs + +- OPNFV has built a complete CI/CD toolchain for continuously deploying and + testing cloud infrastructure +- Reference conformance only requires for the local deployment of the + same components +- all test cases must be delivered as **Docker containers** including the + common test case execution and the unified way to manage all the interactions + +**Integrate smoothly and deploy fastly** + + +### How OPNFV/CNTT helps? + +- **Functest** offers a collection of state-of-the-art virtual infrastructure + test suites +- **Xtesting** helps assembling sparse test cases and accelerating the adoption + of CI/CD best practices +- **XtestingCI** eases deploying anywhere plug-and-play CI/CD toolchains in a + few commands +- **CNTT** defines conformance suites and its playbooks leveraging this 3 + testing frameworks + +**Leverage best opensource tools and practices** + + +### Testing [FTTH](https://en.wikipedia.org/wiki/Fiber_to_the_x) equipment + + + + +### Deploy your CI toolchain + +``` +virtualenv xtesting +. xtesting/bin/activate +pip install ansible +ansible-galaxy install collivier.xtesting +git clone https://gerrit.opnfv.org/gerrit/functest-xtesting functest-xtesting-src +ansible-playbook functest-xtesting-src/ansible/site.yml +deactivate +rm -rf functest-xtesting-src xtesting +``` + +**More scenarios in [[1]](https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004)** + + + +## Testing + + +### Verif', Compliance and Interop' + +- verification allows skipping test if optional services or capabilities are + missing +- compliance forces the full API descriptions as currently proposed by + CNTT and then skips all optional capability testing +- the opensource certifications are driven by simple interoperability testing + (RefStack and OVP are about less than 10% of Functest. **trustability?**) + +**Fine tune the same test frameworks** + + +### [CNTT RC1](https://build.opnfv.org/ci/view/functest/job/functest-leguer-daily/12/) + + + + + +## Conclusion + + +### Key takeaways + +- **test integration requirements** are crucial to smoothly assemble all + test cases +- leverage **existing** OPNFV testing knowledge (projects) and experience + (history) by utilising the OPNFV toolchain design already in-place +- **3000+** functional tests, **3 hours** upstream API and dataplane benchmarks + and VNFs automatically onboarded and tested **can be reused asis** + +**Keep it simple and do not reivent the wheel** diff --git a/docs/com/pres/oran/rc1.png b/docs/com/pres/oran/rc1.png Binary files differnew file mode 100644 index 000000000..9179f3bb6 --- /dev/null +++ b/docs/com/pres/oran/rc1.png diff --git a/docs/com/pres/rfp/index.html b/docs/com/pres/rfp/index.html new file mode 100644 index 000000000..538c0c8c8 --- /dev/null +++ b/docs/com/pres/rfp/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>OPNFV and CNTT in Orange RFP</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="rfp.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/rfp/rfp.md b/docs/com/pres/rfp/rfp.md new file mode 100644 index 000000000..66f7ba9e7 --- /dev/null +++ b/docs/com/pres/rfp/rfp.md @@ -0,0 +1,97 @@ +# OPNFV and CNTT in Orange RFP + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/09/19 + + + +## Vision and contributions + + +### Our guidelines + +- **automate** to bring determinism and to meet the new + software release rate +- test all software layers **independently** (OpenStack, Kubernetes and VNFs) +- run all deployment and verification jobs in **our** continous integration + chains +- leverage best **opensource** tools and practices + +**Integrate smoothly and deploy everywhere fastly** + + +### How OpenSource helps? + +- **Functest** offers a collection of state-of-the-art virtual infrastructure + test suites +- **Xtesting** helps assembling sparse test cases and accelerating the adoption + of CI/CD best practices +- **XtestingCI** eases deploying anywhere plug-and-play CI/CD toolchains in a + few commands +- **CNTT** defines conformance suites and its playbooks leveraging this 3 + softwares + +**Any contribution is more than welcome! +[[1]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + + +## CNTT/OPNFV in Orange + + +### A couple of RFP requirements + +- the **full** CNTT reference conformance for OpenStack results and outputs + (Orange CNTT Field Trial is in a very good shape + [[1]](http://testresults.opnfv.org/functest/field_trial/) + [[2]](https://www.linkedin.com/pulse/cntt-field-trials-c%C3%A9dric-ollivier/)) +- the **success** of the Functest Kubernetes test suites (now released as + part of CNTT RC2 Baraque) +- **first** VNF test cases running in **our** continuous integration chain + thanks to Xtesting and XtestingCI + +**It's implementing Orange and CNTT targets** + + +### Orange CNTT RC1 Field Trial + +- helped detecting a couple of issues in CNTT RC1 +- integrated cinder backup and nova instance_password in Orange IaaS +- to fix 10 remaining single test failures (out 2000+ functional tests, 3 hours + benchmarking and 3 VNFs automatically onboarded and tested) +- to enhance Functest juju_epc to pass proxies + +**99,999%** + + +### Wish list + +- to integrate **more benchmarks** in CNTT conformance (e.g. disk benchmarking) +- to switch from the current Kubernetes interoperability testing to a **true** + CNTT conformance suite +- to build the first **VNF and CNF** conformance suites (**high priority**) + +**We need your contribution helps! +[[1]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + + +## Conclusion + + +### Take aways + +- Orange leverages OPNFV and CNTT in RFP +- we keep contributing in both specification and implementation streams for + the success of Network Function Virtualization +- we expect more OPNFV and CNTT contributions especially for VNF and CNF + conformance suites, our initial CNTT target + +**Try CNTT reference suites, you will love them!** + + + +## Thank you! diff --git a/docs/com/pres/vevent202010/index.html b/docs/com/pres/vevent202010/index.html new file mode 100644 index 000000000..a24c721d2 --- /dev/null +++ b/docs/com/pres/vevent202010/index.html @@ -0,0 +1,52 @@ +<html> +<head> +<title>Orange CNTT RC1 Field Trial feedbacks</title> +<meta name="author" content="Cédric Ollivier"> +<meta name="viewport" + content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> +<link rel="stylesheet" href="../reveal.js/css/reveal.css"> +<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> +<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> +<script> +var link = document.createElement( 'link' ); +link.rel = 'stylesheet'; +link.type = 'text/css'; +link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; +document.getElementsByTagName( 'head' )[0].appendChild( link ); +</script> +</head> +<body> + <div class="reveal"> + <div class="slides"> + <section data-markdown="vevent202010.md" data-separator="^\n\n\n" + data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> + </div> + </div> + <script src="../reveal.js/lib/js/head.min.js"></script> + <script src="../reveal.js/js/reveal.js"></script> + <script> + Reveal.initialize({ + dependencies : [ { + src : '../reveal.js/plugin/markdown/marked.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src : '../reveal.js/plugin/markdown/markdown.js', + condition : function() { + return !!document.querySelector('[data-markdown]'); + } + }, { + src: '../reveal.js/plugin/highlight/highlight.js', + async: true, + callback: function() { + hljs.initHighlightingOnLoad(); + } + }, { + src: '../reveal.js/plugin/notes/notes.js', + async: true + } ] + }); + </script> +</body> +</html> diff --git a/docs/com/pres/vevent202010/vevent202010.md b/docs/com/pres/vevent202010/vevent202010.md new file mode 100644 index 000000000..b128210ac --- /dev/null +++ b/docs/com/pres/vevent202010/vevent202010.md @@ -0,0 +1,62 @@ +# Orange CNTT RC1 Field Trial + +[Cédric Ollivier](mailto:cedric.ollivier@orange.com) + +2020/10/14 + + + +## CNTT RC1 Field Trial + + +### Orange CNTT RC1 Field Trial + +- helped detecting a couple of issues in CNTT RC1 +- integrated cinder backup and nova instance_password in Orange IaaS +- to fix 10 remaining single test failures (out 2000+ functional tests, 3 hours + benchmarking and 3 VNFs automatically onboarded and tested) +- to enhance Functest juju_epc to pass proxies + +**99,999%** + + +### Orange CNTT RC1 feedbacks + +- we easily executed the CNTT RC1 playbook asis and no clear bug was found in + CNTT RC1 Baldy +- CNTT RC1 is now used in our Orange IaaS verification in addition to the + classical Functest containers +- CNTT RC1 is also executed to verify all deployments before onboarding any VNF +- CNTT RC1 Baldy and CNTT RC2 Baraque are listed in **Orange RFP requirements** + (see last ONES + [[1]](http://testresults.opnfv.org/functest/ONES2020NA_OPNFV-CNTT_RFP.1080p.mp4)) + + + +## What else? + + +### Wish list + +- to integrate **more benchmarks** in CNTT conformance (e.g. disk benchmarking) +- to switch from the current Kubernetes interoperability testing to a **true** + CNTT conformance suite +- to build the first **VNF and CNF** conformance suites (**high priority**) + +**We need your contribution helps! +[[2]](https://www.linkedin.com/pulse/call-functest-cntt-rc1-contributions-c%25C3%25A9dric-ollivier/) +[[3]](https://www.linkedin.com/pulse/call-functest-cntt-rc2-contributions-c%25C3%25A9dric-ollivier/)** + + +### New Functest opportunities + +- to implement live monitoring of your OpenStack and Kubernetes deployments via + a subset of Functest (healthcheck? new Rally tasks?) +- to implement new functest testcases to validate and verify OpenStack upgrades + (new Rally tasks)? + +**Try Functest Leguer, you will love it [[4]](https://www.linkedin.com/pulse/opnfv-functest-leguer-out-c%25C3%25A9dric-ollivier/)** + + + +## Thank you! diff --git a/docs/release/release-notes/functest-release.rst b/docs/release/release-notes/functest-release.rst index f6c3d120c..b5e228caa 100644 --- a/docs/release/release-notes/functest-release.rst +++ b/docs/release/release-notes/functest-release.rst @@ -68,9 +68,17 @@ Kubernetes The internal test cases are: + * k8s_quick * k8s_smoke - * xrally_kubernetes * k8s_conformance + * xrally_kubernetes + * kube_hunter + * kube_bench_master + * kube_bench_node + * xrally_kubernetes_full + * k8s_vims + * helm_vims + * cnf_conformance Release Data ============ @@ -100,6 +108,9 @@ Software * https://hub.docker.com/r/opnfv/functest-kubernetes-healthcheck * https://hub.docker.com/r/opnfv/functest-kubernetes-smoke + * https://hub.docker.com/r/opnfv/functest-kubernetes-security + * https://hub.docker.com/r/opnfv/functest-kubernetes-benchmarking + * https://hub.docker.com/r/opnfv/functest-kubernetes-cnf Docker tag for master: latest @@ -130,8 +141,8 @@ Key benefits * all testcases can run in parallel (tested with 4 executors in our gates) * no remaining resources detected in our gates after multiple runs -.. _`Functest toolchains`: https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004 -.. _`Raspberry PI`: https://wiki.opnfv.org/display/functest/Run+Functest+containers+on+Raspberry+PI +.. _`Functest toolchains`: https://github.com/collivier/ansible-role-xtesting +.. _`Raspberry PI`: https://github.com/opnfv/functest/blob/master/PI.md Code quality ------------ @@ -142,9 +153,8 @@ Code quality Useful links ============ - * wiki project page: https://wiki.opnfv.org/display/functest/Opnfv+Functional+Testing * Functest git repository: https://github.com/opnfv/functest - * Functest CI dashboard: https://build.opnfv.org/ci/view/functest/ + * Functest CI dashboard: http://104.154.71.112:8080/view/functest/ * JIRA dashboard: https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611 * Functest IRC channel: #opnfv-functest * Reporting page: http://testresults.opnfv.org/reporting/master/functest/functest.html diff --git a/docs/release/release-notes/index.rst b/docs/release/release-notes/index.rst index c19d55610..25c2cbdfb 100644 --- a/docs/release/release-notes/index.rst +++ b/docs/release/release-notes/index.rst @@ -10,5 +10,3 @@ Functest Release Notes :maxdepth: 4 functest-release.rst - -Build date: |today| diff --git a/docs/results/euphrates/5.0/apex.html b/docs/results/euphrates/5.0/apex.html index 2a75054c3..34f0c1503 100644 --- a/docs/results/euphrates/5.0/apex.html +++ b/docs/results/euphrates/5.0/apex.html @@ -307,13 +307,13 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=http://testresultS.opnfv.org/reporting>os-odl-fdio-ha</a></td> <td><div id="gaugeScenario5"></div></td> @@ -327,7 +327,7 @@ $(document).ready(function (){ <td>4/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/108/console>os-nosdn-ovs_dpdk-ha</a></td> <td><div id="gaugeScenario8"></div></td> @@ -341,9 +341,9 @@ $(document).ready(function (){ <td>16/18</td> <td>2</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/139/console>os-nosdn-fdio-ha</a></td> <td><div id="gaugeScenario12"></div></td> @@ -357,11 +357,11 @@ $(document).ready(function (){ <td>18/20</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/121/console>os-nosdn-bar-ha</a></td> <td><div id="gaugeScenario17"></div></td> @@ -369,7 +369,7 @@ $(document).ready(function (){ <td>26/27</td> <td>4</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/134/console>os-nosdn-nofeature-ha</a></td> <td><div id="gaugeScenario19"></div></td> @@ -413,9 +413,9 @@ $(document).ready(function (){ <td>8/30</td> <td>4</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/114/console>os-ovn-nofeature-noha</a></td> <td><div id="gaugeScenario7"></div></td> @@ -423,9 +423,9 @@ $(document).ready(function (){ <td>6/18</td> <td>2</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/136/console>os-odl-nofeature-noha</a></td> <td><div id="gaugeScenario10"></div></td> @@ -439,9 +439,9 @@ $(document).ready(function (){ <td>4/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=http://testresultS.opnfv.org/reporting>os-odl-fdio_dvr-noha</a></td> <td><div id="gaugeScenario14"></div></td> @@ -461,7 +461,7 @@ $(document).ready(function (){ <td>18/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-euphrates/128/console>os-nosdn-calipso-noha</a></td> <td><div id="gaugeScenario18"></div></td> @@ -469,7 +469,7 @@ $(document).ready(function (){ <td>18/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr> </table> @@ -489,31 +489,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -534,31 +534,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -582,37 +582,37 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> FDS - + </th><th> SNAPS - + </th><th> Domino * @@ -636,34 +636,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Doctor * @@ -693,37 +693,37 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> FDS - + </th><th> SNAPS - + </th><th> Domino * @@ -747,31 +747,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -792,31 +792,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -843,31 +843,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -888,31 +888,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Doctor * @@ -942,34 +942,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Doctor * @@ -996,31 +996,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -1041,31 +1041,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -1089,34 +1089,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Doctor * @@ -1143,37 +1143,37 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> FDS - + </th><th> SNAPS - + </th><th> Domino * @@ -1197,31 +1197,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -1251,31 +1251,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -1302,31 +1302,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -1356,31 +1356,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * @@ -1407,31 +1407,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Doctor * diff --git a/docs/results/euphrates/5.0/compass.html b/docs/results/euphrates/5.0/compass.html index d62ba4d2e..b7f7a20ea 100644 --- a/docs/results/euphrates/5.0/compass.html +++ b/docs/results/euphrates/5.0/compass.html @@ -223,11 +223,11 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-euphrates/81/console>os-odl_l3-nofeature-ha</a></td> <td><div id="gaugeScenario4"></div></td> @@ -247,7 +247,7 @@ $(document).ready(function (){ <td>26/30</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-euphrates/72/console>os-nosdn-ovs_dpdk-ha</a></td> <td><div id="gaugeScenario8"></div></td> @@ -255,9 +255,9 @@ $(document).ready(function (){ <td>2/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-baremetal-daily-euphrates/24/console>os-odl_l2-moon-ha</a></td> <td><div id="gaugeScenario11"></div></td> @@ -301,11 +301,11 @@ $(document).ready(function (){ <td>24/30</td> <td>4</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-euphrates/86/console>os-nosdn-nofeature-noha</a></td> <td><div id="gaugeScenario7"></div></td> @@ -313,7 +313,7 @@ $(document).ready(function (){ <td>26/27</td> <td>6</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-euphrates/82/console>os-nosdn-ovs_dpdk-noha</a></td> <td><div id="gaugeScenario9"></div></td> @@ -327,9 +327,9 @@ $(document).ready(function (){ <td>14/18</td> <td>3</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr> </table> @@ -349,31 +349,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -397,34 +397,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> SFC * @@ -451,34 +451,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * @@ -502,31 +502,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * @@ -550,31 +550,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -598,34 +598,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> SFC * @@ -652,31 +652,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -700,31 +700,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -745,31 +745,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -790,31 +790,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * @@ -838,34 +838,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * @@ -889,31 +889,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * diff --git a/docs/results/euphrates/5.0/daisy.html b/docs/results/euphrates/5.0/daisy.html index 6de8d0895..09580fc19 100644 --- a/docs/results/euphrates/5.0/daisy.html +++ b/docs/results/euphrates/5.0/daisy.html @@ -127,9 +127,9 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr> </table> @@ -149,31 +149,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Domino * @@ -206,34 +206,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Domino * diff --git a/docs/results/euphrates/5.0/fuel@aarch64.html b/docs/results/euphrates/5.0/fuel@aarch64.html index c8a2f2db0..aa9f1eb66 100644 --- a/docs/results/euphrates/5.0/fuel@aarch64.html +++ b/docs/results/euphrates/5.0/fuel@aarch64.html @@ -115,7 +115,7 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-fuel-armband-baremetal-arm-daily-euphrates/31/console>os-nosdn-nofeature-ha</a></td> <td><div id="gaugeScenario2"></div></td> @@ -147,9 +147,9 @@ $(document).ready(function (){ <td>1/9</td> <td>1</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr> </table> @@ -169,31 +169,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -220,31 +220,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -283,34 +283,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Promise * diff --git a/docs/results/euphrates/5.0/fuel@x86.html b/docs/results/euphrates/5.0/fuel@x86.html index 3b918dd74..52b67b37f 100644 --- a/docs/results/euphrates/5.0/fuel@x86.html +++ b/docs/results/euphrates/5.0/fuel@x86.html @@ -151,7 +151,7 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-fuel-baremetal-daily-euphrates/40/console>os-odl-nofeature-ha</a></td> <td><div id="gaugeScenario2"></div></td> @@ -159,9 +159,9 @@ $(document).ready(function (){ <td>29/30</td> <td>6</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-fuel-baremetal-daily-euphrates/38/console>os-nosdn-ovs-ha</a></td> <td><div id="gaugeScenario5"></div></td> @@ -193,7 +193,7 @@ $(document).ready(function (){ <td>25/27</td> <td>10</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-fuel-virtual-daily-euphrates/91/console>os-odl-nofeature-noha</a></td> <td><div id="gaugeScenario3"></div></td> @@ -207,9 +207,9 @@ $(document).ready(function (){ <td>27/27</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr> </table> @@ -229,31 +229,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -280,34 +280,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Promise * @@ -334,34 +334,34 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> ODL - + </th><th> SNAPS - + </th><th> Promise * @@ -388,31 +388,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -439,31 +439,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -490,31 +490,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * diff --git a/docs/results/euphrates/5.0/joid.html b/docs/results/euphrates/5.0/joid.html index d58efcca1..9b6d0f4d0 100644 --- a/docs/results/euphrates/5.0/joid.html +++ b/docs/results/euphrates/5.0/joid.html @@ -163,7 +163,7 @@ $(document).ready(function (){ <th width="10%">Iteration</th> </tr> <tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/84/console>os-ocl-nofeature-ha</a></td> <td><div id="gaugeScenario2"></div></td> @@ -171,7 +171,7 @@ $(document).ready(function (){ <td>3/27</td> <td>10</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/67/console>os-nosdn-openbaton-ha</a></td> <td><div id="gaugeScenario4"></div></td> @@ -185,7 +185,7 @@ $(document).ready(function (){ <td>2/15</td> <td>4</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/81/console>os-nosdn-nofeature-ha</a></td> <td><div id="gaugeScenario7"></div></td> @@ -211,7 +211,7 @@ $(document).ready(function (){ <td>3/27</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/90/console>os-nosdn-nofeature-noha</a></td> <td><div id="gaugeScenario3"></div></td> @@ -219,9 +219,9 @@ $(document).ready(function (){ <td>15/27</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> - + </tr><tr class="tr-ok"> <td><a href=https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-euphrates/88/console>os-nosdn-lxd-noha</a></td> <td><div id="gaugeScenario6"></div></td> @@ -229,7 +229,7 @@ $(document).ready(function (){ <td>3/15</td> <td>9</td> </tr><tr class="tr-ok"> - + </tr> </table> @@ -249,31 +249,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -294,31 +294,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th> </tr> <tr class="tr-weather-weather"> @@ -339,31 +339,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -390,31 +390,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * @@ -441,19 +441,19 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> vPing (ssh) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> Promise * @@ -480,19 +480,19 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> vPing (ssh) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> Promise * @@ -519,31 +519,31 @@ $(document).ready(function (){ <tr> <th> Health (connection) - + </th><th> Health (api) - + </th><th> Health (dhcp) - + </th><th> vPing (ssh) - + </th><th> vPing (userdata) - + </th><th> Tempest (smoke) - + </th><th> Rally (smoke) - + </th><th> Refstack - + </th><th> SNAPS - + </th><th> Promise * diff --git a/docs/results/js/default.css b/docs/results/js/default.css index e32fa5fba..cbb564326 100644 --- a/docs/results/js/default.css +++ b/docs/results/js/default.css @@ -55,7 +55,7 @@ td{ background-color: #0095a2; } -h1 { +h1 { display: block; font-size: 2em; margin-top: 0.67em; diff --git a/docs/results/js/trend.js b/docs/results/js/trend.js index f24213382..47ee4d906 100644 --- a/docs/results/js/trend.js +++ b/docs/results/js/trend.js @@ -63,13 +63,13 @@ var trend = function(container, trend_data) { .attr("d", valueline(trend_data)) .attr("stroke", "steelblue") .attr("fill", "none"); - + trend_svg.selectAll(".dot") .data(trend_data) .enter().append("circle") .attr("r", 2.5) .attr("cx", function(d) { return trend_x(d.date); }) - .attr("cy", function(d) { return trend_y(d.score); }); + .attr("cy", function(d) { return trend_y(d.score); }); return trend; } diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index 94452dcf4..f4dcea507 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -14,6 +14,7 @@ ci Clearwater cloudify Cloudify +cnf cntt conf config @@ -128,6 +129,7 @@ vEPC verifier versioned vIMS +vims Virtualised Virtualized vm diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst index 3978a444c..da5485ef0 100644 --- a/docs/testing/developer/devguide/index.rst +++ b/docs/testing/developer/devguide/index.rst @@ -208,13 +208,6 @@ Additional dashboarding is managed at the testing group level, see `[5]`_ for details. -======= -How TOs -======= - -See How to section on Functest wiki `[6]`_ - - ========== References ========== @@ -229,6 +222,4 @@ _`[4]`: https://wiki.opnfv.org/display/functest/2017+Beijing?preview=%2F11699623 _`[5]`: https://lfanalytics.io/projects/lfn%2Fopnfv/dashboard -_`[6]`: https://wiki.opnfv.org/pages/viewpage.action?pageId=7768932 - IRC support chan: #opnfv-functest diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index d63ddeeb9..47c25adb1 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -13,7 +13,6 @@ Functest Dockers for OpenStack deployment ----------------------------------------- Docker images are available on the dockerhub: - * opnfv/functest-core * opnfv/functest-healthcheck * opnfv/functest-smoke * opnfv/functest-smoke-cntt @@ -53,7 +52,7 @@ See section on OpenStack credentials for details. Create a directory for the different images (attached as a Docker volume):: - mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* + mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | sh -s -- images && ls -1 images/* images/cirros-0.5.1-aarch64-disk.img images/cirros-0.5.1-x86_64-disk.img @@ -124,9 +123,12 @@ Results shall be displayed as follows:: | tempest_full | functest | smoke | 41:52 | PASS | | tempest_scenario | functest | smoke | 08:42 | PASS | | tempest_slow | functest | smoke | 43:42 | PASS | - | patrole | functest | smoke | 02:42 | PASS | + | patrole_admin | functest | smoke | 21:06 | PASS | + | patrole_member | functest | smoke | 21:23 | PASS | + | patrole_reader | functest | smoke | 21:56 | PASS | | tempest_barbican | functest | smoke | 02:30 | PASS | | tempest_octavia | functest | smoke | 00:00 | SKIP | + | tempest_cyborg | functest | smoke | 00:00 | SKIP | +---------------------------+------------------+---------------+------------------+----------------+ Note: if the scenario does not support some tests, they are indicated as SKIP. @@ -228,9 +230,11 @@ Functest Dockers for Kubernetes deployment ------------------------------------------ Docker images are available on the dockerhub: - * opnfv/functest-kubernetes-core * opnfv/functest-kubernetes-healthcheck * opnfv/functest-kubernetes-smoke + * opnfv/functest-kubernetes-security + * opnfv/functest-kubernetes-benchmarking + * opnfv/functest-kubernetes-cnf Preparing your environment ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -256,7 +260,8 @@ Results shall be displayed as follows:: +-------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +-------------------+------------------+---------------------+------------------+----------------+ - | k8s_smoke | functest | healthcheck | 01:09 | PASS | + | k8s_quick | functest | healthcheck | 00:18 | PASS | + | k8s_smoke | functest | healthcheck | 01:14 | PASS | +-------------------+------------------+---------------------+------------------+----------------+ Testing smoke suite @@ -273,10 +278,65 @@ Results shall be displayed as follows:: +---------------------------+------------------+---------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +---------------------------+------------------+---------------+------------------+----------------+ - | xrally_kubernetes | functest | smoke | 22:04 | PASS | - | k8s_conformance | functest | smoke | 173:48 | PASS | + | k8s_conformance | functest | smoke | 94:26 | PASS | + | xrally_kubernetes | functest | smoke | 13:05 | PASS | +---------------------------+------------------+---------------+------------------+----------------+ +Testing security suite +^^^^^^^^^^^^^^^^^^^^^^ + +Run smoke suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-security + +Results shall be displayed as follows:: + + +---------------------------+------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+------------------+------------------+----------------+ + | kube_hunter | functest | security | 00:19 | PASS | + | kube_bench_master | functest | security | 00:02 | PASS | + | kube_bench_node | functest | security | 00:01 | PASS | + +---------------------------+------------------+------------------+------------------+----------------+ + +Testing benchmarking suite +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Run benchmarking suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-benchmarking + +Results shall be displayed as follows:: + + +--------------------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------------+------------------+----------------------+------------------+----------------+ + | xrally_kubernetes_full | functest | benchmarking | 34:16 | PASS | + +--------------------------------+------------------+----------------------+------------------+----------------+ + +Testing cnf suite +^^^^^^^^^^^^^^^^^ + +Run cnf suite:: + + sudo docker run -it --env-file env \ + -v $(pwd)/config:/root/.kube/config \ + opnfv/functest-kubernetes-cnf + +Results shall be displayed as follows:: + + +-------------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+--------------+------------------+----------------+ + | k8s_vims | functest | cnf | 09:06 | PASS | + | helm_vims | functest | cnf | 08:54 | PASS | + | cnf_conformance | functest | cnf | 02:00 | PASS | + +-------------------------+------------------+--------------+------------------+----------------+ + Environment variables ===================== diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index 07ab0e62d..1e73cd622 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -40,7 +40,7 @@ References `[11]`_: Robot Framework web site -`[13]`_: SNAPS wiki +`[13]`_: SNAPS `[14]`_: vRouter @@ -64,12 +64,12 @@ IRC support chan: #opnfv-functest .. _`[9]`: https://github.com/openstack/interop .. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst .. _`[11]`: https://robotframework.org/ -.. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO +.. _`[13]`: https://git.opnfv.org/snaps/ .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter .. _`[15]`: https://aptira.com/testing-openstack-tempest-part-1/ .. _`[16]`: http://testresults.opnfv.org/test/ .. _`OPNFV main site`: https://www.opnfv.org/ -.. _`Functest page`: https://wiki.opnfv.org/display/functest/Opnfv+Functional+Testing +.. _`Functest page`: https://github.com/opnfv/functest/ .. _`OpenRC`: http://docs.openstack.org/user-guide/common/cli_set_environment_variables_using_openstack_rc.html .. _`Rally installation procedure`: https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html .. _`config_functest.yaml` : https://github.com/opnfv/functest/blob/master/functest/ci/config_functest.yaml diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst index 7c4aba0a7..98247d488 100644 --- a/docs/testing/user/userguide/test_details.rst +++ b/docs/testing/user/userguide/test_details.rst @@ -405,7 +405,7 @@ The kubernetes testcases are distributed across various Tiers: .. _`[9]`: https://github.com/openstack/interop .. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst .. _`[11]`: https://robotframework.org/ -.. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO +.. _`[13]`: https://git.opnfv.org/snaps/ .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter .. _`[15]`: https://github.com/RebacaInc/abot_charm .. _`[16]`: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md diff --git a/docs/testing/user/userguide/test_results.rst b/docs/testing/user/userguide/test_results.rst index 8d8f79fe6..10f87d8ec 100644 --- a/docs/testing/user/userguide/test_results.rst +++ b/docs/testing/user/userguide/test_results.rst @@ -56,9 +56,12 @@ Smoke suite:: | tempest_full | functest | smoke | 41:52 | PASS | | tempest_scenario | functest | smoke | 08:42 | PASS | | tempest_slow | functest | smoke | 43:42 | PASS | - | patrole | functest | smoke | 02:42 | PASS | + | patrole_admin | functest | smoke | 21:06 | PASS | + | patrole_member | functest | smoke | 21:23 | PASS | + | patrole_reader | functest | smoke | 21:56 | PASS | | tempest_barbican | functest | smoke | 02:30 | PASS | | tempest_octavia | functest | smoke | 00:00 | SKIP | + | tempest_cyborg | functest | smoke | 00:00 | SKIP | +---------------------------+------------------+---------------+------------------+----------------+ Smoke CNTT suite:: @@ -115,7 +118,8 @@ Kubernetes healthcheck suite:: +-------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +-------------------+------------------+---------------------+------------------+----------------+ - | k8s_smoke | functest | healthcheck | 01:09 | PASS | + | k8s_quick | functest | healthcheck | 00:18 | PASS | + | k8s_smoke | functest | healthcheck | 01:14 | PASS | +-------------------+------------------+---------------------+------------------+----------------+ Kubernetes smoke suite:: @@ -123,10 +127,38 @@ Kubernetes smoke suite:: +---------------------------+------------------+---------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +---------------------------+------------------+---------------+------------------+----------------+ - | xrally_kubernetes | functest | smoke | 22:04 | PASS | - | k8s_conformance | functest | smoke | 173:48 | PASS | + | k8s_conformance | functest | smoke | 94:26 | PASS | + | xrally_kubernetes | functest | smoke | 13:05 | PASS | +---------------------------+------------------+---------------+------------------+----------------+ +Kubernetes security suite:: + + +---------------------------+------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +---------------------------+------------------+------------------+------------------+----------------+ + | kube_hunter | functest | security | 00:19 | PASS | + | kube_bench_master | functest | security | 00:02 | PASS | + | kube_bench_node | functest | security | 00:01 | PASS | + +---------------------------+------------------+------------------+------------------+----------------+ + +Kubernetes benchmarking suite:: + + +--------------------------------+------------------+----------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +--------------------------------+------------------+----------------------+------------------+----------------+ + | xrally_kubernetes_full | functest | benchmarking | 34:16 | PASS | + +--------------------------------+------------------+----------------------+------------------+----------------+ + +Kubernetes cnf suite:: + + +-------------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------+------------------+--------------+------------------+----------------+ + | k8s_vims | functest | cnf | 09:06 | PASS | + | helm_vims | functest | cnf | 08:54 | PASS | + | cnf_conformance | functest | cnf | 02:00 | PASS | + +-------------------------+------------------+--------------+------------------+----------------+ + Results are automatically pushed to the test results database, some additional result files are pushed to OPNFV artifact web sites. diff --git a/docs/testing/user/userguide/troubleshooting.rst b/docs/testing/user/userguide/troubleshooting.rst index 663051f97..d857ed4c4 100644 --- a/docs/testing/user/userguide/troubleshooting.rst +++ b/docs/testing/user/userguide/troubleshooting.rst @@ -261,40 +261,28 @@ Functest offers a possibility to test a customized list of Tempest test cases. To enable that, add a new entry in docker/smoke/testcases.yaml on the "smoke" container with the following content:: - - - case_name: tempest_custom - project_name: functest - criteria: 100 - blocking: false - description: >- - The test case allows running a customized list of tempest - test cases - dependencies: - installer: '' - scenario: '' - run: - module: 'functest.opnfv_tests.openstack.tempest.tempest' - class: 'TempestCustom' - -Also, a list of the Tempest test cases must be provided to the container or -modify the existing one in -/usr/lib/python3.8/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt - -Example of custom list of tests 'my-custom-tempest-tests.txt':: - - tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke] - tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke] + - + case_name: tempest_custom + project_name: functest + criteria: 100 + blocking: false + description: >- + The test case allows running a customized list of tempest + test cases + run: + name: tempest_common + args: + mode: "tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops|\ + tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops" This is an example of running a customized list of Tempest tests in Functest:: sudo docker run --env-file env \ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ -v $(pwd)/images:/home/opnfv/functest/images \ - -v $(pwd)/my-custom-testcases.yaml:/usr/lib/python3.8/site-packages/functest/ci/testcases.yaml \ - -v $(pwd)/my-custom-tempest-tests.txt:/usr/lib/python3.8/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt \ + -v $(pwd)/my-custom-testcases.yaml:/usr/lib/python3.8/site-packages/xtesting/ci/testcases.yaml \ opnfv/functest-smoke run_tests -t tempest_custom - Rally ^^^^^ |
