diff options
Diffstat (limited to 'docs/com/pres')
33 files changed, 0 insertions, 1952 deletions
diff --git a/docs/com/pres/Summit/Berlin-2016/conversation.html b/docs/com/pres/Summit/Berlin-2016/conversation.html deleted file mode 100644 index 356c2ad1..00000000 --- a/docs/com/pres/Summit/Berlin-2016/conversation.html +++ /dev/null @@ -1,251 +0,0 @@ -<!doctype html> -<html lang="en"> - - <head> - <meta charset="utf-8"> - - <title>OPNFV presentation</title> - - <meta name="description" content="Conversation with the Testing community"> - <meta name="author" content="OPNFV Testing Worjing Group"> - - <meta name="apple-mobile-web-app-capable" content="yes" /> - <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> - - <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui"> - - <link rel="stylesheet" href="../../reveal.js/css/reveal.css"> - <link rel="stylesheet" href="../../../css/theme/OPNFV-Berlin.css" id="theme"> - - <!-- Code syntax highlighting --> - <link rel="stylesheet" href="../../reveal.js/lib/css/zenburn.css"> - - <!-- Printing and PDF exports --> - <script> - var link = document.createElement( 'link' ); - link.rel = 'stylesheet'; - link.type = 'text/css'; - link.href = window.location.search.match( /print-pdf/gi ) ? '../../reveal.js/css/print/pdf.css' : '../../reveal.js/css/print/paper.css'; - document.getElementsByTagName( 'head' )[0].appendChild( link ); - </script> - - <!--[if lt IE 9]> - <script src="lib/js/html5shiv.js"></script> - <![endif]--> - </head> - - <body> - - <div class="reveal"> - <!-- Any section element inside of this container is displayed as a slide --> - <div class="slides"> - - <section data-background="../../../img/title-bg-berlin.png" data-background-transition="none"> - <br><br><br><br><br> - <h1>Conversation with the Testing Community</h1> - <br><br> - <h4>OPNFV testing community</h4> - <h5>OPNFV Design Summit, 20/6/2016, Berlin</h5> - <br> - - </section> - - <section data-markdown> - >“You make experiments and I make theories. Do you know the difference? A theory is something nobody believes, except the person who made it. An experiment is something everybody believes, except the person who made it." A.Einstein - </section> - - <section data-markdown> - # Agenda - * Who are we? - * Upstream, Upstream, Upstream - * Towards Telco Cloud KPI - </section> - - <section data-markdown> - # Who are we? -  - This is a representation...not a governance! - </section> - <section> - <section> - <h3>The test projects</h3> - <table> - <thead> - <tr> - <th>Test case</th> - <th>Description</th> - </tr> - </thead> - <tbody> - <tr> - <td>Functest</td> - <td>Umbrella project for Functional testing</td> - </tr> - <tr> - <td>Yardstick</td> - <td>Umbrella project for performance/qualification testing</td> - </tr> - <tr> - <td>CPerf</td> - <td>SDN controller performance testing</td> - </tr> - <tr> - <td>StorPerf</td> - <td>Storage performance testing</td> - </tr> - <tr> - <td>VSPerf</td> - <td>VSwitch qualification</td> - </tr> - <tr> - <td>Bottlenecks</td> - <td>Detect possible bottlenecks</td> - </tr> - <tr> - <td>QTIP</td> - <td>Benchmark to boost performance</td> - </tr> - </tbody> - </table> - </section> - - <section data-markdown> - # Functest - * Functional testing to validate scenarios - * Contributors (raise your hands) - * Functest presentation (here...1h ago :)) - * breakout sessions - * Today 3.10PM-3.40PM (vIMS evolution) 4PM (feature project integration), 5PM (Colorado status) - * tomorrow 11AM (API, test collection), 1PM (work Upstream), 3.15PM (D Release)(to be confirmed...) - </section> - <section data-markdown> - # Yardstick - * Performance testing framework and NFVI Validation - * [Contributors](https://wiki.opnfv.org/display/yardstick/People) - * Yardstick presentation - * How to Do a Pre-deployment NFVI Validation Quickly and Efficiently? Wednesday, June 22, 4:00 PM - 4:30 PM - * Yardstick: A Pre-deployment NFVI Validation Tool. Thursday, June 23, 3:10 PM - 3:30 PM - * breakout sessions - * Tomorrow 2PM (Yardstick in C) - </section> - <section data-markdown> - # CPerf - * SDN Controller Performance Testing - * Upstream test experts + standards/metrics experts - * Goal: Cross-controller perf tests in CI against realistic deployments - </section> - <section data-markdown> - # StorPerf - * Cinder Volume Performance Testing - * Based on SNIA's SSD Test Specification - * Presentation - * StorPerf: Cinder Storage Performance Measurement. - Wednesday, June 22, 3:40 PM - 3:55 PM - * Breakout Session - * Tomorrow 10:15 (StorPerf Test Results Working Session) - </section> - <section data-markdown> - # VSPerf - </section> - <section data-markdown> - # Bottlenecks - </section> - <section data-markdown> - # QTIP - - ## OPNFV Platform Benchmark - - - Bottom up - - testing baremetal components first - - More than result - - comparing against reference - - Setup-Test-Diagnose-Improve - - a closed loop to boost platform performance - - ## Breakout session - - - 14:30~15:45, June 21st/Tuesday - - Room Tegel - </section> - </section> - <section> - <section data-markdown> - # Upstream, upstream, upstream.. - </section> - <section data-markdown> - ## We are on the shoulders of the giants - * Rally (OpenStack) - * ODL, ONOS, ... - * RobotFramework, Teston - * ..... - </section> - <section data-markdown> - ## Time to give back - ### How to improve work with testing upstream community - </section> - - </section> - - <section> - - <section data-markdown> - # Towards Telco Cloud KPI? - </section> - - <section data-markdown> - # Testing group goals - * Define common needs (naming, rules, tools...) - * Unify result collection, reporting, dashboarding, documentation - * Share best practice: docker, CLI, .. - * Identify testing domains - * Encourage testing in poorly covered areas - </section> - - <section data-markdown> - # The ultimate goal: define Telco Cloud KPIs - * Provide methodology and test suites to qualify a Telco Cloud - * Provide reference benchs for plugfest/third party integration: compare apple with apple - </section> - - </section> - - <section data-markdown> - # Thank you - </section> - - </div> - <div class='footer'> - <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> - </div> - </div> - - <script src="../../reveal.js/lib/js/head.min.js"></script> - <script src="../../reveal.js/js/reveal.js"></script> - - <script> - - // Full list of configuration options available at: - // https://github.com/hakimel/reveal.js#configuration - Reveal.initialize({ - controls: true, - progress: true, - history: true, - center: true, - - transition: 'slide', // none/fade/slide/convex/concave/zoom - - // Optional reveal.js plugins - dependencies: [ - { src: '../../reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } }, - { src: '../../reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } }, - { src: '../../reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } }, - { src: '../../reveal.js/plugin/highlight/highlight.js', async: true, condition: function() { return !!document.querySelector( 'pre code' ); }, callback: function() { hljs.initHighlightingOnLoad(); } }, - { src: '../../reveal.js/plugin/zoom-js/zoom.js', async: true }, - { src: '../../reveal.js/plugin/notes/notes.js', async: true } - ] - }); - - </script> - - </body> -</html> diff --git a/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html b/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html deleted file mode 100644 index 97fa66c1..00000000 --- a/docs/com/pres/Summit/Berlin-2016/summit-Berlin.html +++ /dev/null @@ -1,313 +0,0 @@ -<!doctype html> -<html lang="en"> - - <head> - <meta charset="utf-8"> - - <title>OPNFV presentation</title> - - <meta name="description" content="Functest: Many rivers to cross"> - <meta name="author" content="Morgan Richomme and Functest contributors"> - - <meta name="apple-mobile-web-app-capable" content="yes" /> - <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> - - <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui"> - - <link rel="stylesheet" href="../../reveal.js/css/reveal.css"> - <link rel="stylesheet" href="../../../css/theme/OPNFV-Berlin.css" id="theme"> - - <!-- Code syntax highlighting --> - <link rel="stylesheet" href="../../reveal.js/lib/css/zenburn.css"> - - <!-- Printing and PDF exports --> - <script> - var link = document.createElement( 'link' ); - link.rel = 'stylesheet'; - link.type = 'text/css'; - link.href = window.location.search.match( /print-pdf/gi ) ? '../../reveal.js/css/print/pdf.css' : '../../../css/print/paper.css'; - document.getElementsByTagName( 'head' )[0].appendChild( link ); - </script> - - <!--[if lt IE 9]> - <script src="l../../reveal.jsml5shiv.js"></script> - <![endif]--> - </head> - - <body> - - <div class="reveal"> - <!-- Any section element inside of this container is displayed as a slide --> - <div class="slides"> - - <section data-background="../../../img/title-bg-berlin.png" data-background-transition="none"> - <br><br><br><br><br> - <h1>From Arno to Colorado</h1> - <h3>Functest: Many rivers to cross</h3> - <br> - <h4>Functest contributors</h4> - <h5>OPNFV Design Summit, 20/6/2016, Berlin</h5> - <br> - - </section> - - <section> - <blockquote> - “Bad programmers have all the answers. Good testers have all the questions. Gil Zilberfeld - ” - </blockquote> - </section> - - <section data-markdown> - # Agenda - * Functest - * Arno - * Brahmaputra - * Colorado - * D, E, F rivers and beyond - </section> - - <section data-markdown> - # The dream team -  - </section> - - <section> - <section data-markdown> - # Functest: what for? - </section> - <section data-markdown> - ## Our goals - ### make sure OPNFV works...... - </section> - <section data-markdown> - ## so concretely... - * Validate scenarios for the release - * Give confidence on OPNFV fresh releases - * Ensure consistancy towards installers - * Ensure End to End interoperability - </section> - <section data-markdown> - ## and also - * Provide functional test suites to be replayed - * ~ post install sanity check - * usable for plugfests - * coherent with ETSI and Telco needs - * Provide test tools - * run tests - * collect info - * create dashboard - </section> - <section data-markdown> - ## Our goals are not.... - * Performance testing (yardstick, Xperf ) - * Re-do tests done in specific projects - * storage/storeperf - * infra audit/qtip - * Compare results - </section> - - <section data-markdown> - ## Our rules - * Re-use as much as possible upstream suites (Rally, Tempest, Robot, testOn..) - * Keep it simple... - * Automate (if not it is not a test...) - * Create new suites only if tests not covered - * Consider fields not addressed by existing test projects - </section> - - <section data-markdown> - ## Our rules - * Installer neutral - * Controller neutral - * Black box approach - </section> - - </section> - - <section> - <section data-markdown> - # Arno - </section> - <section data-markdown> - ## Proof of concept - * 4 Testcases: vPing (userdata), Tempest, Rally, ODL - * 1 scenario: os-odl_l2-nofeature-ha - * 2 installers: Foreman/Fuel - * Lots of manual operations (reporting, documentation) - </section> - </section> - <section> - <section data-markdown> - # Brahmaputra - </section> - <section data-markdown> - ## More everything.... - * 7 Testcases: vPing (ssh), ONOS, vIMS, Tempest and Rally extended - * 3 feature projects: doctor, promise, sdnvpn - * 13 Scenarios - * 4 Installers (Apex, Compass, Fuel, Joid) - * Test result collection - * Automatic dashboard - </section> - <section> - <h3>Jiras</h3> - <img height="200" data-src="../../../img/jiras.png" alt="Jiras"> - <img height="200" data-src="../../../img/jiras2.png" alt="Jiras"> - </section> - - <section> - <h3>Brahmaputra Scenario matrix</h3> - <img width="800" data-src="../../../img/matrix.png" alt="Jiras"> - </section> - - <section> - Comparison odl_l2-nofeature-ha scenario - <table> - <thead> - <tr> - <th>Project</th> - <th>Apex</th> - <th>Compass</th> - <th>Fuel</th> - <th>Joid</th> - </tr> - </thead> - <tbody> - <tr> - <td>POD</td> - <td>LF POD1</td> - <td>Huawei US</td> - <td>Ericsson POD2</td> - <td>Orange POD2</td> - </tr> - <tr> - <td>Functest</td> - <td>2h50</td> - <td>1h45</td> - <td>2h35</td> - <td>2h30</td> - </tr> - <tr> - <td>Yardstick</td> - <td>1h05</td> - <td>1h00</td> - <td>1h00</td> - <td>0h55</td> - </tr> - </tbody> - </table> - * difference of duration due to POD and test suites - </section> - - <section data-markdown> - ## Lessons learned - * Tempest/Rally - * harmonizing installer related OpenStack configuration differences - * creating customized test lists - * main challenge was resolving SDN controller interworking problems - * vIMS - * complete but complex test case - * very interesting to automate (pre MANO, most of Telco needs met there) - </section> - - </section> - - <section> - <section data-markdown> - # Colorado - </section> - - <section data-markdown> - ## What's new? - * New internal test cases: healthcheck, security - * New controler: OpenContrail? - * New feature projects (domino, multisites, movie, parser, moon, copper, models, onos-sfc...) - * ARM Support (congratulations to ENEA team!) - * Simplified feature project integration journey - </section> - - <section data-markdown> - ## What's new? - * Slicing of the tests (healthcheck / Smoke / SDN controllers / Features / Components / VNFs) - * Better Test duration management - * Refactoring (repo, case management, TestAPI) - * Automatic reporting - * Dashboard evolution to ELK - * CLI - </section> - - - <section> - <h3>Automatic reporting</h3> - <img height="400" data-src="../../../img/reporting.png" alt="Reporting"> - <br><a href="http://testresults.opnfv.org/reporting/functest/release/master/index-status-fuel.html">Validated scenarios for Colorado</a> - </section> - - </section> - - <section> - <section data-markdown> - # Many rivers to cross - ## Functest beyond Colorado - </section> - <section data-markdown> - ## More test cases - * New VNFs (vEPC, vCDN, vWhatever...) - * Multi sites (e.g bgpvpn with different back ends) - * Better test coverage - </section> - <section data-markdown> - ## A Functional testing As a Service framework - * Scenario owner can select their relevant test cases - * Test duration estimation - * Agile dashboarding - * Analytics - </section> - <section data-markdown> - ## Upstream - * Rally: time to give back more - * Functest description for ETSI? - </section> - </section> - - <section> - <h3>Thank you</h3> - <img width="600" data-src="../../../img/colorado.png" alt="tests"> - </section> - - </div> - <div class='footer'> - <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> - </div> - </div> - - <script src="../../reveal.js/lib/js/head.min.js"></script> - <script src="../../reveal.js/js/reveal.js"></script> - - <script> - - // Full list of configuration options available at: - // https://github.com/hakimel/reveal.js#configuration - Reveal.initialize({ - controls: true, - progress: true, - history: true, - center: true, - - transition: 'slide', // none/fade/slide/convex/concave/zoom - - // Optional reveal.js plugins - dependencies: [ - { src: '../../reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } }, - { src: '../../reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } }, - { src: '../../reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } }, - { src: '../../reveal.js/plugin/highlight/highlight.js', async: true, condition: function() { return !!document.querySelector( 'pre code' ); }, callback: function() { hljs.initHighlightingOnLoad(); } }, - { src: '../../reveal.js/plugin/zoom-js/zoom.js', async: true }, - { src: '../../reveal.js/plugin/notes/notes.js', async: true } - ] - }); - - </script> - - </body> -</html> diff --git a/docs/com/pres/Summit/Berlin-2016/testapi.html b/docs/com/pres/Summit/Berlin-2016/testapi.html deleted file mode 100644 index c40637cb..00000000 --- a/docs/com/pres/Summit/Berlin-2016/testapi.html +++ /dev/null @@ -1,274 +0,0 @@ -<!doctype html> -<html lang="en"> - - <head> - <meta charset="utf-8"> - - <title>OPNFV presentation</title> - - <meta name="description" content="TestAPI"> - <meta name="author" content="Serena Feng"> - - <meta name="apple-mobile-web-app-capable" content="yes" /> - <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> - - <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui"> - - <link rel="stylesheet" href="../../reveal.js/css/reveal.css"> - <link rel="stylesheet" href="../../../css/theme/OPNFV-Berlin.css" id="theme"> - - <!-- Code syntax highlighting --> - <link rel="stylesheet" href="../../reveal.js/lib/css/zenburn.css"> - - <!-- Printing and PDF exports --> - <script> - var link = document.createElement( 'link' ); - link.rel = 'stylesheet'; - link.type = 'text/css'; - link.href = window.location.search.match( /print-pdf/gi ) ? '../../reveal.js/css/print/pdf.css' : '../../reveal.js/css/print/paper.css'; - document.getElementsByTagName( 'head' )[0].appendChild( link ); - </script> - - <!--[if lt IE 9]> - <script src="lib/js/html5shiv.js"></script> - <![endif]--> - </head> - - <body> - - <div class="reveal"> - <!-- Any section element inside of this container is displayed as a slide --> - <div class="slides"> - - <section data-background="../../../img/title-bg-berlin.png" data-background-transition="none"> - <br><br><br><br><br> - <h1>TestAPI</h1> - <h3>Clean, Easy, Complete</h3> - <br> - <h4>OPNFV testing community</h4> - <h5>OPNFV Design Summit, 20/6/2016, Berlin</h5> - <br> - </section> - - <section data-markdown> - # Agenda - * TestAPI: what for? - * API overview - * API evolution - * Roadmap - </section> - - <section> - <section data-markdown> - # TestAPI: what for? - </section> - <section data-markdown> - ## Consistant view for - * Resources (Pods) - * Projects - * Test cases - * Results - </section> - - <section data-markdown> - ## Status Currently - * From many projects - * Many formats (log, html, json, ..) - * Many locations (in VM, Jumphost, external DB,...) - </section> - <section data-markdown> - <script type='text/template'> - ## Achieve using TestAPI - * Uniform API: TestAPI - * Uniform format: JSON - * Universal Location: http://testresults.opnfv.org - <aside class='notes'> - By using TestAPI, we hope to provide a uniform way of collection and saving test results to a universal location - </aside> - </script> - </section> - - <section data-markdown> - ## Help building - * Dashboards - * Automatic reporting - </section> - </section> - - <section> - <section data-markdown> - # API overview - </section> - <section data-markdown> - ## Storage structure -  - </section> - <section data-markdown> - ## API in Brahmaputra -  - https://wiki.opnfv.org/display/functest/Collection+Of+Test+Results - </section> - - <section data-markdown> - ## API status in Brahmaputra - * Simple data models aggreed with the testing group - * No unit tests - * Wiki and rst documentation - </section> - - <section data-markdown> - ## Lessons learned in B. - * Painful wiki documentation - * Simple result model - * Misuse version as scenario - * No version actually - * No overall criteria (PASS/FAIL) - * Difficult refactoring - </section> - <section> - <h2>Colorado refactoring</h2> - <ul> - <li>Unit tests</li> - <li>Update of the data model</li> - <li>Tornado_swagger for the documentation</li> - <li>URL and MongoDB collection renaming</li> - <li>Scripts to support databse backup/restore/update</li> - <li>Support installation</li> - <aside class='notes'> - So you can discover and understand the capabilities of the service without - access to source code, documentation, or through network traffic inspection, - and also you can interact with the TestAPI directly through swagger website. - </aside> - </ul> - <p class="fragment fade-up"><b>All done in Colorado!</b></p> - </section> - </section> - - <section> - <section data-markdown> - # API Evoluation - </section> - - <section> - <h2> URI changes...</h2> - <div style="text-align:left""> - <p> testresults.opnfv.org/<span style="color:lightblue">TestAPI</span> => <br>testresults.opnfv.org/<span style="color:yellow">test/api/v1</span> </p> - - <p> /test/api/v1/<b>pods</b></p> - <p> /test/api/v1/<b>projects</b></p> - <p> /test/api/v1/projects/qtip/<b>cases</b></p> - <p> /test/api/v1/<b>results</b></p> - </div> - </section> - <section data-markdown> - ## One new field for pods - | Field | Colorado | - | :--- | :--- | - | role | community_pod, ci_pod, .. | - </section> - <section data-markdown> - ## Changed fields for results - | Item | Brahmaputra | Colorado | - | :--- | :--- | :--- | - | date | creation_date | start_date and stop_date | - | version | used as scenario | opnfv release version | - </section> - <section data-markdown> - <script type='text/template'> - ## New fields for results - | Item | Colorado | - | :--- | :--- | - | scenario | used for scenario | - | criteria | PASS/FAILED | - | trust_indicator | between 0 and 1| - <aside class='notes'> - to detect if the test case is stable over time and further more to avoid - running it systematically and saving time for other long duration test cases - </aside> - </script> - </section> - <section data-markdown> - ## One new filter for results - | Item | Colorado | - | :--- | :--- | - | last | get last N results| - </section> - <section data-markdown> - http://testresults.opnfv.org/test/swagger/spec.html -  - </section> - <section> - <h2>unit tests</h2> - <pre><code class="hljs" data-trim contenteditable> -umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh -Tests running... -WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing -WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms -WARNING:tornado.general:400 POST /api/v1/projects (127.0.0.1): name missing -............ -WARNING:tornado.access:400 POST /api/v1/projects (127.0.0.1) 1.13ms -WARNING:tornado.access:403 PUT /api/v1/projects/functest/cases/vping_1 (127.0.0.1) 2.95ms -WARNING:tornado.general:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1): {'project_name': u'functest', 'name': u'notFound'} could not be found in table [testcases] -WARNING:tornado.access:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1) 2.85ms - -Ran 74 tests in 1.848s -OK - </code></pre> - </section> - </section> - - <section> - <section data-markdown> - # Roadmap - </section> - <section data-markdown> - ## Roadmap - * Dockerize TestAPI - * Automatic update of TestAPI - * Command Line Interface - * Automatic update for pods/projects/testcases - </section> - <section data-markdown> - # suggestions ... - </section> - <section> - <h3>Thank you</h3> - <img width="600" data-src="../../../img/colorado.png" alt="tests"> - </section> - </section> - </div> - <div class='footer'> - <img src="../../../img/logo-OPNFV-Berlin.png" alt="OPNFV logo"> - </div> - </div> - - <script src="../../reveal.js/lib/js/head.min.js"></script> - <script src="../../reveal.js/js/reveal.js"></script> - - <script> - - // Full list of configuration options available at: - // https://github.com/hakimel/reveal.js#configuration - Reveal.initialize({ - controls: true, - progress: true, - history: true, - center: true, - - transition: 'slide', // none/fade/slide/convex/concave/zoom - - // Optional reveal.js plugins - dependencies: [ - { src: '../../reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } }, - { src: '../../reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } }, - { src: '../../reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } }, - { src: '../../reveal.js/plugin/highlight/highlight.js', async: true, condition: function() { return !!document.querySelector( 'pre code' ); }, callback: function() { hljs.initHighlightingOnLoad(); } }, - { src: '../../reveal.js/plugin/zoom-js/zoom.js', async: true }, - { src: '../../reveal.js/plugin/notes/notes.js', async: true } - ] - }); - - </script> - - </body> -</html> diff --git a/docs/com/pres/dashboard/dashboard_status.html b/docs/com/pres/dashboard/dashboard_status.html deleted file mode 100644 index 1321afa0..00000000 --- a/docs/com/pres/dashboard/dashboard_status.html +++ /dev/null @@ -1,146 +0,0 @@ -<!doctype html> -<html lang="en"> - - <head> - <meta charset="utf-8"> - - <title>OPNFV presentation</title> - - <meta name="description" content="Dashboard Status"> - <meta name="author" content="Serena Feng"> - - <meta name="apple-mobile-web-app-capable" content="yes" /> - <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> - - <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui"> - - <link rel="stylesheet" href="../reveal.js/css/reveal.css"> - <link rel="stylesheet" href="../../css/theme/OPNFV.css" id="theme"> - - <!-- Code syntax highlighting --> - <link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> - - <!-- Printing and PDF exports --> - <script> - var link = document.createElement( 'link' ); - link.rel = 'stylesheet'; - link.type = 'text/css'; - link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; - document.getElementsByTagName( 'head' )[0].appendChild( link ); - </script> - - <!--[if lt IE 9]> - <script src="lib/js/html5shiv.js"></script> - <![endif]--> - </head> - - <body> - - <div class="reveal"> - <!-- Any section element inside of this container is displayed as a slide --> - <div class="slides"> - - <section data-background="../../img/title-bg.png" data-background-transition="none"> - <h1>Dashboard Status</h1> - <h3></h3> - <br> - <h4>OPNFV testing community</h4> - <h5></h5> - <br> - </section> - - <section> - <h3>results generation</h3> - <img width="600" data-src="./dashboard_status/generation.png" alt="tests"> - </section> - <section> - <h3>result model</h3> - <img width="600" data-src="./dashboard_status/result_model.png" alt="tests"> - </section> - <section> - <h3>dashboard generation</h3> - <img width="600" data-src="./dashboard_status/dashboard_generate.png" alt="tests"> - </section> - <section> - <h3>selection layout(1)</h3> - <img width="600" data-src="./dashboard_status/selection_1.png" alt="tests"> - </section> - <section> - <h3>selection layout(2)</h3> - <img width="600" data-src="./dashboard_status/selection_2.png" alt="tests"> - </section> - <section> - <h3>display period select</h3> - <img width="600" data-src="./dashboard_status/period_select.png" alt="tests"> - </section> - <section> - <h3>duration display</h3> - <img width="600" data-src="./dashboard_status/duration.png" alt="tests"> - </section> - <section> - <h3>tests/failures display</h3> - <img width="600" data-src="./dashboard_status/number.png" alt="tests"> - </section> - <section> - <h3>success_percentage display</h3> - <img width="600" data-src="./dashboard_status/percentage.png" alt="tests"> - </section> - <section data-markdown> - # possible layouts - - * project - * testcase - * pod - * installer - * scenario - * version - * period - </section> - <section data-markdown> - # possible display domains - - * duration - * tests/failures - * success_percentage - </section> - <section> - <h3>Thank you</h3> - <img width="600" data-src="./dashboard_status/danube.jpg" alt="tests"> - </section> - - </div> - <div class='footer'> - <img src="../../img/logo-OPNFV.png" alt="OPNFV logo"> - </div> - </div> - - <script src="../reveal.js/lib/js/head.min.js"></script> - <script src="../reveal.js/js/reveal.js"></script> - - <script> - - // Full list of configuration options available at: - // https://github.com/hakimel/reveal.js#configuration - Reveal.initialize({ - controls: true, - progress: true, - history: true, - center: true, - - transition: 'slide', // none/fade/slide/convex/concave/zoom - - // Optional reveal.js plugins - dependencies: [ - { src: '../reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } }, - { src: '../reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } }, - { src: '../reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } }, - { src: '../reveal.js/plugin/highlight/highlight.js', async: true, condition: function() { return !!document.querySelector( 'pre code' ); }, callback: function() { hljs.initHighlightingOnLoad(); } }, - { src: '../reveal.js/plugin/zoom-js/zoom.js', async: true }, - { src: '../reveal.js/plugin/notes/notes.js', async: true } - ] - }); - - </script> - - </body> -</html> diff --git a/docs/com/pres/dashboard/dashboard_status/._danube.jpg b/docs/com/pres/dashboard/dashboard_status/._danube.jpg Binary files differdeleted file mode 100644 index 15669502..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._danube.jpg +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._dashboard_generate.png b/docs/com/pres/dashboard/dashboard_status/._dashboard_generate.png Binary files differdeleted file mode 100644 index 1f2afc65..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._dashboard_generate.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._duration.png b/docs/com/pres/dashboard/dashboard_status/._duration.png Binary files differdeleted file mode 100644 index 7ff52194..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._duration.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._generation.png b/docs/com/pres/dashboard/dashboard_status/._generation.png Binary files differdeleted file mode 100644 index 4154da2e..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._generation.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._number.png b/docs/com/pres/dashboard/dashboard_status/._number.png Binary files differdeleted file mode 100644 index 7ff52194..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._number.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._percentage.png b/docs/com/pres/dashboard/dashboard_status/._percentage.png Binary files differdeleted file mode 100644 index 8e72b4a2..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._percentage.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._period_select.png b/docs/com/pres/dashboard/dashboard_status/._period_select.png Binary files differdeleted file mode 100644 index 7ff52194..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._period_select.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._result_model.png b/docs/com/pres/dashboard/dashboard_status/._result_model.png Binary files differdeleted file mode 100644 index b9c8c2e1..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._result_model.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._selection_1.png b/docs/com/pres/dashboard/dashboard_status/._selection_1.png Binary files differdeleted file mode 100644 index 7ff52194..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._selection_1.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/._selection_2.png b/docs/com/pres/dashboard/dashboard_status/._selection_2.png Binary files differdeleted file mode 100644 index 7ff52194..00000000 --- a/docs/com/pres/dashboard/dashboard_status/._selection_2.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/danube.jpg b/docs/com/pres/dashboard/dashboard_status/danube.jpg Binary files differdeleted file mode 100644 index 4cf4ecce..00000000 --- a/docs/com/pres/dashboard/dashboard_status/danube.jpg +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/dashboard_generate.png b/docs/com/pres/dashboard/dashboard_status/dashboard_generate.png Binary files differdeleted file mode 100644 index 991f26c7..00000000 --- a/docs/com/pres/dashboard/dashboard_status/dashboard_generate.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/duration.png b/docs/com/pres/dashboard/dashboard_status/duration.png Binary files differdeleted file mode 100644 index 33497a9a..00000000 --- a/docs/com/pres/dashboard/dashboard_status/duration.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/generation.png b/docs/com/pres/dashboard/dashboard_status/generation.png Binary files differdeleted file mode 100644 index ec5ae169..00000000 --- a/docs/com/pres/dashboard/dashboard_status/generation.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/logo-OPNFV.png b/docs/com/pres/dashboard/dashboard_status/logo-OPNFV.png Binary files differdeleted file mode 100644 index 1519503e..00000000 --- a/docs/com/pres/dashboard/dashboard_status/logo-OPNFV.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/number.png b/docs/com/pres/dashboard/dashboard_status/number.png Binary files differdeleted file mode 100644 index a5f4ff43..00000000 --- a/docs/com/pres/dashboard/dashboard_status/number.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/percentage.png b/docs/com/pres/dashboard/dashboard_status/percentage.png Binary files differdeleted file mode 100644 index 640ef748..00000000 --- a/docs/com/pres/dashboard/dashboard_status/percentage.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/period_select.png b/docs/com/pres/dashboard/dashboard_status/period_select.png Binary files differdeleted file mode 100644 index 609a0d86..00000000 --- a/docs/com/pres/dashboard/dashboard_status/period_select.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/result_model.png b/docs/com/pres/dashboard/dashboard_status/result_model.png Binary files differdeleted file mode 100644 index 4ac6d671..00000000 --- a/docs/com/pres/dashboard/dashboard_status/result_model.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/selection_1.png b/docs/com/pres/dashboard/dashboard_status/selection_1.png Binary files differdeleted file mode 100644 index d264df81..00000000 --- a/docs/com/pres/dashboard/dashboard_status/selection_1.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/selection_2.png b/docs/com/pres/dashboard/dashboard_status/selection_2.png Binary files differdeleted file mode 100644 index a3d23667..00000000 --- a/docs/com/pres/dashboard/dashboard_status/selection_2.png +++ /dev/null diff --git a/docs/com/pres/dashboard/dashboard_status/title-bg.png b/docs/com/pres/dashboard/dashboard_status/title-bg.png Binary files differdeleted file mode 100644 index 6a84a44f..00000000 --- a/docs/com/pres/dashboard/dashboard_status/title-bg.png +++ /dev/null diff --git a/docs/com/pres/dockerslicing/dockerslicing.md b/docs/com/pres/dockerslicing/dockerslicing.md deleted file mode 100644 index a6645324..00000000 --- a/docs/com/pres/dockerslicing/dockerslicing.md +++ /dev/null @@ -1,124 +0,0 @@ -# Docker slicing - -[Cédric Ollivier](mailto:cedric.ollivier@orange.com) - -2017/10/19 - - - -## Danube issues - - -### OPNFV projects' setup.py - -- no requirements were installed when calling *python setup.py install* as none of the next keys was set: - - install_requires - - tests_require - - dependency_links -- shell scripts were not installed neither in $PATH nor in dist-packages -- all requirements were not synchronized over the OPNFV projects - - -### Functest's Dockerfile - -- it copied all the files hosted by the third-party projects (e.g. docs, .git...) -- several requirements were downgraded/upgraded when building the container as they were managed one after the other -- it could download packages from [PyPI](https://pypi.python.org/pypi) (e.g. [networking-bgpvpn](https://pypi.python.org/pypi/networking-bgpvpn)...) instead of cloning git repository -- build dependencies couldn't be removed to save space as it created multiple layers (>70) - - - -## Management of the requirements - - -### Rely on [pbr](https://docs.openstack.org/developer/pbr/) - -- pbr injects requirements into the install_requires, tests_require and/or dependency_links arguments to setup -- it supports conditional dependencies which can be added to the requirements (e.g. dnspython>=1.14.0;python_version=='2.7') - - -### Split requirements into 3 files - -- **requirements.txt** which should list all abstract (i.e. [not associated with any particular index](https://packaging.python.org/requirements/)) dependencies of the OPNFV packages -- **test-requirements.txt** which could list all abstract dependencies required for testing the OPNFV packages -- **upper-constraints.txt** which should list all concrete dependencies required by Functest Docker container or the testing virtual environments - - -### Follow [OpenStack requirements management](https://specs.openstack.org/openstack/openstack-specs/specs/requirements-management.html) - -- OPNFV (test-)requirements.txt have been updated according to stable/ocata global-requirements.txt. -- Functest simply use (and complete) stable/ocata upper-constraints.txt in Docker files and tox configuration (testing virtual environments). - - -### On the road - -- we have fixed lots of hardcoded paths hidden by the previous design -- some files were outside the python packages -- lots of (console) scripts added in OPNFV packages to ease the maintenance of Functest testcases.yaml - - - -## Docker slicing - - -### 8 Functest containers - -```bash -$ sudo docker search opnfv |grep functest- -opnfv/functest-core OPNFV Functest core image -opnfv/functest-restapi OPNFV Functest restapi image -opnfv/functest-features OPNFV Functest vnf image -opnfv/functest-healthcheck OPNFV Functest healthcheck image -opnfv/functest-smoke OPNFV Functest smoke image -opnfv/functest-vnf OPNFV Functest vnf image -opnfv/functest-components OPNFV Functest components image -opnfv/functest-parser OPNFV Functest parser image -``` - - -### 8 Functest containers - -- Alpine 3.6 is now used as base image -- one container per test suite has been published (5). All of them are built on top of functest-core. -- Parser is hosted in its own containers (it requires librairies released for OpenStack Pike) -- a full container is dedicated to our REST API. - -Please see [Run Alpine Functest containers](https://wiki.opnfv.org/display/functest/Run+Alpine+Functest+containers) - - - -## Next steps - - -### Functest and XCI - -- the purpose is simply to allow any OPNFV project integrated by Functest to build their own containers on top of opnfv/functest-core -- it will allow testing one specific change of these OPNFV projects before merging it in tree -- it induces that all requirements are synchronized between the different OPNFV projects - - -### F-release - -- to allow building opnfv/functest-core from a gerrit change (see https://gerrit.opnfv.org/gerrit/#/c/40909/) -- to split Functest core/ci and the Functest testcases in two separated Python packages -- to unlink prepare_env.py and tempest/rally - - -### F-release - -- to add python3 support for Functest ci scripts (Functest core already supports both versions) -- to unlink functest-core from others OPNFV projects (mainly releng and snaps) -- to write a generic Dockerfile using a set of python packages as input - - -### 2 OPNFV project proposals - -- requirements, the counterpart of [OpenStack requirements](https://wiki.openstack.org/wiki/Requirements), -focused on additional OPNFV project needs -- xtesting which would be derived from Functest (core and ci parts) as proposed in -[Functional testing gating](https://wiki.opnfv.org/display/functest/Functional+testing+gating) - - - -## Thank you! - diff --git a/docs/com/pres/dockerslicing/index.html b/docs/com/pres/dockerslicing/index.html deleted file mode 100644 index 1c65e4fa..00000000 --- a/docs/com/pres/dockerslicing/index.html +++ /dev/null @@ -1,52 +0,0 @@ -<html> -<head> -<title>Docker slicing</title> -<meta name="author" content="Cédric Ollivier"> -<meta name="viewport" - content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> -<link rel="stylesheet" href="../reveal.js/css/reveal.css"> -<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> -<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> -<script> -var link = document.createElement( 'link' ); -link.rel = 'stylesheet'; -link.type = 'text/css'; -link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; -document.getElementsByTagName( 'head' )[0].appendChild( link ); -</script> -</head> -<body> - <div class="reveal"> - <div class="slides"> - <section data-markdown="dockerslicing.md" data-separator="^\n\n\n" - data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> - </div> - </div> - <script src="../reveal.js/lib/js/head.min.js"></script> - <script src="../reveal.js/js/reveal.js"></script> - <script> - Reveal.initialize({ - dependencies : [ { - src : '../reveal.js/plugin/markdown/marked.js', - condition : function() { - return !!document.querySelector('[data-markdown]'); - } - }, { - src : '../reveal.js/plugin/markdown/markdown.js', - condition : function() { - return !!document.querySelector('[data-markdown]'); - } - }, { - src: '../reveal.js/plugin/highlight/highlight.js', - async: true, - callback: function() { - hljs.initHighlightingOnLoad(); - } - }, { - src: '../reveal.js/plugin/notes/notes.js', - async: true - } ] - }); - </script> -</body> -</html> diff --git a/docs/com/pres/euphrates_functest_evolution/euphrates.md b/docs/com/pres/euphrates_functest_evolution/euphrates.md deleted file mode 100644 index 49aab16c..00000000 --- a/docs/com/pres/euphrates_functest_evolution/euphrates.md +++ /dev/null @@ -1,281 +0,0 @@ -# Functest -### Feedback on Euphrates evolutions - -2017/10/09 - - -### Main Framework evolutions - * Functest Framework refactoring - * Docker slicing with Alpine - * Building Functest dockers - * Requirement management...1st time...in OPNFV - * Functest API - - -## Functest Framework refactoring - -### The framework - * Initiated in Danube - * Finalized in euphrates - * Goal: ease the integration of feature/vnf tests - -### See complete presentation on the framework -http://testresults.opnfv.org/functest/framework/ - - -## Docker slicing - - -#### Short story of Functest Docker - - -### Many rivers to cross -| Version | Comment | -|-------------|-----------------------------------------------------| -| Arno | No docker, all tests initiated from the jumphost | -| Brahmaputra | Introduction of Docker | -| Colorado | Growth of Historical Docker | -| Danube | Growth of Historical Docker | -| Euphrates | Introduction of Alpine | - - -### Focus on the Historical Docker - * Based on ubuntu 14.04 - -| Version | Compressed Size | -|--------------|-----------------| -| Brahmaputra | 354 MB | -| Colorado.1.0 | 627 MB | -| Danube.1.0 | 713 MB | -| cvp.0.2.0 | 710 MB | - -* src: https://hub.docker.com/r/opnfv/functest/tags/ - - -### Introduction to Alpine -* Alpine Linux is a GNU/Linux distribution based on musl and Busybox -* Hardened kernel, compiles all user space binaries as position-independent executables with stack-smashing protection. -* Docker Alpine leverage Alpine Linux: https://docs.docker.com/samples/library/alpine/ - - -### Our goals -* Lighten docker / save bandwidth / save CI resources -* Slice testing -* Simplify Requirement management -* Isolate "exotic" test cases - - -### From 1 to many dockers -| Docker | Size | Role | -|-------------|-------|-------------------------------------------------| -| core | 122MB | baseline (tools, env) | -| healthcheck | 122MB | OS connectivity, API, DHCP testing | -| smoke | 131MB | vpings, Tempest/Refstack, Rally, odl, Snaps | -| features | 214MB | doctor, domino, sdnvpn, sfc, promise, barometer | - - -### From 1 to many dockers -| Docker | Size | Role | -|-------------|-------|-------------------------------------------------| -| components | 122MB | Full Tempest, Full Rally | -| vnf | 155MB | vIMS, vRouter | -| parser | 127MB | parser (feature needs pike clients) | - - -### Easy way to run -src: https://wiki.opnfv.org/display/functest/Run+Alpine+Functest+containers - * env: OPNFV env variables - * openstack.creds: OpenStack rc file - -``` -sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ - -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-smoke:euphrates -``` - - -#### Env -``` -cat env -INSTALLER_TYPE=Compass -INSTALLER_IP=XXX -EXTERNAL_NETWORK=ext-net -DEPLOY_SCENARIO=os-nosdn-nofeature-ha -``` - - -#### openstack.creds -``` -cat openstack.creds -export OS_AUTH_URL=XXX -export OS_USER_DOMAIN_NAME=XXX -export OS_PROJECT_DOMAIN_NAME=XXX -export OS_USERNAME=XXX -export OS_PROJECT_NAME=XXX -export OS_PASSWORD=XXX -export OS_IDENTITY_API_VERSION=3 -``` - - -#### Retrieve all the images -``` -mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* -images/CentOS-7-aarch64-GenericCloud.qcow2 -images/CentOS-7-aarch64-GenericCloud.qcow2.xz -images/CentOS-7-x86_64-GenericCloud.qcow2 -images/cirros-0.3.5-x86_64-disk.img -images/cirros-0.3.5-x86_64-lxc.tar.gz -images/cirros-d161201-aarch64-disk.img -images/cirros-d161201-aarch64-initramfs -images/cirros-d161201-aarch64-kernel -images/cloudify-manager-premium-4.0.1.qcow2 -images/img -images/trusty-server-cloudimg-amd64-disk1.img -images/ubuntu-14.04-server-cloudimg-amd64-disk1.img -images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img -images/ubuntu-16.04-server-cloudimg-amd64-disk1.img -images/vyos-1.1.7.img -``` -* could be improved (retrieve only needed images Tier/config) - - -#### Results -``` -+----------------------+--------- + ------+----------+--------+ -| TEST CASE | PROJECT | TIER | DURATION | RESULT | -+----------------------+--------- + ------+----------+--------+ -| vping_ssh | functest | smoke | 01:19 | PASS | -| vping_userdata | functest | smoke | 01:56 | PASS | -| tempest_smoke_serial | functest | smoke | 26:30 | PASS | -| rally_sanity | functest | smoke | 19:42 | PASS | -| refstack_defcore | functest | smoke | 22:00 | PASS | -| snaps_smoke | functest | smoke | 41:14 | PASS | -| odl | functest | smoke | 00:16 | PASS | -| odl_netvirt | functest | smoke | 00:00 | SKIP | -| fds | functest | smoke | 00:00 | SKIP | -+----------------------+--------- + ------+----------+--------+ - -``` - - -#### Easy way to customize - * Change list of testcase -v your_config.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml - * Change logger param -v your_logger.ini:/usr/lib/python2.7/site-packages/functest/ci/logging.ini - - -#### Docker slicing technical presentation -http://testresults.opnfv.org/functest/dockerslicing/ - - - -## Building Functest dockers - - -#### Until Danube - * the unique Functest docker was "produced" from Releng - - -#### For Euphrates, build was done on Docker hub - * Releng adaptations not ready in time - * more capabilities using Docker Hub - * Use of private ollivier then official opnfv Docker Hub - * Euphrates 5.1: come back to releng with at least same level of features than Docker Hub/Travis CI - - - -## Requirement management -Until now, requirements were managed as follow: - - -# ? - - -### Requirement management - * Nothing was done - * Danube, Colorado, .. dockers may run but no garantee on mid/long term because - * No control of upstream or internal project dependencies - * the dependencies of the last feature project overwrite the previous ones - - -### Requirement management - * Nothing done in any OPNFV project - * No dependency list, no reco (e.g. support Python 3) - * Only high level wiki declarative intentions for OpenStack (e.g. Euphrates => Ocata) - * Nothing equivalent to OpenStack https://releases.openstack.org/ocata/ - - -### Functest evolutions - * creation of requirements.txt, upper-constraints.text - https://git.opnfv.org/functest/tree/requirements.txt - https://git.opnfv.org/functest/tree/upper-constraints.txt - ``` - pbr>=1.8 # Apache-2.0 - PyYAML>=3.10.0 # MIT - GitPython>=1.0.1 # BSD License (3 clause) - keystoneauth1>=2.18.0 # Apache-2.0 - python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0 - python-glanceclient>=2.5.0 # Apache-2.0 - python-heatclient>=1.6.1 # Apache-2.0 - ... - ``` - * sync with ocata done manually by Cédric - - -### Functest evolutions - * Code of the feature projects under project responsibility - * code moved to their own repo - * no more mix between Functest and Feature project code - * All python OPNFV projects imported properly as python modules - * all dependencies a priori under control... - - - -## Functest Rest API - - -### Introduction -* A Rest API has been introduced in Euphrates (Linda) -* Goal: allow third party to invoke Functest resources - * pseudo micro services approach - * avoid overlap (e.g. deployement of vIMS from other project to run perfo tests) -src: https://wiki.opnfv.org/display/functest/Functest+REST+API - - -### Functest rest API -| resource | Methods | Description | -|-------------|----------|-------------------------------------------------| -| environment | GET,POST | show, prepare environment | -| openstack | GET,POST | show, check, clean, update credentials | -| testcases | GET,POST | list, show, run | -| tiers | GET,POST | list, show | -| tasks | GET | Get the result of the task id | - - -### Examples -``` -curl -X POST --header "Content-Type: application/json" \ - --data '{"action":"run_test_case", "args": {"opts": {}, "testcase": "vping_ssh"}}' \ - http://127.0.0.1:5000/api/v1/functest/testcases/action -{ - "task_id": "1a9f3c5d-ce0b-4354-862e-dd08b26fc484", - "testcase":"vping_ssh" -} -``` - - - -## Conclusions - * Framework heavily refactored for Euphrates - * Better code quality and rules (coverage, pylint, pep8, ..) - * Much more "trustable", light and evolutive - - -## Next steps - * Still lots of work for Fraser - * Adaptation to use it for XCI gating (xTesting) - * Generic dockerfile Functest customized docker on demand - * split framework and testcases in order to be able to reuse Functest for k8 or even beyond OPNFV - * integrate k8 tests - * better management of the images - * ... diff --git a/docs/com/pres/euphrates_functest_evolution/index.html b/docs/com/pres/euphrates_functest_evolution/index.html deleted file mode 100644 index 5b8fb260..00000000 --- a/docs/com/pres/euphrates_functest_evolution/index.html +++ /dev/null @@ -1,52 +0,0 @@ -<html> -<head> -<title>OPNFV Functest Feedback on Alpines</title> -<meta name="author" content="Cédric Ollivier"> -<meta name="viewport" - content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> -<link rel="stylesheet" href="../reveal.js/css/reveal.css"> -<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> -<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> -<script> -var link = document.createElement( 'link' ); -link.rel = 'stylesheet'; -link.type = 'text/css'; -link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; -document.getElementsByTagName( 'head' )[0].appendChild( link ); -</script> -</head> -<body> - <div class="reveal"> - <div class="slides"> - <section data-markdown="euphrates.md" data-separator="^\n\n\n" - data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> - </div> - </div> - <script src="../reveal.js/lib/js/head.min.js"></script> - <script src="../reveal.js/js/reveal.js"></script> - <script> - Reveal.initialize({ - dependencies : [ { - src : '../reveal.js/plugin/markdown/marked.js', - condition : function() { - return !!document.querySelector('[data-markdown]'); - } - }, { - src : '../reveal.js/plugin/markdown/markdown.js', - condition : function() { - return !!document.querySelector('[data-markdown]'); - } - }, { - src: '../reveal.js/plugin/highlight/highlight.js', - async: true, - callback: function() { - hljs.initHighlightingOnLoad(); - } - }, { - src: '../reveal.js/plugin/notes/notes.js', - async: true - } ] - }); - </script> -</body> -</html> diff --git a/docs/com/pres/framework/framework.md b/docs/com/pres/framework/framework.md deleted file mode 100644 index 109d8a03..00000000 --- a/docs/com/pres/framework/framework.md +++ /dev/null @@ -1,407 +0,0 @@ -# Functest Framework - -created by [Cédric Ollivier](mailto:cedric.ollivier@orange.com) - -2017/06/05 - -Note: - -- Functest integrates lots of heterogeneous testcases: - - python vs bash - - internal vs external -- it aims to benefit from object programming - - to define common operations - - to avoid conditional instructions regarding the testcases - - to avoid duplicating code - - to ease the integration of third-party testcases (written in Bash or Python) - - - -## Quick overview - - -### Functest function calls - -- **CI** calls *run_tests.py* (please see [jenkins jobs](https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=jjb/functest)) -- *run_tests.py* parses *functest/ci/testcases.yaml* to: - - check which testcase(s) must be run - - execute the common operations on every testcase (run, push its results to db...) -<!-- .element: class="fragment highlight-red"--> - - return the right status code to **CI** - - -### Our target - -- limit run_tests.py instructions by defining: - - the basic testcase attributes - - all common operations - - the status codes expected -- avoid duplicating codes between testcases -- ease the development of third-party testcases (aka features) - - - -## class TestCase - -base model for single test case - - -### instance attributes - -- project_name (default: 'functest') -- case_name -- criteria -- result -- start_time -- stop_time -- details - - -### methods - -| Method | Purpose | -|-------------------|------------------------------| -| run(**kwargs) | run the test case | -| is_successful() | interpret the results | -| get_duration() | return the duration | -| push_to_db() | push the results to the DB | -| clean() | clean the resources | - - -### run(**kwargs) - -- the subclasses must override the default implementation which is false on purpose -- the new implementation must set the following attributes to push the results to DB: - - result - - start_time - - stop_time - - -### class attributes - -| Status code | Returned when | -|--------------------|---------------------| -| EX_OK | everything is OK | -| EX_RUN_ERROR | run() failed | -| EX_TESTCASE_FAILED | results are false | -| EX_PUSH_TO_DB_ERROR| push_to_db() failed | - - -### run_tests.py - -```python -module = importlib.import_module(run_dict['module']) -cls = getattr(module, run_dict['class']) -test_dict = ft_utils.get_dict_by_test(test_name) -test_case = cls(**test_dict) -try: - kwargs = run_dict['args'] - result = test_case.run(**kwargs) -except KeyError: - result = test_case.run() -if result == testcase.TestCase.EX_OK: - if GlobalVariables.REPORT_FLAG: - test_case.push_to_db() - result = test_case.is_successful() -``` - - - -## Your first test case - - -### first.py - -```python -#!/usr/bin/env python - -import time - -from functest.core import testcase - -class Test(testcase.TestCase): - - def run(self, **kwargs): - self.start_time = time.time() - print "Hello World" - self.result = 100 - self.stop_time = time.time() - return testcase.TestCase.EX_OK -``` - - -### functest/ci/testcases.yaml - -```yaml -case_name: first -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'first' - class: 'Test' -``` - - - -## class Feature -bases: TestCase - -base model for single feature - - -### methods - -| Method | Purpose | -|-------------------|---------------------------| -| run(**kwargs) | run the feature | -| execute(**kwargs) | execute the Python method | - - -### run(**kwargs) - -- allows executing any Python method by calling execute() -- sets the following attributes required to push the results to DB: - - result - - start_time - - stop_time -- doesn't fulfill details when pushing the results to the DB. - - -### execute(**kwargs) - -- the subclasses must override the default implementation which is false on purpose -- the new implementation must return 0 if success or anything else if failure. - - - -## Your second test case - - -### second.py - -```python -#!/usr/bin/env python - -from functest.core import feature - -class Test(feature.Feature): - - def execute(self, **kwargs): - print "Hello World" - return 0 -``` - - -### functest/ci/testcases.yaml - -```yaml -case_name: second -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'second' - class: 'Test' -``` - - - -## class BashFeature -bases: Feature - -class designed to run any bash command - - -### execute(**kwargs) - -execute the cmd passed as arg. - - - -## Your third test case - - -### functest/ci/testcases.yaml - -``` -case_name: third -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'functest.core.feature' - class: 'BashFeature' - args: - cmd: 'echo Hello World; exit 0' -``` - - - -## class Suite -bases: TestCase - -base model for running unittest.TestSuite - - -### run(**kwargs) - -- allows running any unittest.TestSuite -- sets the following attributes required to push the results to DB: - - result - - start_time - - stop_time - - details - - - -## Your fourth test case - - -### fourth.py - -```python -#!/usr/bin/env python - -import unittest - -class TestStringMethods(unittest.TestCase): - - def test_upper(self): - self.assertEqual('Hello World'.upper(), - 'HELLO WORLD') -``` - - -### functest/ci/testcases.yaml - -``` -case_name: fourth -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'functest.core.unit' - class: 'Suite' - args: - name: 'fourth' -``` - - - -## class VNF -bases: TestCase - -base model for VNF onboarding testing - - -### methods - -| Method | Purpose | -|-----------------------|---------------------------------------------------| -| prepare() | prepare VNF env (user, tenant, security group,..) | -| run(**kwargs) | run VNF test case | -| deploy_orchestrator() | deploy cloudify, ONAP, OpenBaton,... (optional) | -| deploy_vnf() | deploy the VNF | -| test_vnf() | run tests on the VNF | - - -### run(**kwargs) - -- deploys an orchestrator if needed (e.g. heat, OpenBaton, Cloudify, ONAP, Juju) -- deploys the VNF -- performs tests on the VNF - - -### prepare() - -- creates a user -- creates a Tenant/Project -- allocates admin role to the user on this tenant - - -### deploy_orchestrator() - -- deploys an orchestrator (optional) -- if this function is overridden then raise orchestratorDeploymentException if error during orchestrator deployment - - -### deploy_vnf() - -- **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases. -- The deployment can be executed via a specific orchestrator or using build-in orchestrators such as heat, openbaton, cloudify, juju, ONAP, ... -- returns: - True if the VNF is properly deployed - False if the VNF is not deployed -- raises VnfDeploymentException if error during VNF deployment - - -### test_vnf() - -- **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases. -- Once a VNF is deployed, it is assumed that specific test suite can be run to validate the VNF. -- returns: - True if VNF tests are PASS - False if test suite is FAIL -- raises VnfTestException if error during VNF tests - - - -## Your fifth test case - - -### fifth.py - -```python -#!/usr/bin/env python - -from functest.core import vnf - -class Vnf(vnf.VnfOnBoarding): - - def deploy_vnf(self): - print "Deploy your VNF here" - print "Feed orchestrator with VNF descriptor" - return 0 - - def test_vnf(self): - print "Test your VNF here" - return 0 -``` - - -### functest/ci/testcases.yaml - -```yaml -case_name: fifth -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'fifth' - class: 'Vnf' -``` - - - -## Thank You! diff --git a/docs/com/pres/framework/index.html b/docs/com/pres/framework/index.html deleted file mode 100644 index 950c2beb..00000000 --- a/docs/com/pres/framework/index.html +++ /dev/null @@ -1,52 +0,0 @@ -<html> -<head> -<title>OPNFV Functest Framework</title> -<meta name="author" content="Cédric Ollivier"> -<meta name="viewport" - content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> -<link rel="stylesheet" href="../reveal.js/css/reveal.css"> -<link rel="stylesheet" href="../reveal.js/css/theme/white.css"> -<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css"> -<script> -var link = document.createElement( 'link' ); -link.rel = 'stylesheet'; -link.type = 'text/css'; -link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css'; -document.getElementsByTagName( 'head' )[0].appendChild( link ); -</script> -</head> -<body> - <div class="reveal"> - <div class="slides"> - <section data-markdown="framework.md" data-separator="^\n\n\n" - data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section> - </div> - </div> - <script src="../reveal.js/lib/js/head.min.js"></script> - <script src="../reveal.js/js/reveal.js"></script> - <script> - Reveal.initialize({ - dependencies : [ { - src : '../reveal.js/plugin/markdown/marked.js', - condition : function() { - return !!document.querySelector('[data-markdown]'); - } - }, { - src : '../reveal.js/plugin/markdown/markdown.js', - condition : function() { - return !!document.querySelector('[data-markdown]'); - } - }, { - src: '../reveal.js/plugin/highlight/highlight.js', - async: true, - callback: function() { - hljs.initHighlightingOnLoad(); - } - }, { - src: '../reveal.js/plugin/notes/notes.js', - async: true - } ] - }); - </script> -</body> -</html> diff --git a/docs/com/pres/reveal.js b/docs/com/pres/reveal.js deleted file mode 160000 -Subproject a349ff43c58c23f9c837b8ea9b5fc7d4761b8de |
