Publish framework presentation

This is the first version prepared before merging "Switch TestCase attribute criteria to result" [1]. It will also be updated to integrate VNF classes as soon as they are ready (UT, pylint, docstrings...). [1] https://gerrit.opnfv.org/gerrit/#/c/33695/ Change-Id: I2fc8f4ae377afee70e083cd51e5725eccda15431 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
author: Cédric Ollivier <cedric.ollivier@orange.com> 2017-04-20 16:58:41 +0200
committer: Cédric Ollivier <cedric.ollivier@orange.com> 2017-05-02 09:49:09 +0200
commit: cc584596a43af40db096015105d44e63c55cafbb (patch)
tree: 7bb671f7aaf2711dd860358aae44f9fadef6ecff /docs/com
parent: 90238c6c2c6b8c63062f3fe1c1611d594d72604b (diff)
2 files changed, 318 insertions, 0 deletions
diff --git a/docs/com/pres/framework/framework.html b/docs/com/pres/framework/framework.html
new file mode 100644
index 000000000..950c2bebf
--- /dev/null
+++ b/docs/com/pres/framework/framework.html
@@ -0,0 +1,52 @@
+<html>
+<head>
+<title>OPNFV Functest Framework</title>
+<meta name="author" content="Cédric Ollivier">
+<meta name="viewport"
+    content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
+<link rel="stylesheet" href="../reveal.js/css/reveal.css">
+<link rel="stylesheet" href="../reveal.js/css/theme/white.css">
+<link rel="stylesheet" href="../reveal.js/lib/css/zenburn.css">
+<script>
+var link = document.createElement( 'link' );
+link.rel = 'stylesheet';
+link.type = 'text/css';
+link.href = window.location.search.match( /print-pdf/gi ) ? '../reveal.js/css/print/pdf.css' : '../reveal.js/css/print/paper.css';
+document.getElementsByTagName( 'head' )[0].appendChild( link );
+</script>
+</head>
+<body>
+    <div class="reveal">
+        <div class="slides">
+            <section data-markdown="framework.md" data-separator="^\n\n\n"
+                data-separator-vertical="^\n\n" data-separator-notes="^Note:"></section>
+        </div>
+    </div>
+    <script src="../reveal.js/lib/js/head.min.js"></script>
+    <script src="../reveal.js/js/reveal.js"></script>
+    <script>
+        Reveal.initialize({
+            dependencies : [ {
+                src : '../reveal.js/plugin/markdown/marked.js',
+                condition : function() {
+                    return !!document.querySelector('[data-markdown]');
+                }
+            }, {
+                src : '../reveal.js/plugin/markdown/markdown.js',
+                condition : function() {
+                    return !!document.querySelector('[data-markdown]');
+                }
+            }, {
+                src: '../reveal.js/plugin/highlight/highlight.js',
+                async: true,
+                callback: function() {
+                    hljs.initHighlightingOnLoad();
+                }
+            }, {
+            src: '../reveal.js/plugin/notes/notes.js',
+        async: true
+            } ]
+        });
+    </script>
+</body>
+</html>
diff --git a/docs/com/pres/framework/framework.md b/docs/com/pres/framework/framework.md
new file mode 100644
index 000000000..b80ad3dd7
--- /dev/null
+++ b/docs/com/pres/framework/framework.md
@@ -0,0 +1,266 @@
+# Functest Framework
+
+created by [Cédric Ollivier](mailto:cedric.ollivier@orange.com)
+
+2017/04/24
+
+Note:
+
+- Functest integrates lots of heteregeounous testcases:
+    - python vs bash
+    - internal vs external
+- it aims to benefit from object programming
+    - to define common operations
+    - to avoid conditional instructions regarding the testcases
+    - to avoid duplicating code
+    - to ease the integration of third-party testcases (written in Bash or Python)
+
+
+
+## Quick overview
+
+
+### Functest function calls
+
+- **CI** calls *run_tests.py* (please see [jenkins jobs](https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=jjb/functest))
+- *run_tests.py* parses *functest/ci/testcases.yaml* to:
+    - check which testcase(s) must be run
+    - execute the common operations on every testcase (run, push its results to db...)
+<!-- .element: class="fragment highlight-red"-->
+    - return the right status code to **CI**
+
+
+### Our target
+
+- limit run_tests.py instructions by defining:
+    - the basic testcase attritutes
+    - all common operations
+    - the status codes expected
+- avoid duplicating codes between testcases
+- ease the developpement of third-party testcases (aka features)
+
+
+
+## class TestCase
+
+base model for single test case
+
+
+### instance attributes
+
+- project_name (default: 'functest')
+- case_name
+- criteria
+- start_time
+- stop_time
+- details
+
+
+### methods
+
+| Method            | Purpose                                    |
+|-------------------|--------------------------------------------|
+| run(**kwargs)     | run the test case                          |
+| check_criteria()  | interpret the results of the test case     |
+| push_to_db()      | push the results of the test case to the DB|
+
+
+### run(**kwargs)
+
+- the subclasses must override the default implementation which is false on purpose
+- the new implementation must set the following attributes to push the results to DB:
+    - criteria
+    - start_time
+    - stop_time
+
+
+### class attributes
+
+| Status code        | Returned when       |
+|--------------------|---------------------|
+| EX_OK              | everything is OK    |
+| EX_RUN_ERROR       | run() failed        |
+| EX_TESTCASE_FAILED | results are false   |
+| EX_PUSH_TO_DB_ERROR| push_to_db() failed |
+
+
+### run_tests.py
+
+```python
+module = importlib.import_module(run_dict['module'])
+cls = getattr(module, run_dict['class'])
+test_dict = ft_utils.get_dict_by_test(test_name)
+test_case = cls(**test_dict)
+try:
+    kwargs = run_dict['args']
+    result = test_case.run(**kwargs)
+except KeyError:
+    result = test_case.run()
+if result == testcase.TestCase.EX_OK:
+    if GlobalVariables.REPORT_FLAG:
+        test_case.push_to_db()
+    result = test_case.check_criteria()
+```
+
+
+
+## Your first test case
+
+
+### first.py
+
+```python
+#!/usr/bin/env python
+
+import time
+
+from functest.core import testcase
+
+class Test(testcase.TestCase):
+
+    def run(self, **kwargs):
+        self.start_time = time.time()
+        print "Hello World"
+        self.criteria = 'PASS'
+        self.stop_time = time.time()
+        return testcase.TestCase.EX_OK
+```
+
+
+### functest/ci/testcases.yaml
+
+```yaml
+case_name: first
+project_name: functest
+criteria: 'status == "PASS"'
+blocking: true
+clean_flag: false
+description: ''
+dependencies:
+    installer: ''
+    scenario: ''
+run:
+    module: 'first'
+    class: 'Test'
+```
+
+
+
+## class Feature
+bases: TestCase
+
+base model for single feature
+
+
+### methods
+
+| Method            | Purpose                   |
+|-------------------|---------------------------|
+| run(**kwargs)     | run the feature           |
+| execute(**kwargs) | execute the Python method |
+
+
+### run(**kwargs)
+
+- allows executing any Python method by calling execute()
+- sets the following attributes required to push the results to DB:
+    - criteria
+    - start_time
+    - stop_time
+- doesn't fulfill details when pushing the results to the DB.
+
+
+### execute(**kwargs)
+
+- the subclasses must override the default implementation which is false on purpose
+- the new implementation must return 0 if success or anything else if failure.
+
+
+
+## Your second test case
+
+
+### second.py
+
+```python
+#!/usr/bin/env python
+
+from functest.core import feature
+
+class Test(feature.Feature):
+
+    def execute(self, **kwargs):
+        print "Hello World"
+        return 0
+```
+
+
+### functest/ci/testcases.yaml
+
+```yaml
+case_name: second
+project_name: functest
+criteria: 'status == "PASS"'
+blocking: true
+clean_flag: false
+description: ''
+dependencies:
+    installer: ''
+    scenario: ''
+run:
+    module: 'second'
+    class: 'Test'
+```
+
+
+
+## class BashFeature
+bases: Feature
+
+class designed to run any bash command
+
+
+### execute(**kwargs)
+
+execute the cmd passed as arg.
+
+
+
+## Your third test case
+
+
+### functest/ci/testcases.yaml
+
+```
+case_name: third
+project_name: functest
+criteria: 'status == "PASS"'
+blocking: true
+clean_flag: false
+description: ''
+dependencies:
+    installer: ''
+    scenario: ''
+run:
+    module: 'functest.core.feature'
+    class: 'BashFeature'
+    args:
+        cmd: 'echo Hello World; exit 0'
+```
+
+
+
+## Euphrates
+
+
+### Next actions
+
+- __to finish VNF abstraction (coverage + pylint)__
+- to publish doc API
+- to manage criteria as written in testcases.yaml
+
+Please see [Functest Euphrates page](https://wiki.opnfv.org/display/functest/Functest+Euphrates+page) for more details
+
+
+
+## Thank You!
author	Cédric Ollivier <cedric.ollivier@orange.com>	2017-04-20 16:58:41 +0200
committer	Cédric Ollivier <cedric.ollivier@orange.com>	2017-05-02 09:49:09 +0200
commit	cc584596a43af40db096015105d44e63c55cafbb (patch)
tree	7bb671f7aaf2711dd860358aae44f9fadef6ecff /docs/com
parent	90238c6c2c6b8c63062f3fe1c1611d594d72604b (diff)