From 8ac28e7adc50aa09efbd4b1ddbcc504348ee80c2 Mon Sep 17 00:00:00 2001 From: Morgan Richomme Date: Thu, 2 Jun 2016 15:38:40 +0200 Subject: Presentation Test community Design Summit Change-Id: I34393b0398f1c7a4f7aef9fea79f265ec5c915b5 Signed-off-by: Morgan Richomme --- docs/com/pres/conversation.html | 204 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 204 insertions(+) create mode 100755 docs/com/pres/conversation.html (limited to 'docs/com') diff --git a/docs/com/pres/conversation.html b/docs/com/pres/conversation.html new file mode 100755 index 000000000..fab46247b --- /dev/null +++ b/docs/com/pres/conversation.html @@ -0,0 +1,204 @@ + + + + + + + OPNFV presentation + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ +
+
+

Conversation with the Testing Community

+

+

OPNFV testing community

+
OPNFV Design Summit, 20/6/2016, Berlin
+
+ +
+ +
+ >“You make experiments and I make theories. Do you know the difference? A theory is something nobody believes, except the person who made it. An experiment is something everybody believes, except the person who made it." A.Einstein +
+ +
+ # Agenda + * Who are we? + * Upstream, Upstream, Upstream + * Towards Telco Cloud KPI +
+ +
+ # Who are we? + ![alt text](https://wiki.opnfv.org/download/attachments/2926690/Testing.png "OPNFV testing group") +
+
+
+

The test projects

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Test caseDescription
FunctestUmbrella project for Functional testing
YardstickUmbrella project for performance/qualification testing
CPerfSDN Controller performance testing
StorPerfStorage performance testing
VSPerfVSwitch qualification
BottlenecksDetect possible bottlenecks
+
+ +
+ # Functest + * Functional testing to validate scenarios + * Contributors (raise your hands) + * Functest presentation (here...1h ago :)) slide available here + * breakout sessions + * Today 4PM (feature project integration), 5PM (Colorado status) + * tomorrow 11AM (API, test collection), 1PM (work Upstream), 3.15PM (D Release)(to be confirmed...) +
+
+ # Yardstick +
+
+ # CPerf +
+
+ # StorPerf +
+
+ # VSPerf +
+
+ # Bottlenecks +
+
+
+
+ # Upstream, upstream, upstream.. +
+
+ ## We are on the shoulders of the giants + * Rally (OpenStack) + * ODL, ONOS, ... + * RobotFramework, teston + * ..... +
+
+ ## Time to give back +
+ +
+ ## How to improve work with testing upstream community +
+ +
+ +
+
+ # Towards telco Cloud API... +
+ +
+ ## +
+ +
+ +
+

Thank you

+ tests +
+ +
+
+ OPNFV logo +
+
+ + + + + + + + -- cgit 1.2.3-korg From 3e54264eca6870e52fb8adcff06d6710d17c8c71 Mon Sep 17 00:00:00 2001 From: Morgan Richomme Date: Fri, 3 Jun 2016 15:29:38 +0200 Subject: Add presentation for test API Change-Id: I23a474247680751bb4f9ac0cc17b00229edd8886 Signed-off-by: Morgan Richomme --- docs/com/img/testapi0.png | Bin 0 -> 37907 bytes docs/com/img/testapi1.png | Bin 0 -> 63056 bytes docs/com/pres/testapi.html | 236 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 236 insertions(+) create mode 100644 docs/com/img/testapi0.png create mode 100644 docs/com/img/testapi1.png create mode 100755 docs/com/pres/testapi.html (limited to 'docs/com') diff --git a/docs/com/img/testapi0.png b/docs/com/img/testapi0.png new file mode 100644 index 000000000..06df74d75 Binary files /dev/null and b/docs/com/img/testapi0.png differ diff --git a/docs/com/img/testapi1.png b/docs/com/img/testapi1.png new file mode 100644 index 000000000..e9a697449 Binary files /dev/null and b/docs/com/img/testapi1.png differ diff --git a/docs/com/pres/testapi.html b/docs/com/pres/testapi.html new file mode 100755 index 000000000..4479b93b4 --- /dev/null +++ b/docs/com/pres/testapi.html @@ -0,0 +1,236 @@ + + + + + + + OPNFV presentation + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ +
+

Test API

+

Clean, Easy, Complete

+
+

OPNFV testing community

+
OPNFV Design Summit, 20/6/2016, Berlin
+
+
+ +
+ # Agenda + * A test API: what for? + * API overview + * API evolution +
+ +
+
+ # A test API: what for? +
+
+ ## Give a consistant view for + * Test projects + * Test cases + * Test resources (Pods) + * Test results +
+ +
+ ## Unify result display + * From many projects + * Many formats (log, html, json, ..) + * Many locations (in VM, Jumphost, external DB,...) +
+ +
+ ## Help building + * Dashboards + * Automatic reporting +
+
+ +
+
+ # API overview +
+
+ ## API in Brahmaputra + * Tornado + MongoDB + * Simple data models aggreed with the testing group + * No unit tests + * Wiki and rst documentation +
+ +
+ ## API in Brahmaputra + ![testapi](https://wiki.opnfv.org/download/attachments/2926452/results_collection_structure.png?version=1&modificationDate=1459196347000&api=v2 "OPNFV API page") +
+
+ ## API in Brahmaputra + ![testapi](../img/testapi0.png) + https://wiki.opnfv.org/display/functest/Collection+Of+Test+Results +
+ +
+ ## Lessons learned in B. + * Wiki documentation is painful + * result modl too simple + * version used for scenario + * no version + * no overall criteria (passed/failed) + * need unit tests for data model evolution +
+
+ ## Colorado refactoring + * done by Serena Feng (ZTE) + * update of the data model (based on lesson learned) + * creation of a swagger tornado framework for the doc + * creation of unit tests +
+
+ +
+
+ #API evolution +
+
+

Test API evolution

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldBrahmaputraColorado
datecreation_datestart_date and stop_date
versionused as scenariogit version
scenarioN.Rused for scenario
test_criteriaN.Rpassed/failed
trust_indictatorN.Rbetween 0 and 1
lastN.Rget last N results
+
+
+ ## Swagger doc + ![alt text](../img/testapi1.png "Test API swagger interface") +
+
+

unit tests

+ 

+umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh 
+Tests running...
+WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing
+WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms
+WARNING:tornado.general:400 POST /api/v1/projects (127.0.0.1): name missing
+............
+WARNING:tornado.access:400 POST /api/v1/projects (127.0.0.1) 1.13ms
+WARNING:tornado.access:403 PUT /api/v1/projects/functest/cases/vping_1 (127.0.0.1) 2.95ms
+WARNING:tornado.general:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1): {'project_name': u'functest', 'name': u'notFound'} could not be found in table [testcases]
+WARNING:tornado.access:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1) 2.85ms
+
+Ran 74 tests in 1.848s
+OK
+
+
+
+
+

Thank you

+ tests +
+ +
+
+ OPNFV logo +
+
+ + + + + + + + -- cgit 1.2.3-korg