Test API

Clean, Easy, Complete


OPNFV testing community

OPNFV Design Summit, 20/6/2016, Berlin

# Agenda * A test API: what for? * API overview * API evolution
# A test API: what for?
## Give a consistant view for * Test projects * Test cases * Test resources (Pods) * Test results
## Unify result display * From many projects * Many formats (log, html, json, ..) * Many locations (in VM, Jumphost, external DB,...)
## Help building * Dashboards * Automatic reporting
# API overview
## API in Brahmaputra * Tornado + MongoDB * Simple data models aggreed with the testing group * No unit tests * Wiki and rst documentation
## API in Brahmaputra ![testapi](https://wiki.opnfv.org/download/attachments/2926452/results_collection_structure.png?version=1&modificationDate=1459196347000&api=v2 "OPNFV API page")
## API in Brahmaputra ![testapi](../img/testapi0.png) https://wiki.opnfv.org/display/functest/Collection+Of+Test+Results
## Lessons learned in B. * Wiki documentation is painful * result modl too simple * version used for scenario * no version * no overall criteria (passed/failed) * need unit tests for data model evolution
## Colorado refactoring * done by Serena Feng (ZTE) * update of the data model (based on lesson learned) * creation of a swagger tornado framework for the doc * creation of unit tests
#API evolution

Test API evolution

Field Brahmaputra Colorado
date creation_date start_date and stop_date
version used as scenario git version
scenario N.R used for scenario
test_criteria N.R passed/failed
trust_indictator N.R between 0 and 1
last N.R get last N results
## Swagger doc ![alt text](../img/testapi1.png "Test API swagger interface")

unit tests


umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh 
Tests running...
WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing
WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms
WARNING:tornado.general:400 POST /api/v1/projects (127.0.0.1): name missing
............
WARNING:tornado.access:400 POST /api/v1/projects (127.0.0.1) 1.13ms
WARNING:tornado.access:403 PUT /api/v1/projects/functest/cases/vping_1 (127.0.0.1) 2.95ms
WARNING:tornado.general:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1): {'project_name': u'functest', 'name': u'notFound'} could not be found in table [testcases]
WARNING:tornado.access:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1) 2.85ms

Ran 74 tests in 1.848s
OK

Thank you

tests
OPNFV logo