Test API

Clean, Easy, Complete


OPNFV testing community

OPNFV Design Summit, 20/6/2016, Berlin

# Agenda * A test API: what for? * API overview * API evolution
# A test API: what for?
## Give a consistant view for * Test projects * Test cases * Test resources (Pods) * Test results
## Unify result display * From many projects * Many formats (log, html, json, ..) * Many locations (in VM, Jumphost, external DB,...)
## Help building * Dashboards * Automatic reporting
# API overview
## API in Brahmaputra * Tornado + MongoDB * Simple data models aggreed with the testing group * No unit tests * Wiki and rst documentation
## API in Brahmaputra ![testapi](https://wiki.opnfv.org/download/attachments/2926452/results_collection_structure.png?version=1&modificationDate=1459196347000&api=v2 "OPNFV API page")
## API in Brahmaputra ![testapi](../img/testapi0.png) https://wiki.opnfv.org/display/functest/Collection+Of+Test+Results
## Lessons learned in B. * Wiki documentation is painful * Result model too simple * version used instead of scenario * no version * no overall criteria (PASS/FAIL) * need unit tests for data model evolution
## Colorado refactoring * Done by Serena Feng (ZTE) * Update of the data model (based on lessons learned) * New swagger tornado framework for the doc * URL and Mongo collection renaming * Scripts to manage backup/restore/DB update * Setup scripts * Unit tests
# Result management changes
## main changes An URI change... testresults.opnfv.org/test/testapi => testresults.opnfv.org/test/api/v1 http://testresults.opnfv.org/test/api/v1/projects http://testresults.opnfv.org/test/api/v1/projects/yardstick/cases http://testresults.opnfv.org/test/api/v1/results?case=Tempest&period=1

One new field for pods

Field Brahmaputra Colorado
role N.R community_pod, ci_pod, ..

New fields and filters for results

Item Brahmaputra Colorado
date creation_date start_date and stop_date
version used as scenario git version
scenario N.R used for scenario
test_criteria N.R passed/failed
trust_indicator N.R between 0 and 1
last N.R get last N results
http://testresults.opnfv.org/test/swagger/spec.html ![alt text](../img/testapi1.png "Test API swagger interface")

unit tests


umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh 
Tests running...
WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing
WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms
WARNING:tornado.general:400 POST /api/v1/projects (127.0.0.1): name missing
............
WARNING:tornado.access:400 POST /api/v1/projects (127.0.0.1) 1.13ms
WARNING:tornado.access:403 PUT /api/v1/projects/functest/cases/vping_1 (127.0.0.1) 2.95ms
WARNING:tornado.general:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1): {'project_name': u'functest', 'name': u'notFound'} could not be found in table [testcases]
WARNING:tornado.access:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1) 2.85ms

Ran 74 tests in 1.848s
OK

Thank you

tests
OPNFV logo