Test API

Clean, Easy, Complete

OPNFV testing community

OPNFV Design Summit, 20/6/2016, Berlin

# Agenda * A test API: what for? * API overview * API evolution

# A test API: what for?

## Give a consistant view for * Test projects * Test cases * Test resources (Pods) * Test results

## Unify result display * From many projects * Many formats (log, html, json, ..) * Many locations (in VM, Jumphost, external DB,...)

## Help building * Dashboards * Automatic reporting

# API overview

## API in Brahmaputra * Tornado + MongoDB * Simple data models aggreed with the testing group * No unit tests * Wiki and rst documentation

## API in Brahmaputra ![testapi](https://wiki.opnfv.org/download/attachments/2926452/results_collection_structure.png?version=1&modificationDate=1459196347000&api=v2 "OPNFV API page")

## API in Brahmaputra ![testapi](../img/testapi0.png) https://wiki.opnfv.org/display/functest/Collection+Of+Test+Results

## Lessons learned in B. * Wiki documentation is painful * result modl too simple * version used for scenario * no version * no overall criteria (passed/failed) * need unit tests for data model evolution

## Colorado refactoring * done by Serena Feng (ZTE) * update of the data model (based on lesson learned) * creation of a swagger tornado framework for the doc * creation of unit tests

#API evolution

Test API evolution

Field	Brahmaputra	Colorado
date	creation_date	start_date and stop_date
version	used as scenario	git version
scenario	N.R	used for scenario
criteria	N.R	PASS/FAILED
trust_indictator	N.R	between 0 and 1
last	N.R	get last N results

## Swagger doc ![alt text](../img/testapi1.png "Test API swagger interface")

unit tests


umry8364@umry8364-Latitude-E6400:~/Dev/OPNFV/releng/utils/test/result_collection_api$ ./run_test.sh 
Tests running...
WARNING:tornado.general:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1): Project name missing
WARNING:tornado.access:404 GET /dashboard/v1/results?case=vPing&pod=zte-pod1&version=C&installer=fuel&period=5 (127.0.0.1) 2.30ms
WARNING:tornado.general:400 POST /api/v1/projects (127.0.0.1): name missing
............
WARNING:tornado.access:400 POST /api/v1/projects (127.0.0.1) 1.13ms
WARNING:tornado.access:403 PUT /api/v1/projects/functest/cases/vping_1 (127.0.0.1) 2.95ms
WARNING:tornado.general:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1): {'project_name': u'functest', 'name': u'notFound'} could not be found in table [testcases]
WARNING:tornado.access:404 PUT /api/v1/projects/functest/cases/notFound (127.0.0.1) 2.85ms

Ran 74 tests in 1.848s
OK

Thank you