diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/devguide/index.rst | 364 |
1 files changed, 209 insertions, 155 deletions
diff --git a/docs/devguide/index.rst b/docs/devguide/index.rst index 210774be6..cda38e0ed 100644 --- a/docs/devguide/index.rst +++ b/docs/devguide/index.rst @@ -301,7 +301,7 @@ page `[3]`_. Overall Architecture ==================== -The Test result management in Brahmaputra can be summarized as follows:: +The Test result management can be summarized as follows:: +-------------+ +-------------+ +-------------+ | | | | | | @@ -314,7 +314,7 @@ The Test result management in Brahmaputra can be summarized as follows:: +-----------------------------------------+ | | | Test Rest API front end | - | http://testresults.opnfv.org/testapi | + | http://testresults.opnfv.org/test | | | +-----------------------------------------+ A | @@ -333,24 +333,16 @@ The Test result management in Brahmaputra can be summarized as follows:: | | +----------------------+ -The Test dashboard URL is: TODO LF -A alternate Test dashboard has been realized to provide a view per installer and -per scenario for Brahmaputra release:: - - http://testresults.opnfv.org/proto/brahma/ - -This Dashboard consumes the results retrieved thanks to the Test API. - Test API description ==================== The Test API is used to declare pods, projects, test cases and test results. An -additional method dashboard has been added to post-process the raw results. The -data model is very basic, 4 objects are created: +additional method dashboard has been added to post-process the raw results in release Brahmaputra. +The data model is very basic, 4 objects are created: * Pods - * Test projects - * Test cases - * Test results + * Projects + * Testcases + * Results Pods:: @@ -359,10 +351,11 @@ Pods:: "details": <URL description of the POD>, "creation_date": YYYY-MM-DD HH:MM:SS , "name": <The POD Name>, - "mode": <metal or virtual> + "mode": <metal or virtual>, + "role": <ci-pod or community-pod or single-node> }, -Test project:: +Projects:: { "id": <ID>, @@ -371,185 +364,246 @@ Test project:: "description": <Short description> }, -Test case:: +Testcases:: { "id": <ID>, "name":<Name of the test case>, + "project_name":<Name of belonged project>, "creation_date": "YYYY-MM-DD HH:MM:SS", "description": <short description>, "url":<URL for longer description> }, -Test results:: +Results:: { "_id": <ID, + "case_name": <Reference to the test case> "project_name": <Reference to project>, "pod_name": <Reference to POD where the test was executed>, - "version": <Scenario on which the test was executed>, "installer": <Installer Apex or Compass or Fuel or Joid>, - "description": <Short description>, - "creation_date": "YYYY-MM-DD HH:MM:SS", - "case_name": <Reference to the test case> - "details":{ - <- the results to be put here -> - } - + "version": <master or Colorado or Brahmaputra>, + "start_date": "YYYY-MM-DD HH:MM:SS", + "stop_date": "YYYY-MM-DD HH:MM:SS", + "build_tag": <such as "jenkins-functest-fuel-baremetal-daily-master-108">, + "scenario": <Scenario on which the test was executed>, + "criteria": <PASS or FAILED>, + "trust_indicator": <0 ~ 1> -For Brahmaputra, we got: +The API can described as follows. For detailed information, please go to +http://testresults.opnfv.org/test/swagger/spec.html - * 16 pods - * 18 projects - * 101 test cases +Authentication: opnfv/api@opnfv -The projects and the test cases have been frozen in December. -But all were not ready for Brahmaputra. - - - -The API can described as follows: +Please notes that POST/DELETE/PUT operations for test or study purpose via +swagger website is not allowed, because it will change the real data in +the database. Version: +--------+--------------------------+-----------------------------------------+ | Method | Path | Description | +========+==========================+=========================================+ - | GET | /version | Get API version | + | GET | /versions | Get all supported API versions | +--------+--------------------------+-----------------------------------------+ Pods: - +--------+--------------------------+-----------------------------------------+ - | Method | Path | Description | - +========+==========================+=========================================+ - | GET | /pods | Get the list of declared Labs (PODs) | - +--------+--------------------------+-----------------------------------------+ - | POST | /pods | Declare a new POD | - | | | Content-Type: application/json | - | | | { | - | | | "name": "pod_foo", | - | | | "creation_date": "YYYY-MM-DD HH:MM:SS" | - | | | } | - +--------+--------------------------+-----------------------------------------+ + +--------+----------------------------+-----------------------------------------+ + | Method | Path | Description | + +========+============================+=========================================+ + | GET | /api/v1/pods | Get the list of declared Labs (PODs) | + +--------+----------------------------+-----------------------------------------+ + | POST | /api/v1/pods | Declare a new POD | + | | | Content-Type: application/json | + | | | { | + | | | "name": "pod_foo", | + | | | "mode": "metal", | + | | | "role": "ci-pod", | + | | | "details": "it is a ci pod" | + | | | } | + +--------+----------------------------+-----------------------------------------+ + | GET | /api/v1/pods/{pod_name} | Get a declared POD | + +--------+----------------------------+-----------------------------------------+ Projects: - +--------+--------------------------+-----------------------------------------+ - | Method | Path | Description | - +========+==========================+=========================================+ - | GET | /test_projects | Get the list of test projects | - +--------+--------------------------+-----------------------------------------+ - | GET |/test_projects/{project} | Get details on {project} | - | | | | - +--------+--------------------------+-----------------------------------------+ - | POST | /test_projects | Add a new test project | - | | | Content-Type: application/json | - | | | { | - | | | "name": "project_foo", | - | | | "description": "whatever you want" | - | | | } | - +--------+--------------------------+-----------------------------------------+ - | PUT | /test_projects/{project} | Update a test project | - | | | | - | | | Content-Type: application/json | - | | | { | - | | | <the field(s) you want to modify> | - | | | } | - +--------+--------------------------+-----------------------------------------+ - | DELETE | /test_projects/{project} | Delete a test project | - +--------+--------------------------+-----------------------------------------+ + +--------+----------------------------+-----------------------------------------+ + | Method | Path | Description | + +========+============================+=========================================+ + | GET | /api/v1/projects | Get the list of declared projects | + +--------+----------------------------+-----------------------------------------+ + | POST | /api/v1/projects | Declare a new test project | + | | | Content-Type: application/json | + | | | { | + | | | "name": "project_foo", | + | | | "description": "whatever you want" | + | | | } | + +--------+----------------------------+-----------------------------------------+ + | DELETE | /api/v1/projects/{project} | Delete a test project | + +--------+----------------------------+-----------------------------------------+ + | GET | /api/v1/projects/{project} | Get details on a {project} | + | | | | + +--------+----------------------------+-----------------------------------------+ + | PUT | /api/v1/projects/{project} | Update a test project | + | | | | + | | | Content-Type: application/json | + | | | { | + | | | <the field(s) you want to modify> | + | | | } | + +--------+----------------------------+-----------------------------------------+ + + +Testcases: + + +--------+----------------------------+-----------------------------------------+ + | Method | Path | Description | + +========+============================+=========================================+ + | GET | /api/v1/projects/{project}/| Get the list of testcases of {project} | + | | cases | | + +--------+----------------------------+-----------------------------------------+ + | POST | /api/v1/projects/{project}/| Add a new test case to {project} | + | | cases | Content-Type: application/json | + | | | { | + | | | "name": "case_foo", | + | | | "description": "whatever you want" | + | | | "url": "whatever you want" | + | | | } | + +--------+----------------------------+-----------------------------------------+ + | DELETE | /api/v1/projects/{project}/| Delete a test case | + | | cases/{case} | | + +--------+----------------------------+-----------------------------------------+ + | GET | /api/v1/projects/{project}/| Get a declared test case | + | | cases/{case} | | + +--------+----------------------------+-----------------------------------------+ + | PUT | /api/v1/projects/{project}?| Modify a test case of {project} | + | | cases/{case} | | + | | | Content-Type: application/json | + | | | { | + | | | <the field(s) you want to modify> | + | | | } | + +--------+----------------------------+-----------------------------------------+ + +Results: + + +--------+----------------------------+------------------------------------------+ + | Method | Path | Description | + +========+============================+==========================================+ + | GET | /api/v1/results | Get all the test results | + +--------+----------------------------+------------------------------------------+ + | POST | /api/v1/results | Add a new test results | + | | | Content-Type: application/json | + | | | { | + | | | "project_name": "project_foo", | + | | | "scenario": "odl-l2", | + | | | "stop_date": "2016-05-28T14:42:58.384Z", | + | | | "trust_indicator": 0.5, | + | | | "case_name": "vPing", | + | | | "build_tag": "", | + | | | "version": "Colorado", | + | | | "pod_name": "pod_foo", | + | | | "criteria": "PASS", | + | | | "installer": "fuel", | + | | | "start_date": "2016-05-28T14:41:58.384Z",| + | | | "details": <your results> | + | | | } | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of {case} | + | | case={case} | | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of build_tag | + | | build_tag={tag} | {tag}. | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get last {N} records of test results | + | | last={N} | | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of scenario | + | | scenario={scenario} | {scenario}. | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of trust_indicator | + | | trust_indicator={ind} | {ind}. | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of last days | + | | period={period} | {period}. | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of {project} | + | | project={project} | | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of version | + | | version={version} | {version}. | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of criteria | + | | criteria={criteria} | {criteria}. | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | get the results on pod {pod} | + | | pod={pod} | | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the test results of installer {inst} | + | | installer={inst} | | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results? | Get the results according to combined | + | | <query conditions> | query conditions supported above | + +--------+----------------------------+------------------------------------------+ + | GET | /api/v1/results/{result_id}| Get the test result by result_id | + +--------+----------------------------+------------------------------------------+ -Test cases: - - +--------+--------------------------+-----------------------------------------+ - | Method | Path | Description | - +========+==========================+=========================================+ - | GET | /test_projects/{project}/| Get the list of test cases of {project} | - | | cases | | - +--------+--------------------------+-----------------------------------------+ - | POST | /test_projects/{project}/| Add a new test case to {project} | - | | cases | Content-Type: application/json | - | | | { | - | | | "name": "case_foo", | - | | | "description": "whatever you want" | - | | | "creation_date": "YYYY-MM-DD HH:MM:SS" | - | | | "url": "whatever you want" | - | | | } | - +--------+--------------------------+-----------------------------------------+ - | PUT | /test_projects/{project}?| Modify a test case of {project} | - | | case_name={case} | | - | | | Content-Type: application/json | - | | | { | - | | | <the field(s) you want to modify> | - | | | } | - +--------+--------------------------+-----------------------------------------+ - | DELETE | /test_projects/{project}/| Delete a test case | - | | case_name={case} | | - +--------+--------------------------+-----------------------------------------+ - -Test Results: +Dashboard: - +--------+--------------------------+-----------------------------------------+ - | Method | Path | Description | - +========+==========================+=========================================+ - | GET |/results/project={project}| Get the test results of {project} | - +--------+--------------------------+-----------------------------------------+ - | GET |/results/case={case} | Get the test results of {case} | - +--------+--------------------------+-----------------------------------------+ - | GET |/results?pod={pod} | get the results on pod {pod} | - +--------+--------------------------+-----------------------------------------+ - | GET |/results?installer={inst} | Get the test results of installer {inst}| - +--------+--------------------------+-----------------------------------------+ - | GET |/results?version={version}| Get the test results of scenario | - | | | {version}. Initially the version param | - | | | was reflecting git version, in Functest | - | | | it was decided to move to scenario | - +--------+--------------------------+-----------------------------------------+ - | GET |/results?project={project}| Get all the results of the test case | - | |&case={case} | {case} of the project {project} with | - | |&version={scenario} | version {scenario} installed by | - | |&installer={installer} | {installer} on POD {pod} stored since | - | |&pod={pod} | {days} days | - | | | {project_name} and {case_name} are | - | |&period={days} | mandatory, the other parameters are | - | | | optional. | - +--------+--------------------------+-----------------------------------------+ - | POST | /results | Add a new test results | - | | | Content-Type: application/json | - | | | { | - | | | "project_name": "project_foo", | - | | | "case_name": "case_foo", | - | | | "pod_name": "pod_foo", | - | | | "installer": "installer_foo", | - | | | "version": "scenario_foo", | - | | | "details": <your results> | - | | | } | - +--------+--------------------------+-----------------------------------------+ + +--------+----------------------------+-----------------------------------------+ + | Method | Path | Description | + +========+============================+=========================================+ + | GET |/dashboard/v1/results? | Get all the dashboard ready results of | + | |&project={project} | {case} of the project {project} | + | |&case={case} | | + +--------+----------------------------+-----------------------------------------+ + | GET |/dashboard/v1/results? | Get all the dashboard ready results of | + | |&project={project} | {case} of the project {project} | + | |&case={case} | version {version} | + | |&version={version} | | + +--------+----------------------------+-----------------------------------------+ + | GET |/dashboard/v1/results? | Get all the dashboard ready results of | + | |&project={project} | {case} of the project {project} | + | |&case={case} | since {days} days | + | |&period={days} | | + +--------+----------------------------+-----------------------------------------+ + | GET |/dashboard/v1/results? | Get all the dashboard ready results of | + | |&project={project} | {case} of the project {project} | + | |&case={case} | installed by {installer} | + | |&installer={installer} | | + +--------+----------------------------+-----------------------------------------+ + | GET |/dashboard/v1/results? | Get all the dashboard ready results of | + | |&project={project} | {case} of the project {project} | + | |&case={case} | on POD {pod} | + | |&pod={pod} | | + +--------+----------------------------+-----------------------------------------+ + | GET |/dashboard/v1/results? | Get all the dashboard ready results of | + | |&project={project} | {case} of the project {project} | + | |&case={case} | and combined by other query conditions | + | |&<query conditions> | supported above. | + +--------+----------------------------+-----------------------------------------+ + | GET |/dashboard/v1/projects? | Get all the dashboard ready projects | + +--------+----------------------------+-----------------------------------------+ + +Dashboard description +===================== +The results with dashboard method are post-processed from raw results. +Please note that dashboard results are not stored. Only raw results are stored. -Dashboard: +Release Brahmaputra +------------------- - +--------+--------------------------+-----------------------------------------+ - | Method | Path | Description | - +========+==========================+=========================================+ - | GET |/dashboard? | Get all the dashboard ready results of | - | |&project={project} | {case} of the project {project} | - | |&case={case} | version {scenario} installed by | - | |&version={scenario} | {installer} on POD {pod} stored since | - | |&installer={installer} | {days} days | - | |&pod={pod} | | - | |&period={days} | {project_name} and {case_name} are | - | | | mandatory, the other parameters are | - | | | optional. | - +--------+--------------------------+-----------------------------------------+ +Dashboard url: http://testresults.opnfv.org/dashboard/ +Release Colorado +---------------- -The results with dashboard method are post-processed from raw results. -Please note that dashboard results are not stored. Only raw results are stored. +Dashboard url: http://testresults.opnfv.org/kibana_dashboards/ +Credentials for a guest account: opnfvuser/kibana =============================================== How to push your results into the Test Database |