diff options
Diffstat (limited to 'docs/testing/user/userguide/api_testing_guide.rst')
| -rw-r--r-- | docs/testing/user/userguide/api_testing_guide.rst | 373 |
1 files changed, 373 insertions, 0 deletions
diff --git a/docs/testing/user/userguide/api_testing_guide.rst b/docs/testing/user/userguide/api_testing_guide.rst new file mode 100644 index 00000000..119beff7 --- /dev/null +++ b/docs/testing/user/userguide/api_testing_guide.rst @@ -0,0 +1,373 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Technologies Co.,Ltd and others. + +=============================== +Running Dovetail by RESTful API +=============================== + +Overview +-------- + +Dovetail framework provides RESTful APIs for end users to run all OVP test cases. +Also it provides a Swagger UI for users to find out all APIs and try them out. + + +Definitions and abbreviations +----------------------------- + +- REST - Representational State Transfer +- API - Application Programming Interface +- OVP - OPNFV Verification Program +- UI - User Interface + + +Environment Preparation +----------------------- + + +Install Docker +^^^^^^^^^^^^^^ + +The main prerequisite software for Dovetail is Docker. Please refer to official +Docker installation guide that is relevant to your Test Host's operating system. + + +Configuring the Test Host Environment +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For convenience and as a convention, we will create a home directory for storing +all Dovetail related config items and results files: + +.. code-block:: bash + + $ mkdir -p ${HOME}/dovetail + $ export DOVETAIL_HOME=${HOME}/dovetail + + +Installing Dovetail API +----------------------- + +The Dovetail project maintains a Docker image that has both Dovetail API and +Dovetail CLI preinstalled. This Docker image is tagged with versions. +Before pulling the Dovetail image, check the OPNFV's OVP web page first to +determine the right tag for OVP testing. + + +Downloading Dovetail Docker Image +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The first version of Dovetail API is ovp-3.0.0. + +.. code-block:: bash + + $ sudo docker pull opnfv/dovetail:ovp-3.0.0 + ovp-3.0.0: Pulling from opnfv/dovetail + 6abc03819f3e: Pull complete + 05731e63f211: Pull complete + 0bd67c50d6be: Pull complete + 3f737f5d00b2: Pull complete + c93fd0792ebd: Pull complete + 77d9a9603ec6: Pull complete + 9463cdd9c628: Pull complete + Digest: sha256:45e2ffdbe217a4e6723536afb5b6a3785d318deff535da275f34cf8393af458d + Status: Downloaded newer image for opnfv/dovetail:ovp-3.0.0 + + +Deploying Dovetail API +^^^^^^^^^^^^^^^^^^^^^^ + +The Dovetail API can be deployed by running a Dovetail container with the Docker +image downloaded before. + +.. code-block:: bash + + $ docker run -itd -p <swagger_port>:80 -p <api_port>:5000 --privileged=true \ + -e SWAGGER_HOST=<host_ip>:<api_port> -e DOVETAIL_HOME=/home/ovp \ + -v /home/ovp:/home/ovp -v /var/run/docker.sock:/var/run/docker.sock \ + opnfv/dovetail:<version> + + +In the container, it uses 2 ports for Swagger UI (port 80) and API (port 5000) +respectively. So in order to access to these 2 services outside the container, +it needs to map them to the host ports. It can be any available ports in the host. + +The env SWAGGER_HOST is optional. If you will access the Swagger UI webpage with +the same host deploying this container, there is no need to set SWAGGER_HOST. +Otherwise, if you will access the Swagger UI webpage from other machines, then +it needs to set SWAGGER_HOST. + + +Using Dovetail API +------------------ + +Here give the guide of where to find out all APIs and how to use them. + + +Swagger UI Webpage +^^^^^^^^^^^^^^^^^^ + +After deploying Dovetail container, the Swagger UI webpage can be accessed with +any browser. The url is ``http://localhost:<swagger_port>/dovetail-api/index.html`` +if accessing from the same host as deploying this container. Otherwise, the url +is ``http://<host_ip>:<swagger_port>/dovetail-api/index.html``. + + +Calling APIs +^^^^^^^^^^^^ + +There are totally 5 APIs provided by Dovetail. + + * Get all test suites + + * Get all test cases + + * Run test cases + + * Run test cases with execution ID + + * Get status of test cases + +Here give some easy guide of how to call these APIs. For more detailed infomation, +please refer to the Swagger UI page. + + +Getting All Test Suites +======================= + + * This is a **GET** function with no parameter to get all test suites defined + in Dovetail container. + + * The request URL is ``http://<host_ip>:<api_port>/api/v1/scenario/nfvi/testsuites``. + + * The response body is structured as: + + .. code-block:: bash + + { + "testsuites": { + "debug": { + "name": "debug", + "testcases_list": { + "optional": [ + "functest.vping.userdata" + ] + } + }, + "healthcheck": { + "name": "healthcheck", + "testcases_list": { + "optional": [ + "functest.healthcheck.connection_check" + ] + } + } + } + } + + +Getting All Test Cases +====================== + + * This is a **GET** function without no parameter to get all test cases integrated + in Dovetail container. + + * The request URL is ``http://<host_ip>:<api_port>/api/v1/scenario/nfvi/testcases``. + + * The response body is structured as: + + .. code-block:: bash + + { + "testcases": [ + { + "description": "This test case will verify the high availability of the user service provided by OpenStack (keystone) on control node.", + "scenario": "nfvi", + "subTestCase": null, + "testCaseName": "yardstick.ha.keystone" + }, + { + "description": "testing for vping using userdata", + "scenario": "nfvi", + "subTestCase": null, + "testCaseName": "functest.vping.userdata" + }, + { + "description": "tempest smoke test cases about volume", + "scenario": "nfvi", + "subTestCase": [ + "tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance[compute,id-fff42874-7db5-4487-a8e1-ddda5fb5288d,smoke]", + "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern[compute,id-557cd2c2-4eb8-4dce-98be-f86765ff311b,image,slow,volume]" + ], + "testCaseName": "functest.tempest.volume" + } + ] + } + + +Running Test Cases +================== + + * This is a **POST** function with some parameters to run a subset of the whole test cases. + + * The request URL is ``http://<host_ip>:<api_port>/api/v1/scenario/nfvi/execution``. + + * The request body is structured as following. The ``conf`` section is used to + give all configuration items those are required to run test cases. They are + the same as all configuration files provided under ``$DOVETAIL_HOME/pre_config/``. + If you already have these files under this directory, the whole ``conf`` section + can be ignored. If you provide these configuration items with the request body, + then the corresponding files under ``$DOVETAIL_HOME/pre_config/`` will be ignored + by Dovetail. The ``testcase``, ``testsuite``, ``testarea`` and ``deploy_scenario`` + correspond to ``--testcase``, ``--testsuite``, ``--testarea`` and ``--deploy-scenario`` + defined with Dovetail CLI. The ``options`` section support to set all options + which have already been implemented by Dovetail CLI including ``--optional``, + ``--mandatory``, ``--no-clean``, ``--no-api-validation``, ``--offline``, + ``--report``, ``--stop`` and ``--debug``. For options list in ``options`` section, + they are set to be ``True``, otherwise, they are set to be ``False``. + + .. code-block:: bash + + { + "conf": { + "vm_images": "/home/ovp/images", + "pods": { + "nodes": [ + { + "name": "node1", + "role": "Controller", + "ip": "192.168.117.222", + "user": "root", + "password": "root", + } + ], + "process_info": [ + { + "testcase_name": "yardstick.ha.rabbitmq", + "attack_host": "node1", + "attack_process": "rabbitmq" + } + ] + }, + "tempest_conf": { + "compute": { + "min_compute_nodes": "2", + "volume_device_name": "vdb", + "max_microversion": "2.65" + } + }, + "hosts": { + "192.168.141.101": [ + "volume.os.com", + "compute.os.com" + ] + }, + "envs": { + "OS_USERNAME": "admin", + "OS_PASSWORD": "admin", + "OS_AUTH_URL": "https://192.168.117.222:5000/v3", + "EXTERNAL_NETWORK": "ext-net" + } + }, + "testcase": [ + "functest.vping.ssh", + "yardstick.ha.rabbitmq" + ], + "testsuite": "ovp.2019.12", + "testarea": [ + "vping", + "ha" + ], + "deploy_scenario": "os-nosdn-ovs-ha", + "options": [ + "debug", + "report" + ] + } + + + * The response body is structured as: + + .. code-block:: bash + + { + "result": [ + { + "endTime": null, + "executionId": "a65e24c0-1803-11ea-84f4-0242ac110004", + "results": null, + "scenario": "nfvi", + "status": "IN_PROGRESS", + "testCaseName": "functest.vping.ssh", + "testSuiteName": "ovp.2019.12", + "timestart": null + } + ] + } + + +Running Test Cases with Execution ID +==================================== + + * This is a **POST** function with some parameters to run a subset of + whole test cases and set the execution ID instead of using the random one. + + * The request URL is ``http://<host_ip>:<api_port>/api/v1/scenario/nfvi/execution/{exec_id}``. + + * It's almost the same as the above running test cases API except the execution ID. + + +Getting Status of Test Cases +============================ + + * This is a **POST** function to get the status of some test cases by using + the execution ID received in the response body of `Running Test Cases`_ or + `Running Test Cases with Execution ID`_ APIs. + + * The request URL is ``http://<host_ip>:<api_port>/api/v1/scenario/nfvi/execution/status/{exec_id}``. + + * The request body is structured as: + + .. code-block:: bash + + { + "testcase": [ + "functest.vping.ssh" + ] + } + + * The response body is structured as: + + .. code-block:: bash + + { + "result": [ + { + "endTime": "2019-12-06 08:39:23", + "executionId": "a65e24c0-1803-11ea-84f4-0242ac110004", + "results": { + "criteria": "PASS", + "sub_testcase": [], + "timestart": "2019-12-06 08:38:40", + "timestop":"2019-12-06 08:39:23" + }, + "scenario": "nfvi", + "status": "COMPLETED", + "testCaseName": "functest.vping.ssh", + "testSuiteName": "ovp.2019.12", + "timestart":"2019-12-06 08:38:40" + } + ] + } + + + + +Getting Test Results +^^^^^^^^^^^^^^^^^^^^ + +Each time you call the running test case API, Dovetail creates a directory with the +execution ID as the name under ``$DOVETAIL_HOME`` to store results on the host. +You can find all result files under ``$DOVETAIL_HOME/<executionId>/results``. +If you run test cases with ``report`` option, then there will be a tarball file +under ``$DOVETAIL_HOME/<executionId>`` which can be upload to OVP portal. |