diff options
| -rw-r--r-- | INFO | 2 | ||||
| -rw-r--r-- | docs/release/release-notes/euphrates.rst | 137 | ||||
| -rw-r--r-- | docs/testing/user/userguide/cli.rst | 55 | ||||
| -rw-r--r-- | qtip/scripts/quickstart.sh | 2 | ||||
| -rw-r--r-- | resources/ansible_roles/opnfv-testapi/tasks/report.yml | 2 | ||||
| -rw-r--r-- | resources/ansible_roles/qtip-generator/files/compute/host_vars/localhost.yml | 2 | ||||
| -rw-r--r-- | resources/ansible_roles/ramspeed/tasks/main.yml | 4 | ||||
| -rwxr-xr-x | tests/ci/experimental.sh | 2 | ||||
| -rwxr-xr-x | tests/ci/periodic.sh | 2 | ||||
| -rw-r--r-- | tests/ci/utils/start_services.sh | 1 |
10 files changed, 179 insertions, 30 deletions
@@ -12,9 +12,11 @@ IRC: #opnfv-qtip@freenode Committers: +Akhil Batra akhil.batra@research.iiit.ac.in Morgan Richomme morgan.richomme@orange.com Prabu Kuppuswamy prabu.kuppuswamy@spirent.com Prakash Ramchandran prakash.ramchandran@huawei.com +Taseer Ahmed taseer94@gmail.com Trevor Cooper trevor.cooper@intel.com Yujun Zhang zhang.yujunz@zte.com.cn Zhifeng Jiang Jiang.ZhiFeng@zte.com.cn diff --git a/docs/release/release-notes/euphrates.rst b/docs/release/release-notes/euphrates.rst new file mode 100644 index 00000000..dabff3d4 --- /dev/null +++ b/docs/release/release-notes/euphrates.rst @@ -0,0 +1,137 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +********* +Euphrates +********* + +This document provides the release notes of QTIP for OPNFV Euphrates release + +.. contents:: + :depth: 3 + :local: + +Version history +=============== + ++--------------------+--------------------+--------------------+--------------------+ +| **Date** | **Ver.** | **Author** | **Comment** | +| | | | | ++--------------------+--------------------+--------------------+--------------------+ +| 2017-10-20 | Euphrates 1.0 | Yujun Zhang | | +| | | | | ++--------------------+--------------------+--------------------+--------------------+ + +Summary +======= + +QTIP Euphrates release continues working on **QPI**, a.k.a. QTIP Performance Index, which is calculated from metrics +collected in performance tests. + +Besides compute performance benchmark, QTIP has integrated OPNFV storperf for storage performance benchmarking. + +A PoC of web portal is implemented as the starting point of Benchmarking as a Service. + +Release Data +============ + ++--------------------------------------+--------------------------------------+ +| **Project** | QTIP | +| | | ++--------------------------------------+--------------------------------------+ +| **Repo/commit-ID** | qtip/euphrates.1.0 | +| | | ++--------------------------------------+--------------------------------------+ +| **Release designation** | stable version | +| | | ++--------------------------------------+--------------------------------------+ +| **Release date** | 2017-10-20 | +| | | ++--------------------------------------+--------------------------------------+ +| **Purpose of the delivery** | release with OPNFV cycle | +| | | ++--------------------------------------+--------------------------------------+ + +Version change +-------------- + +Module version changes +^^^^^^^^^^^^^^^^^^^^^^ + +The following Python packages are used in this release:: + + humanfriendly==4.4.1 + connexion==1.1.11 + Jinja2==2.9.6 + Django==1.11.5 + asq==1.2.1 + six==1.11.0 + ansible==2.4.0.0 + requests==2.18.4 + prettytable==0.7.2 + numpy==1.13.1 + click==6.7 + pbr==3.1.1 + PyYAML==3.12 + +It is considered as a baseline for future releases. + +Reason for version +------------------ + +Features additions +^^^^^^^^^^^^^^^^^^ + +* Storage QPI (QTIP Performance Index) specification and benchmarking project + +Framework evolution +^^^^^^^^^^^^^^^^^^^ + +Ansible is used as the backbone of QTIP framework. Not only the main testing procedure is built as Ansible roles, but +also the inventory discovery is implemented as Ansible module, the calculation and collection actions are Ansible +plugins. Even the testing project itself is generated using jinja2 template rendering driven by Ansible. + +Deliverables +------------ + +Software +^^^^^^^^ + +- `QTIP Docker image <https://hub.docker.com/r/opnfv/qtip>`_ (tag: euphrates.1.0) + +Documentation +^^^^^^^^^^^^^ + +- `Installation & Configuration <http://docs.opnfv.org/en/stable-danube/qtip/docs/testing/user/configguide>`_ +- `User Guide <http://docs.opnfv.org/en/stable-danube/submodules/qtip/docs/testing/user/userguide>`_ +- `Developer Guide <http://docs.opnfv.org/en/stable-euphrates/submodules/qtip/docs/testing/developer/devguide>`_ + +Known Limitations, Issues and Workarounds +========================================= + +Limitations +----------- + +- Supporting on legacy OPNFV fuel installer is no longer maintained. + +Known issues +------------ + +Test Result +=========== + +QTIP has undergone QA test runs with the following results: + ++---------------------------------------------------+--------------------------------------+ +| **TEST-SUITES** | **Results:** | +| | | ++---------------------------------------------------+--------------------------------------+ +| qtip-verify-euphrates | 53/53 passed, 86% lines coverage | +| | | ++---------------------------------------------------+--------------------------------------+ +| qtip-compute-apex-euphrates | passed | +| | | ++---------------------------------------------------+--------------------------------------+ +| qtip-storage-apex-euphrates | passed | +| | | ++---------------------------------------------------+--------------------------------------+ diff --git a/docs/testing/user/userguide/cli.rst b/docs/testing/user/userguide/cli.rst index 8a983155..f0894b01 100644 --- a/docs/testing/user/userguide/cli.rst +++ b/docs/testing/user/userguide/cli.rst @@ -7,7 +7,7 @@ CLI User Manual *************** QTIP consists of a number of benchmarking tools or metrics, grouped under QPI's. QPI's map to the different -components of a NFVI ecosystem, such as compute, network and storage. Depending on the type of application, +components of a NFVi ecosystem, such as compute, network and storage. Depending on the type of application, a user may group them under plans. Bash Command Completion @@ -26,53 +26,60 @@ QTIP CLI provides interface to all of the above the components. A help page prov along with a short description. :: - qtip [-h|--help] + qtip --help Usage ===== -Typically a complete plan is executed at the target environment. QTIP defaults to a number of sample plans. -A list of all the available plans can be viewed +QTIP is currently supports two different QPI's, compute and storage. To list all the supported QPI :: - qtip plan list + qtip qpi list -In order to view the details about a specific plan. +The details of any QPI can be viewed as follows :: - qtip plan show <plan_name> +qtip qpi show <qpi_name> -where *plan_name* is one of those listed from the previous command. - -To execute a complete plan +In order to benchmark either one of them, their respective templates need to be generated :: - qtip plan run <plan_name> -p <path_to_result_directory> + qtip create --project-template [compute|storage] <workspace_name> + +By default, the compute template will be generated. An interactive prompt would gather all parameters specific to +OpenStack installation. -QTIP does not limit result storage at a specific directory. Instead a user may specify his own result storage -as above. An important thing to remember is to provide absolute path of result directory. +Once the template generation is complete, configuration for OpenStack needs to be generated. :: - mkdir result - qtip plan run <plan_name> -p $PWD/result + cd <workspace_name> + qtip setup -Similarly, the same commands can be used for the other two components making up the plans, i.e QPI's and metrics. -For example, in order to run a single metric +This step generates the inventory, populating it with target nodes. + +QTIP can now be run :: - qtip metric run <metric_name> -p $PWD/result + qtip run -The same can be applied for a QPI. +This would start the complete testing suite, which is either compute or storage. Each suite normally takes about +half an hour to complete. -QTIP also provides the utility to view benchmarking results on the console. One just need to provide to where -the results are stored. Extending the example above +Benchmarking report is made for each and every individual section in a QPI, on a particular target node. It consists of +the actual test values on that node along with scores calculated by comparison against a baseline. :: - qtip report show <metric_name> -p $PWD/result + qtip report show [-n|--node] <node> <section_name> + Debugging options ================= -Debug option helps identify the error by providing a detailed traceback. It can be enabled as +QTIP uses Ansible as the runner. One can use all of Ansible's CLI option with QTIP. In order to enable verbose mode +:: + + qtip setup -v + +One may also be able to achieve the different levels of verbosity :: - qtip [-d|--debug] plan run <plan_name> + qtip run [-v|-vv|-vvv] diff --git a/qtip/scripts/quickstart.sh b/qtip/scripts/quickstart.sh index 58a12459..826ed656 100644 --- a/qtip/scripts/quickstart.sh +++ b/qtip/scripts/quickstart.sh @@ -26,7 +26,7 @@ verify_connectivity(){ fi sleep 1 done - error "Can not talk to $ip." + echo "Can not talk to $ip." } #Getoptions diff --git a/resources/ansible_roles/opnfv-testapi/tasks/report.yml b/resources/ansible_roles/opnfv-testapi/tasks/report.yml index bb034d84..830f9398 100644 --- a/resources/ansible_roles/opnfv-testapi/tasks/report.yml +++ b/resources/ansible_roles/opnfv-testapi/tasks/report.yml @@ -26,6 +26,6 @@ scenario: "{{ scenario }}" start_date: "{{ ansible_date_time.date }}" stop_date: "{{ ansible_date_time.date }}" - criteria: "{{ pod_result.score }}" + criteria: 2048 details: score: "{{ pod_result.score }}" diff --git a/resources/ansible_roles/qtip-generator/files/compute/host_vars/localhost.yml b/resources/ansible_roles/qtip-generator/files/compute/host_vars/localhost.yml index 815e2ea3..cc587c69 100644 --- a/resources/ansible_roles/qtip-generator/files/compute/host_vars/localhost.yml +++ b/resources/ansible_roles/qtip-generator/files/compute/host_vars/localhost.yml @@ -13,4 +13,4 @@ project_name: qtip case_name: "{{ case_name|default('compute') }}" pod_name: "{{ pod_name|default('qtip-pod') }}" scenario: "{{ scenario|default('generic') }}" -version: "{{ version|default('master') }}" +version: "{{ lookup('env','OPNFV_RELEASE')|default('master') }}" diff --git a/resources/ansible_roles/ramspeed/tasks/main.yml b/resources/ansible_roles/ramspeed/tasks/main.yml index dbf16567..6f0b10fc 100644 --- a/resources/ansible_roles/ramspeed/tasks/main.yml +++ b/resources/ansible_roles/ramspeed/tasks/main.yml @@ -14,7 +14,9 @@ - name: downloading ramsmp get_url: - url: http://www.alasir.com/software/ramspeed/ramsmp-3.5.0.tar.gz +# alasir.com domain name expired on 2017-09-30 05:47:11 +# url: http://www.alasir.com/software/ramspeed/ramsmp-3.5.0.tar.gz + url: http://sources.buildroot.net/ramsmp-3.5.0.tar.gz dest: "{{ workdir }}" checksum: "sha256:39fb15493fb3c293575746d56f6ab9faaa1d876d8b1f0d8e5a4042d2ace95839" diff --git a/tests/ci/experimental.sh b/tests/ci/experimental.sh index bb9694b7..7fa18f2f 100755 --- a/tests/ci/experimental.sh +++ b/tests/ci/experimental.sh @@ -31,7 +31,7 @@ qtip_repo='/home/opnfv/repos/qtip' docker cp . ${TEST_SUITE}_qtip:${qtip_repo} docker exec ${TEST_SUITE}_qtip bash -c "cd ${qtip_repo} && pip install -U -e ." -docker exec -t ${TEST_SUITE}_qtip bash -x ${qtip_repo}/qtip/scripts/quickstart.sh +docker exec ${TEST_SUITE}_qtip bash -x ${qtip_repo}/qtip/scripts/quickstart.sh echo "QTIP: Verify ${TEST_SUITE} done!" exit 0 diff --git a/tests/ci/periodic.sh b/tests/ci/periodic.sh index 5c488b87..87fa7627 100755 --- a/tests/ci/periodic.sh +++ b/tests/ci/periodic.sh @@ -13,7 +13,7 @@ script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" source ${script_dir}/utils/start_services.sh -docker exec -t ${TEST_SUITE}_qtip bash -x /home/opnfv/repos/qtip/qtip/scripts/quickstart.sh +docker exec ${TEST_SUITE}_qtip bash -x /home/opnfv/repos/qtip/qtip/scripts/quickstart.sh echo "${TEST_SUITE} QPI done!" diff --git a/tests/ci/utils/start_services.sh b/tests/ci/utils/start_services.sh index 3c481be4..5c95ed7e 100644 --- a/tests/ci/utils/start_services.sh +++ b/tests/ci/utils/start_services.sh @@ -15,6 +15,7 @@ TEST_SUITE=${TEST_SUITE} NODE_NAME=${NODE_NAME:-opnfv-pod} SCENARIO=${DEPLOY_SCENARIO:-generic} TESTAPI_URL=${TESTAPI_URL:-} +OPNFV_RELEASE=${OPNFV_RELEASE:-} EOF export SSH_CREDENTIALS=${SSH_CREDENTIALS:-/root/.ssh} |