diff options
Diffstat (limited to 'docs')
6 files changed, 364 insertions, 25 deletions
diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst index 873a35cf9..4ebf0eceb 100644 --- a/docs/release/release-notes/release-notes.rst +++ b/docs/release/release-notes/release-notes.rst @@ -30,13 +30,16 @@ This document describes the release note of Yardstick project. Version History --------------- -+------------------+-----------+---------------------------------+ -| *Date* | *Version* | *Comment* | -| | | | -+------------------+-----------+---------------------------------+ -| October 20, 2017 | 5.0.0 | Yardstick for Euphrates release | -| | | | -+------------------+-----------+---------------------------------+ ++-------------------+-----------+---------------------------------+ +| *Date* | *Version* | *Comment* | +| | | | ++-------------------+-----------+---------------------------------+ +| December 15, 2017 | 5.1.0 | Yardstick for Euphrates release | +| | | | ++-------------------+-----------+---------------------------------+ +| October 20, 2017 | 5.0.0 | Yardstick for Euphrates release | +| | | | ++-------------------+-----------+---------------------------------+ Important Notes @@ -129,19 +132,19 @@ Release Data | **Project** | Yardstick | | | | +--------------------------------+-----------------------+ -| **Repo/tag** | yardstick/opnfv-5.0.0 | +| **Repo/tag** | yardstick/opnfv-5.1.0 | | | | +--------------------------------+-----------------------+ -| **Yardstick Docker image tag** | opnfv-5.0.0 | +| **Yardstick Docker image tag** | opnfv-5.1.0 | | | | +--------------------------------+-----------------------+ | **Release designation** | Euphrates | | | | +--------------------------------+-----------------------+ -| **Release date** | October 20, 2017 | +| **Release date** | December 15, 2017 | | | | +--------------------------------+-----------------------+ -| **Purpose of the delivery** | OPNFV Euphrates 5.0.0 | +| **Purpose of the delivery** | OPNFV Euphrates 5.1.0 | | | | +--------------------------------+-----------------------+ @@ -161,7 +164,7 @@ Software Deliverables ^^^^^^^^^^^^^^^^^^^^^ - - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: opnfv-5.0.0) + - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: opnfv-5.1.0) New Contexts @@ -475,6 +478,52 @@ Known Issues/Faults Corrected Faults ^^^^^^^^^^^^^^^^ +Euphrates 5.1.0: + ++---------------------+-------------------------------------------------------------------------+ +| **JIRA REFERENCE** | **DESCRIPTION** | +| | | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-841 | Fix various NSB license issues | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-73 | How To Work with Test Cases | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-500 | VNF testing documentation | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-826 | Allow overriding Heat IP addresses to match traffic generator profile | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-828 | Refactor doc/testing/user/userguide "Yardstick Installation" | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-830 | build_yardstick_image Ansible mount module doesn't work on Ubuntu 14.04 | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-833 | ansible_common transform password into lower case | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-847 | tc006, tc079, tc082 miss grafana dashboard in local deployment | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-849 | kill process do not accurately kill the process like "nova-api" | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-850 | tc023 miss description and tc050-58 wrong description | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-852 | tc078 cpu2006 fails in some situation | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-854 | yardstick docker lack of trex_client | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-867 | testcase tc078 have no data stored or dashboard to show results | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-871 | Remove img_modify_playbook assignation in build_yardstick_image.yml | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-829 | "nsb_setup.sh" doesn't parse the controller IP correctly | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-839 | NSB Prox BM test cases to be fixed for incorporating scale-up | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-840 | NSB Prox test documentation of vPE and LW-AFTR test cases | ++---------------------+-------------------------------------------------------------------------+ +| JIRA: YARDSTICK-848 | NSB "Prox" : Cleanup duplicated traffic profile | ++---------------------+-------------------------------------------------------------------------+ + + + + Euphrates 5.0.0: +---------------------+--------------------------------------------+ @@ -507,18 +556,6 @@ Euphratess 5.0.0 known restrictions/issues +-----------+-----------+----------------------------------------------+ -Open JIRA tickets ------------------ - -+---------------------+----------------------------------------------+ -| **JIRA REFERENCE** | **DESCRIPTION** | -| | | -+---------------------+----------------------------------------------+ -| JIRA: YARDSTICK-412 | IPv6 test case should add support for newton | -| | | -+---------------------+----------------------------------------------+ - - Useful links ------------ diff --git a/docs/testing/developer/devguide/devguide.rst b/docs/testing/developer/devguide/devguide.rst index 1d47e5605..dade49b75 100755 --- a/docs/testing/developer/devguide/devguide.rst +++ b/docs/testing/developer/devguide/devguide.rst @@ -361,7 +361,7 @@ Verify your patch locally before submitting Once you finish a patch, you can submit it to Gerrit for code review. A developer sends a new patch to Gerrit will trigger patch verify job on Jenkins -CI. The yardstick patch verify job includes python flake8 check, unit test and +CI. The yardstick patch verify job includes python pylint check, unit test and code coverage test. Before you submit your patch, it is recommended to run the patch verification in your local environment first. diff --git a/docs/testing/user/userguide/04-installation.rst b/docs/testing/user/userguide/04-installation.rst index dc528db6c..828c49581 100644 --- a/docs/testing/user/userguide/04-installation.rst +++ b/docs/testing/user/userguide/04-installation.rst @@ -331,6 +331,91 @@ For uninstalling Yardstick, just delete the virtual environment:: rm -rf ~/yardstick_venv +Install Yardstick directly in OpenSUSE +-------------------------------------- + +.. _install-framework: + +You can install Yardstick framework directly in OpenSUSE. + + +Install Yardstick +^^^^^^^^^^^^^^^^^ + +Prerequisite preparation:: + + sudo -EH zypper -n install -y gcc \ + wget \ + git \ + sshpass \ + qemu-tools \ + kpartx \ + libffi-devel \ + libopenssl-devel \ + python \ + python-devel \ + python-virtualenv \ + libxml2-devel \ + libxslt-devel \ + python-setuptools-git + +Create a virtual environment:: + + virtualenv ~/yardstick_venv + export YARDSTICK_VENV=~/yardstick_venv + source ~/yardstick_venv/bin/activate + sudo -EH easy_install -U setuptools + +Download the source code and install Yardstick from it:: + + git clone https://gerrit.opnfv.org/gerrit/yardstick + export YARDSTICK_REPO_DIR=~/yardstick + cd yardstick + sudo -EH python setup.py install + sudo -EH pip install -r requirements.txt + +Install missing python modules:: + + sudo -EH pip install pyyaml \ + oslo_utils \ + oslo_serialization \ + oslo_config \ + paramiko \ + python.heatclient \ + python.novaclient \ + python.glanceclient \ + python.neutronclient \ + scp \ + jinja2 + + +Configure the Yardstick environment +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Source the OpenStack environment variables:: + + source DEVSTACK_DIRECTORY/openrc + +Export the Openstack external network. The default installation of Devstack +names the external network public:: + + export EXTERNAL_NETWORK=public + export OS_USERNAME=demo + +Change the API version used by Yardstick to v2.0 (the devstack openrc sets it +to v3):: + + export OS_AUTH_URL=http://PUBLIC_IP_ADDRESS:5000/v2.0 + + +Uninstall Yardstick +^^^^^^^^^^^^^^^^^^^ + +For unistalling Yardstick, just delete the virtual environment:: + + rm -rf ~/yardstick_venv + + Verify the installation ----------------------- diff --git a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst index 43aa3d69a..895837283 100644 --- a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst +++ b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst @@ -26,3 +26,5 @@ NSB PROX Test Case Descriptions tc_prox_context_mpls_tagging_port tc_prox_context_buffering_port tc_prox_context_load_balancer_port + tc_prox_context_vpe_port + tc_prox_context_lw_after_port diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_lw_aftr_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_lw_aftr_port.rst new file mode 100644 index 000000000..5a1fada05 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_lw_aftr_port.rst @@ -0,0 +1,107 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +************************************************ +Yardstick Test Case Description: NSB PROX LwAFTR +************************************************ + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_lw_aftr-{port_num} | +| | | +| | * context = baremetal or heat_context; | +| | * port_num = 4; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The PROX LW_AFTR test will take packets in from one | +| | port and remove the ipv6 encapsulation and forward them to | +| | another port. While forwarded packets in other direction | +| | will be encapsulated in an ipv6 header. | +| | | +| | The lw_aftr test cases are implemented to run in baremetal | +| | and heat context an require 4 port topology to run the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The LW_AFTR test cases are listed below: | +| | | +| | * tc_prox_baremetal_lw_aftr-4.yaml | +| | * tc_prox_heat_context_lw_aftr-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | The minimum packet size for MPLS test is 68 bytes. This is | +| | set in the traffic profile and can be configured to use | +| | higher packet sizes. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | The PROX lwAFTR test cases can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(LW_AFTR workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 86 bytes | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_vpe_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_vpe_port.rst new file mode 100644 index 000000000..6827b0525 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_vpe_port.rst @@ -0,0 +1,108 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +********************************************** +Yardstick Test Case Description: NSB PROXi VPE +********************************************** + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_vpe-{port_num} | +| | | +| | * context = baremetal or heat_context; | +| | * port_num = 4; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The PROX VPE test handles packet processing, routing, QinQ | +| | encapsulation, flows, ACL rules, adds/removes MPLS tagging | +| | and performs QoS before forwarding packet to another port. | +| | The reverse applies to forwarded packets in the other | +| | direction. | +| | | +| | The VPE test cases are implemented to run in baremetal | +| | and heat context an require 4 port topology to run the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The VPE test cases are listed below: | +| | | +| | * tc_prox_baremetal_vpe-4.yaml | +| | * tc_prox_heat_context_vpe-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | The minimum packet size for VPE test is 68 bytes. This is | +| | set in the traffic profile and can be configured to use | +| | higher packet sizes. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | The PROX VPE test cases can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(VPE workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 68 bytes | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + |