diff options
| author |   Maciej Skrocki <maciej.skrocki@intel.com> | 2017-10-19 14:02:00 -0700 |
|---|---|---|
| committer |   Jing Lu <lvjing5@huawei.com> | 2017-10-20 23:41:17 +0000 |
| commit | e9730038f1bc4c2f781518faf3a7335ad58d0441 (patch) | |
| tree | e6290a76bbec85108298c171b25704498cdbe3ed /docs/testing/user/userguide | |
| parent | 1f04604b780674d6aede2bac75a643bb1d993576 (diff) | |
Added NSB sample Test Case documentation.
Update:
Added Prox NSB test case documentation
Change-Id: Ia2f47d823581fc7f193d3123f080dd87f29d4b99
Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com>
Signed-off-by: Abhijit Sinha <abhijit.sinha@intel.com>
(cherry picked from commit 10c595ad0301d02944ae412e994e65027e9d69e6)
Diffstat (limited to 'docs/testing/user/userguide')
10 files changed, 911 insertions, 0 deletions
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index c3dc57a9f..61e157e52 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -27,5 +27,6 @@ Yardstick User Guide 12-nsb_installation 13-nsb_operation 15-list-of-tcs + nsb/nsb-list-of-tcs glossary references diff --git a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst new file mode 100644 index 000000000..43aa3d69a --- /dev/null +++ b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst @@ -0,0 +1,28 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +===================== +NSB Sample Test Cases +===================== + +Abstract +======== + +This chapter lists available NSB test cases. + +NSB PROX Test Case Descriptions +=============================== + +.. toctree:: + :maxdepth: 1 + + tc_prox_context_acl_port + tc_prox_context_bng_port + tc_prox_context_bng_qos_port + tc_prox_context_l2fwd_port + tc_prox_context_l3fwd_port + tc_prox_context_mpls_tagging_port + tc_prox_context_buffering_port + tc_prox_context_load_balancer_port diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_acl_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_acl_port.rst new file mode 100644 index 000000000..137486151 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_acl_port.rst @@ -0,0 +1,112 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +********************************************* +Yardstick Test Case Description: NSB PROX ACL +********************************************* + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_acl-{port_num} | +| | | +| | * context = baremetal or heat_context; | +| | * port_num = 2 or 4; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | This test allows to measure how well the SUT can exploit | +| | structures in the list of ACL rules. | +| | The ACL rules are matched against a 7-tuple of the input | +| | packet: the regular 5-tuple and two VLAN tags. | +| | The rules in the rule set allow the packet to be forwarded | +| | and the rule set contains a default "match all" rule. | +| | | +| | The KPI is measured with the rule set that has a moderate | +| | number of rules with moderate similarity between the rules & | +| | the fraction of rules that were used. | +| | | +| | The ACL test cases are implemented to run in baremetal | +| | and heat context for 2 port and 4 port configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The ACL test cases are listed below: | +| | | +| | * tc_prox_baremetal_acl-2.yaml | +| | * tc_prox_baremetal_acl-4.yaml | +| | * tc_prox_heat_context_acl-2.yaml | +| | * tc_prox_heat_context_acl-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | Packet size set as 64 bytes in traffic profile. | +| | These can be configured | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This PROX ACL test cases can be configured with different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. The test need multi-queue enabled in Glance image. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(ACL workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 64 bytes | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_bng_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_bng_port.rst new file mode 100644 index 000000000..b674846ff --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_bng_port.rst @@ -0,0 +1,109 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +********************************************* +Yardstick Test Case Description: NSB PROX BNG +********************************************* + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_bng-{port_num} | +| | | +| | * context = baremetal or heat_context; | +| | * port_num = 4; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The BNG workload converts packets from QinQ to GRE tunnels, | +| | handles routing and adds/removes MPLS tags. | +| | This use case simulates a realistic and complex application. | +| | The number of users is 32K per port and the number of routes | +| | is 8K. | +| | | +| | The BNG test cases are implemented to run in baremetal | +| | and heat context an require 4 port topology to run the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The BNG test cases are listed below: | +| | | +| | * tc_prox_baremetal_bng-2.yaml | +| | * tc_prox_baremetal_bng-4.yaml | +| | * tc_prox_heat_context_bng-2.yaml | +| | * tc_prox_heat_context_bng-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | The minimum packet size for BNG test is 78 bytes. This is | +| | set in the BNG traffic profile and can be configured to use | +| | a higher packet size for the test. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | The PROX BNG test cases can be configured with different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. The test need multi-queue enabled in Glance image. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(BNG workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 78 bytes | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_bng_qos_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_bng_qos_port.rst new file mode 100644 index 000000000..5270634ac --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_bng_qos_port.rst @@ -0,0 +1,111 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +************************************************* +Yardstick Test Case Description: NSB PROX BNG_QoS +************************************************* + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_bng_qos-{port_num} | +| | | +| | * context = baremetal or heat_context; | +| | * port_num = 4; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The BNG+QoS workload converts packets from QinQ to GRE | +| | tunnels, handles routing and adds/removes MPLS tags and | +| | performs a QoS. | +| | This use case simulates a realistic and complex application. | +| | The number of users is 32K per port and the number of routes | +| | is 8K. | +| | | +| | The BNG_QoS test cases are implemented to run in baremetal | +| | and heat context an require 4 port topology to run the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The BNG_QoS test cases are listed below: | +| | | +| | * tc_prox_baremetal_bng_qos-2.yaml | +| | * tc_prox_baremetal_bng_qos-4.yaml | +| | * tc_prox_heat_context_bng_qos-2.yaml | +| | * tc_prox_heat_context_bng_qos-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | The minumum packet size for BNG_QoS test is 78 bytes. This | +| | is set in the bng_qos traffic profile and can be configured | +| | to use a higher packet size for the test. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This PROX BNG_QoS test cases can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. The test need multi-queue enabled in Glance image. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(BNG_QoS workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 78 bytes | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_buffering_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_buffering_port.rst new file mode 100644 index 000000000..0ab242cf2 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_buffering_port.rst @@ -0,0 +1,107 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +********************************************************** +Yardstick Test Case Description: NSB PROX Packet Buffering +********************************************************** + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_buffering-{port_num} | +| | | +| | * context = baremetal or heat_context | +| | * port_num = 1 | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | This test measures the impact of the condition when packets | +| | get buffered, thus they stay in memory for the extended | +| | period of time, 125ms in this case. | +| | | +| | The Packet Buffering test cases are implemented to run in | +| | baremetal and heat context. | +| | | +| | The test runs only on the first port of the SUT. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The Packet Buffering test cases are listed below: | +| | | +| | * tc_prox_baremetal_buffering-1.yaml | +| | * tc_prox_heat_context_buffering-1.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | The minimum packet size for Buffering test is 64 bytes. This | +| | is set in the traffic profile and can be configured to use | +| | a higher packet size for the test. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | The PROX Packet Buffering test cases can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. The test need multi-queue enabled in Glance image. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(Packet Buffering workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI in this test is the maximum number of packets that | +| | can be forwarded given the requirement that the latency of | +| | each packet is at least 125 millisecond. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_l2fwd_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_l2fwd_port.rst new file mode 100644 index 000000000..82cb64fe5 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_l2fwd_port.rst @@ -0,0 +1,120 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +*********************************************** +Yardstick Test Case Description: NSB PROX L2FWD +*********************************************** + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_l2fwd-{port_num} | +| | | +| | * context = baremetal or heat_context; | +| | * port_num = 2 or 4; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The PROX L2FWD test has 3 types of test cases: | +| | L2FWD: The application will take packets in from one port | +| | and forward them unmodified to another port | +| | L2FWD_Packet_Touch: The application will take packets in | +| | from one port, update src and dst MACs and forward them to | +| | another port. | +| | L2FWD_Multi_Flow: The application will take packets in | +| | from one port, update src and dst MACs and forward them to | +| | another port. This test case exercises the softswitch | +| | with 200k flows. | +| | | +| | The above test cases are implemented for baremetal and heat | +| | context for 2 port and 4 port configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The L2FWD test cases are listed below: | +| | | +| | * tc_prox_baremetal_l2fwd-2.yaml | +| | * tc_prox_baremetal_l2fwd-4.yaml | +| | * tc_prox_baremetal_l2fwd_pktTouch-2.yaml | +| | * tc_prox_baremetal_l2fwd_pktTouch-4.yaml | +| | * tc_prox_baremetal_l2fwd_multiflow-2.yaml | +| | * tc_prox_baremetal_l2fwd_multiflow-4.yaml | +| | * tc_prox_heat_context_l2fwd-2.yaml | +| | * tc_prox_heat_context_l2fwd-4.yaml | +| | * tc_prox_heat_context_l2fwd_pktTouch-2.yaml | +| | * tc_prox_heat_context_l2fwd_pktTouch-4.yaml | +| | * tc_prox_heat_context_l2fwd_multiflow-2.yaml | +| | * tc_prox_heat_context_l2fwd_multiflow-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | Packet size set as 64 bytes in traffic profile | +| | These can be configured | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | The PROX L2FWD test cases can be configured with different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(L2FWD workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 64 bytes | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_l3fwd_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_l3fwd_port.rst new file mode 100644 index 000000000..87cc10360 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_l3fwd_port.rst @@ -0,0 +1,105 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +*********************************************** +Yardstick Test Case Description: NSB PROX L3FWD +*********************************************** + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_l3fwd-{port_num} | +| | | +| | * context = baremetal or heat_context; | +| | * port_num = 2 or 4; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The PROX L3FWD application performs basic routing of packets | +| | with LPM based look-up method. | +| | | +| | The L3FWD test cases are implemented for baremetal and heat | +| | context for 2 port and 4 port configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The L3FWD test cases are listed below: | +| | | +| | * tc_prox_baremetal_l3fwd-2.yaml | +| | * tc_prox_baremetal_l3fwd-4.yaml | +| | * tc_prox_heat_context_l3fwd-2.yaml | +| | * tc_prox_heat_context_l3fwd-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | The minimum packet size for L3FWD test is 64 bytes. This is | +| | set in the traffic profile and can be configured to use | +| | a higher packet size for the test. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This PROX L3FWD test cases can be configured with different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. The test need multi-queue enabled in Glance image. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(L3FWD workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packet to the VNF. If the number of dropped | +| | packets is more than the tolerated loss the line rate | +| | or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 64 byte | +| | packets with an accepted minimal packet loss for the default | +| | configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_load_balancer_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_load_balancer_port.rst new file mode 100644 index 000000000..ada02f829 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_load_balancer_port.rst @@ -0,0 +1,109 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +******************************************************* +Yardstick Test Case Description: NSB PROX Load Balancer +******************************************************* + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_lb-{port_num} | +| | | +| | * context = baremetal or heat_context | +| | * port_num = 4 | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The applciation transmits packets on one port and revieves | +| | them on 4 ports. | +| | The conventional 5-tuple is used in this test as it requires | +| | some extraction steps and allows defining enough distinct | +| | values to find the performance limits. | +| | | +| | The load is increased (adding more ports if needed) while | +| | packets are load balanced using a hash table of 8M entries | +| | | +| | The number of packets per second that can be forwarded | +| | determines the KPI. The default packet size is 64 bytes. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The Load Balancer test cases are listed below: | +| | | +| | * tc_prox_baremetal_lb-4.yaml | +| | * tc_prox_heat_context_lb-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | Packet size set as 64 bytes in traffic profile. | +| | These can be configured | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | The PROX Load Balancer test cases can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. The test need multi-queue enabled in Glance image. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(Load Balancer workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 78 bytes | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_mpls_tagging_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_mpls_tagging_port.rst new file mode 100644 index 000000000..cef859996 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_prox_context_mpls_tagging_port.rst @@ -0,0 +1,109 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2017 Intel Corporation. + +****************************************************** +Yardstick Test Case Description: NSB PROX MPLS Tagging +****************************************************** + ++-----------------------------------------------------------------------------+ +|NSB PROX test for NFVI characterization | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_prox_{context}_mpls_tagging-{port_num} | +| | | +| | * context = baremetal or heat_context; | +| | * port_num = 2 or 4; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * Dropped packets; | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The PROX MPLS Tagging test will take packets in from one | +| | port add an MPLS tag and forward them to another port. | +| | While forwarding packets in other direction MPLS tags will | +| | be removed. | +| | | +| | The MPLS test cases are implemented to run in baremetal | +| | and heat context an require 4 port topology to run the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The MPLS Tagging test cases are listed below: | +| | | +| | * tc_prox_baremetal_mpls_tagging-2.yaml | +| | * tc_prox_baremetal_mpls_tagging-4.yaml | +| | * tc_prox_heat_context_mpls_tagging-2.yaml | +| | * tc_prox_heat_context_mpls_tagging-4.yaml | +| | | +| | Test duration is set as 300sec for each test. | +| | The minimum packet size for MPLS test is 68 bytes. This is | +| | set in the traffic profile and can be configured to use | +| | higher packet sizes. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | PROX | +| | PROX is a DPDK application that can simulate VNF workloads | +| | and can generate traffic and used for NFVI characterization | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | The PROX MPLS Tagging test cases can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * test durations; | +| | * tolerated loss; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | For Openstack test case image (yardstick-samplevnfs) needs | +|conditions | to be installed into Glance with Prox and Dpdk included in | +| | it. | +| | | +| | For Baremetal tests cases Prox and Dpdk must be installed in | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | For Baremetal test: The TG and VNF are started on the hosts | +| | based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(MPLS workload) based on the test flavor. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the TG and VNF by using ssh. | +| | The test will resolve the topology and instantiate the VNF | +| | and TG and collect the KPI's/metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | The TG will send packets to the VNF. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 68 bytes | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | In Baremetal test: The test quits the application and unbind | +| | the dpdk ports. | +| | | +| | In Heat test: Two host VMs are deleted on test completion. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++--------------+--------------------------------------------------------------+ + |