diff options
Diffstat (limited to 'docs/testing/user/userguide/nsb')
9 files changed, 1266 insertions, 1 deletions
diff --git a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst index 895837283..6c18c7d89 100644 --- a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst +++ b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst @@ -27,4 +27,12 @@ NSB PROX Test Case Descriptions tc_prox_context_buffering_port tc_prox_context_load_balancer_port tc_prox_context_vpe_port - tc_prox_context_lw_after_port + tc_prox_context_lw_aftr_port + tc_epc_default_bearer_landslide + tc_epc_dedicated_bearer_landslide + tc_epc_saegw_tput_relocation_landslide + tc_epc_network_service_request_landslide + tc_epc_ue_service_request_landslide + tc_vfw_rfc2544 + tc_vfw_rfc2544_correlated + tc_vfw_rfc3511 diff --git a/docs/testing/user/userguide/nsb/tc_epc_dedicated_bearer_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_dedicated_bearer_landslide.rst new file mode 100644 index 000000000..c8865ed93 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_epc_dedicated_bearer_landslide.rst @@ -0,0 +1,156 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2018 Intel Corporation. + +********************************************************* +Yardstick Test Case Description: NSB EPC DEDICATED BEARER +********************************************************* + ++-----------------------------------------------------------------------------+ +|NSB EPC dedicated bearer test case | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_epc_{initiator}_dedicated_bearer_landslide | +| | | +| | * initiator: dedicated bearer creation initiator side could | +| | be UE (ue) or Network (network). | +| | | ++--------------+--------------------------------------------------------------+ +|metric | All metrics provided by Spirent Landslide traffic generator | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The Spirent Landslide product provides one box solution which| +| | allows to fully emulate all EPC network nodes including | +| | mobile users, network host and generate control and data | +| | plane traffic. | +| | | +| | This test allows to check processing capability under | +| | different levels of load (number of subscriber, generated | +| | traffic throughput, etc.) for case when default and dedicated| +| | bearers are creating and using for traffic transferring. | +| | | +| | It's easy to replace emulated node or multiple nodes in test | +| | topology with real node or corresponding vEPC VNF as DUT and | +| | check it's processing capabilities under specific test case | +| | load conditions. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The EPC dedicated bearer test cases are listed below: | +| | | +| | * tc_epc_ue_dedicated_bearer_create_landslide.yaml | +| | * tc_epc_network_dedicated_bearer_create_landslide.yaml | +| | | +| | Test duration: | +| | | +| | * is set as 60sec (specified in test session profile); | +| | | +| | Traffic type: | +| | | +| | * UDP; | +| | | +| | Packet sizes: | +| | | +| | * 512 bytes; | +| | | +| | Traffic transaction rate: | +| | | +| | * 5 trans/s.; | +| | | +| | Number of mobile subscribers: | +| | | +| | * 20000; | +| | | +| | Number of default bearers per subscriber: | +| | | +| | * 1; | +| | | +| | Number of dedicated bearers per default bearer: | +| | | +| | * 1. | +| | | +| | The above fields and values are the main options used for the| +| | test case. Other configurable options could be found in test | +| | session profile yaml file. All these options have default | +| | values which can be overwritten in test case file. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Spirent Landslide | +| | | +| | The Spirent Landslide is a tool for functional and | +| | performance testing of different types of mobile networks. | +| | It emulates real-world control and data traffic of mobile | +| | subscribers moving through virtualized EPC network. | +| | Detailed description of Spirent Landslide product could be | +| | found here: https://www.spirent.com/Products/Landslide | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This EPC DEDICATED BEARER test cases can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * traffic transaction rate; | +| | * number of subscribers sessions; | +| | * number of default bearers per subscriber; | +| | * number of dedicated bearers per default; | +| | * subscribers connection rate; | +| | * subscribers disconnection rate; | +| | * dedicated bearers activation timeout; | +| | * DMF (traffic profile); | +| | * enable/disable Fireball DMF threading model that provides | +| | optimized performance; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI-NFV-TST001 | +| | | +| | 3GPP TS 32.455 | +| | | ++--------------+--------------------------------------------------------------+ +| pre-test | * All Spirent Landslide dependencies need to be installed. | +| conditions | The steps are described in NSB installation chapter for the| +| | Spirent Landslide vEPC tests; | +| | | +| | * The pod.yaml file contains all necessary information (TAS | +| | VM IP address, NICs, emulated SUTs and Test Nodes | +| | parameters (names, types, ip addresses, etc.). | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | Spirent Landslide components are running on the hosts | +| | specified in the pod file. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with Spirent Landslide Test | +| | Administrator Server (TAS) by TCL and REST API. The test | +| | will resolve the topology and instantiate all emulated EPC | +| | network nodes. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Test scenarios run, which performs the following steps: | +| | | +| | * Start the emulated EPC network nodes; | +| | * Establish the subscribers connections to EPC network | +| | (default bearers); | +| | * Establish the number of dedicated bearers as per per | +| | default bearer for each subscriber; | +| | * Create the sessions and transmit traffic through EPC | +| | network nodes during the specified traffic duration time; | +| | * Disconnect dedicated bearers; | +| | * Disconnect subscribers at the end of the test. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | During test run, all the metrics provided by Spirent | +| | Landslide are stored in the yardstick dispatcher. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will create the test session in Spirent | +| | Landslide with the test case parameters and store the results| +| | in the database for benchmarking purposes. The aim is only | +| | to collect all the metrics that are provided by Spirent | +| | Landslide product for each test specific scenario. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/nsb/tc_epc_default_bearer_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_default_bearer_landslide.rst new file mode 100644 index 000000000..9e6d77825 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_epc_default_bearer_landslide.rst @@ -0,0 +1,149 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2018 Intel Corporation. + +******************************************************* +Yardstick Test Case Description: NSB EPC DEFAULT BEARER +******************************************************* + ++-----------------------------------------------------------------------------+ +|NSB EPC default bearer test case | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_epc_default_bearer_landslide_{dmf_setup} | +| | | +| | * dmf_setup: single or multi dmf test session setup; | +| | | ++--------------+--------------------------------------------------------------+ +|metric | All metrics provided by Spirent Landslide traffic generator | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The Spirent Landslide product provides one box solution which| +| | allows to fully emulate all EPC network nodes including | +| | mobile users, network host and generate control and data | +| | plane traffic. | +| | | +| | This test allows to check processing capability of EPC under | +| | different levels of load (number of subscriber, generated | +| | traffic throughput) for case when only one default bearer is | +| | using for transferring traffic from UE to Network. | +| | | +| | It's easy to replace emulated node or multiple nodes in test | +| | topology with real node or corresponding vEPC VNF as DUT and | +| | check it's processing capabilities under specific test case | +| | load conditions. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The EPC default bearer test cases are listed below: | +| | | +| | * tc_epc_default_bearer_create_landslide.yaml | +| | * tc_epc_default_bearer_create_landslide_multi_dmf.yaml | +| | | +| | Test duration: | +| | | +| | * is set as 60sec (specified in test session profile); | +| | | +| | Traffic type: | +| | | +| | * UDP - for single DMF test case; | +| | * UDP and TCP - for multi DMF test case; | +| | | +| | Packet sizes: | +| | | +| | * 512 bytes for UDP packets; | +| | * 1518 bytes for TCP packets; | +| | | +| | Traffic transaction rate: | +| | | +| | * 5 trans/s.; | +| | | +| | Number of mobile subscribers: | +| | | +| | * 20000; | +| | | +| | Number of default bearers per subscriber: | +| | | +| | * 1. | +| | | +| | The above fields and values are the main options used for the| +| | test case. Other configurable options could be found in test | +| | session profile yaml file. All these options have default | +| | values which can be overwritten in test case file. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Spirent Landslide | +| | | +| | The Spirent Landslide is a tool for functional & performance | +| | testing of different types of mobile networks. It emulates | +| | real-world control and data traffic of mobile subscribers | +| | moving through virtualized EPC network. | +| | Detailed description of Spirent Landslide product could be | +| | found here: https://www.spirent.com/Products/Landslide | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This EPC DEFAULT BEARER test cases can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * traffic transaction rate; | +| | * number of subscribers sessions; | +| | * number of default bearers per subscriber; | +| | * subscribers connection rate; | +| | * subscribers disconnection rate; | +| | * DMF (traffic profile); | +| | * enable/disable Fireball DMF threading model that provides | +| | optimized performance; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI-NFV-TST001 | +| | | +| | 3GPP TS 32.455 | +| | | ++--------------+--------------------------------------------------------------+ +| pre-test | * All Spirent Landslide dependencies are installed (detailed | +| conditions | installation steps are described in Chapter 13- | +| | nsb-installation.rst and 14-nsb-operation.rst file for NSB | +| | Spirent Landslide vEPC tests; | +| | | +| | * The pod.yaml file contains all necessary information | +| | (TAS VM IP address, NICs, emulated SUTs and Test Nodes | +| | parameters (names, types, ip addresses, etc.). | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | Spirent Landslide components are running on the hosts | +| | specified in the pod file. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with Spirent Landslide Test | +| | Administration Server (TAS) by TCL and REST API. The test | +| | will resolve the topology and instantiate all emulated EPC | +| | network nodes. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Test scenarios run, which performs the following steps: | +| | | +| | * Start emulated EPC network nodes; | +| | * Establish subscribers connections to EPC network (only | +| | default bearers are established); | +| | * Create the sessions and transmit traffic through EPC | +| | network nodes during the specified traffic duration time; | +| | * Disconnect subscribers at the end of the test. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | During test run, all the metrics provided by Spirent | +| | Landslide are stored in the yardstick dispatcher. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will create the test session in Spirent | +| | Landslide with the test case parameters and store the | +| | results in the database for benchmarking purposes. The aim | +| | is only to collect all the metrics that are provided by | +| | Spirent Landslide product for each test specific scenario. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/nsb/tc_epc_network_service_request_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_network_service_request_landslide.rst new file mode 100644 index 000000000..85e6ce11a --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_epc_network_service_request_landslide.rst @@ -0,0 +1,159 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2018 Intel Corporation. + +**************************************************************** +Yardstick Test Case Description: NSB EPC NETWORK SERVICE REQUEST +**************************************************************** + ++-----------------------------------------------------------------------------+ +|NSB EPC network service request test case | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_epc_network_service_request_landslide | +| | | +| | * initiator: service request initiator side could be UE (ue) | +| | or Network (network). | +| | | ++--------------+--------------------------------------------------------------+ +|metric | All metrics provided by Spirent Landslide traffic generator | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The Spirent Landslide product provides one box solution which| +| | allows to fully emulate all EPC network nodes including | +| | mobile users, network host and generate control and data | +| | plane traffic. | +| | | +| | This test covers case of network initiated service request & | +| | allows to check processing capabilities of EPC handling high | +| | amount of continuous Downlink Data Notification messages from| +| | network to UEs which are in Idle state. | +| | | +| | It's easy to replace emulated node or multiple nodes in test | +| | topology with real node or corresponding vEPC VNF as DUT and | +| | check it's processing capabilities under specific test case | +| | load conditions. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The EPC network service request test cases are listed below: | +| | | +| | * tc_epc_network_service_request_landslide.yaml | +| | | +| | Test duration: | +| | | +| | * is set as 60sec (specified in test session profile); | +| | | +| | Traffic type: | +| | | +| | * UDP; | +| | | +| | Packet sizes: | +| | | +| | * 512 bytes; | +| | | +| | Traffic transaction rate: | +| | | +| | * 0.1 trans/s.; | +| | | +| | Number of mobile subscribers: | +| | | +| | * 20000; | +| | | +| | Number of default bearers per subscriber: | +| | | +| | * 1; | +| | | +| | Idle entry time (timeout after which UE goes to Idle state): | +| | | +| | * 5s; | +| | | +| | Traffic start delay: | +| | | +| | * 1000ms. | +| | | +| | The above fields and values are the main options used for the| +| | test case. Other configurable options could be found in test | +| | session profile yaml file. All these options have default | +| | values which can be overwritten in test case file. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Spirent Landslide | +| | | +| | The Spirent Landslide is a tool for functional & performance | +| | testing of different types of mobile networks. It emulates | +| | real-world control and data traffic of mobile subscribers | +| | moving through virtualized EPC network. | +| | Detailed description of Spirent Landslide product could be | +| | found here: https://www.spirent.com/Products/Landslide | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This EPC NETWORK SERVICE REQUEST test case can be configured | +| | with different: | +| | | +| | * packet sizes; | +| | * traffic transaction rate; | +| | * number of subscribers sessions; | +| | * number of default bearers per subscriber; | +| | * subscribers connection rate; | +| | * subscribers disconnection rate; | +| | * timeout after which UE goes to Idle state; | +| | * Traffic start delay; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI-NFV-TST001 | +| | | +| | 3GPP TS 32.455 | +| | | ++--------------+--------------------------------------------------------------+ +| pre-test | * All Spirent Landslide dependencies are installed (detailed | +| conditions | installation steps are described in Chapter 13- | +| | nsb-installation.rst and 14-nsb-operation.rst file for NSB | +| | Spirent Landslide vEPC tests; | +| | | +| | * The pod.yaml file contains all necessary information | +| | (TAS VM IP address, NICs, emulated SUTs and Test Nodes | +| | parameters (names, types, ip addresses, etc.). | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | Spirent Landslide components are running on the hosts | +| | specified in the pod file. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with Spirent Landslide Test | +| | Administration Server (TAS) by TCL and REST API. The test | +| | will resolve the topology and instantiate all emulated EPC | +| | network nodes. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Test scenarios run, which performs the following steps: | +| | | +| | * Start emulated EPC network nodes; | +| | * Establish subscribers connections to EPC network (default | +| | bearers); | +| | * Switch UE to Idle state after specified in test case | +| | timeout; | +| | * Send Downlink Data Notification from network to UE, that | +| | will return UE to active state. This process is continuous | +| | and during whole test run UEs will be going to Idle state | +| | and will be switched back to active state after Downlink | +| | Data Notification was received; | +| | * Disconnect subscribers at the end of the test. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | During test run, all the metrics provided by Spirent | +| | Landslide are stored in the yardstick dispatcher. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will create the test session in Spirent | +| | Landslide with the test case parameters and store the | +| | results in the database for benchmarking purposes. The aim | +| | is only to collect all the metrics that are provided by | +| | Spirent Landslide product for each test specific scenario. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/nsb/tc_epc_saegw_tput_relocation_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_saegw_tput_relocation_landslide.rst new file mode 100644 index 000000000..102517562 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_epc_saegw_tput_relocation_landslide.rst @@ -0,0 +1,167 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2018 Intel Corporation. + +********************************************************* +Yardstick Test Case Description: NSB EPC SAEGW RELOCATION +********************************************************* + ++-----------------------------------------------------------------------------+ +|NSB EPC SAEGW throughput with relocation test case | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_epc_saegw_tput_relocation_landslide | +| | | +| | | ++--------------+--------------------------------------------------------------+ +|metric | All metrics provided by Spirent Landslide traffic generator | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The Spirent Landslide product provides one box solution which| +| | allows to fully emulate all EPC network nodes including | +| | mobile users, network host and generate control and data | +| | plane traffic. | +| | | +| | This test allows to check processing capability of EPC | +| | handling large amount of subscribers X2 handovers between | +| | different eNBs while UEs are sending traffic. | +| | | +| | It's easy to replace emulated node or multiple nodes in test | +| | topology with real node or corresponding vEPC VNF as DUT and | +| | check it's processing capabilities under specific test case | +| | load conditions. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The EPC SAEGW throughput with relocation tests are listed | +| | below: | +| | | +| | * tc_epc_saegw_tput_relocation_landslide.yaml | +| | | +| | Test duration: | +| | | +| | * is set as 60sec (specified in test session profile); | +| | | +| | Traffic type: | +| | | +| | * UDP; | +| | | +| | Packet sizes: | +| | | +| | * 512 bytes; | +| | | +| | Traffic transaction rate: | +| | | +| | * 5 trans/s.; | +| | | +| | Number of mobile subscribers: | +| | | +| | * 20000; | +| | | +| | Number of default bearers per subscriber: | +| | | +| | * 1; | +| | | +| | Handover type: | +| | | +| | * X2 handover; | +| | | +| | Mobility time (timeout after sessions were established after | +| | which handover will start): | +| | | +| | * 10000ms; | +| | | +| | Handover start type: | +| | | +| | * When all sessions started; | +| | | +| | Mobility mode: | +| | | +| | * Single handoff; | +| | | +| | Mobility Rate: | +| | | +| | * 120 subscribers/s. | +| | | +| | The above fields and values are the main options used for the| +| | test case. Other configurable options could be found in test | +| | session profile yaml file. All these options have default | +| | values which can be overwritten in test case file. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Spirent Landslide | +| | | +| | The Spirent Landslide is a tool for functional & performance | +| | testing of different types of mobile networks. It emulates | +| | real-world control and data traffic of mobile subscribers | +| | moving through virtualized EPC network. | +| | Detailed description of Spirent Landslide product could be | +| | found here: https://www.spirent.com/Products/Landslide | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This EPC UE SERVICE REQUEST test cases can be configured with| +| | different: | +| | | +| | * packet sizes; | +| | * traffic transaction rate; | +| | * number of subscribers sessions; | +| | * handover type; | +| | * mobility rate; | +| | * mobility time; | +| | * mobility mode; | +| | * handover start condition; | +| | * subscribers disconnection rate; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI-NFV-TST001 | +| | | +| | 3GPP TS 32.455 | +| | | ++--------------+--------------------------------------------------------------+ +| pre-test | * All Spirent Landslide dependencies are installed (detailed | +| conditions | installation steps are described in Chapter 13- | +| | nsb-installation.rst and 14-nsb-operation.rst file for NSB | +| | Spirent Landslide vEPC tests; | +| | | +| | * The pod.yaml file contains all necessary information | +| | (TAS VM IP address, NICs, emulated SUTs and Test Nodes | +| | parameters (names, types, ip addresses, etc.). | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | Spirent Landslide components are running on the hosts | +| | specified in the pod file. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with Spirent Landslide Test | +| | Administration Server (TAS) by TCL and REST API. The test | +| | will resolve the topology and instantiate all emulated EPC | +| | network nodes. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Test scenarios run, which performs the following steps: | +| | | +| | * Start emulated EPC network nodes; | +| | * Establish subscribers connections to EPC network (default | +| | bearers); | +| | * Start run traffic; | +| | * After specified in test case mobility timeout, start | +| | handover process on specified mobility rate; | +| | * Disconnect subscribers at the end of the test. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | During test run, all the metrics provided by Spirent | +| | Landslide are stored in the yardstick dispatcher. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will create the test session in Spirent | +| | Landslide with the test case parameters and store the | +| | results in the database for benchmarking purposes. The aim | +| | is only to collect all the metrics that are provided by | +| | Spirent Landslide product for each test specific scenario. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/nsb/tc_epc_ue_service_request_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_ue_service_request_landslide.rst new file mode 100644 index 000000000..0711a0ce3 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_epc_ue_service_request_landslide.rst @@ -0,0 +1,174 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2018 Intel Corporation. + +*********************************************************** +Yardstick Test Case Description: NSB EPC UE SERVICE REQUEST +*********************************************************** + ++-----------------------------------------------------------------------------+ +|NSB EPC UE service request test case | +| | ++--------------+--------------------------------------------------------------+ +|test case id | tc_epc_{initiator}_service_request_landslide | +| | | +| | * initiator: service request initiator side could be UE (ue) | +| | or Network (nw). | +| | | ++--------------+--------------------------------------------------------------+ +|metric | All metrics provided by Spirent Landslide traffic generator | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The Spirent Landslide product provides one box solution which| +| | allows to fully emulate all EPC network nodes including | +| | mobile users, network host and generate control and data | +| | plane traffic. | +| | | +| | This test allows to check processing capabilities of EPC | +| | under high user connections rate and traffic load for case | +| | when UEs initiates service request (UE initiates bearer | +| | modification request to provide dedicated bearer for new | +| | type of traffic) | +| | | +| | It's easy to replace emulated node or multiple nodes in test | +| | topology with real node or corresponding vEPC VNF as DUT and | +| | check it's processing capabilities under specific test case | +| | load conditions. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | The EPC ue service request test cases are listed below: | +| | | +| | * tc_epc_ue_service_request_landslide.yaml | +| | | +| | Test duration: | +| | | +| | * is set as 60sec (specified in test session profile); | +| | | +| | Traffic type: | +| | | +| | * UDP; | +| | | +| | Packet sizes: | +| | | +| | * 512 bytes; | +| | | +| | Traffic transaction rate: | +| | | +| | * 5 trans/s.; | +| | | +| | Number of mobile subscribers: | +| | | +| | * 20000; | +| | | +| | Number of default bearers per subscriber: | +| | | +| | * 1; | +| | | +| | Number of dedicated bearers per default bearer: | +| | | +| | * 1. | +| | | +| | TFT settings for dedicated bearers: | +| | | +| | * TFT configured to filter TCP traffic (Protocol ID 6) | +| | | +| | Modified TFT settings: | +| | | +| | * Create new TFT to filter UDP traffic (Protocol ID 17) from | +| | 2002 local port and 2003 remote port; | +| | | +| | Modified QoS settings: | +| | | +| | * Set QCI 5 for dedicated bearers; | +| | | +| | The above fields and values are the main options used for the| +| | test case. Other configurable options could be found in test | +| | session profile yaml file. All these options have default | +| | values which can be overwritten in test case file. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Spirent Landslide | +| | | +| | The Spirent Landslide is a tool for functional & performance | +| | testing of different types of mobile networks. It emulates | +| | real-world control and data traffic of mobile subscribers | +| | moving through virtualized EPC network. | +| | Detailed description of Spirent Landslide product could be | +| | found here: https://www.spirent.com/Products/Landslide | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This EPC UE SERVICE REQUEST test case can be configured with | +| | different: | +| | | +| | * packet sizes; | +| | * traffic transaction rate; | +| | * number of subscribers sessions; | +| | * number of default bearers per subscriber; | +| | * number of dedicated bearers per default; | +| | * subscribers connection rate; | +| | * subscribers disconnection rate; | +| | * dedicated bearers activation timeout; | +| | * DMF (traffic profile); | +| | * enable/disable Fireball DMF threading model that provides | +| | optimized performance; | +| | * Starting TFT settings for dedicated bearers; | +| | * Modified TFT settings for dedicated bearers; | +| | * Modified QoS settings for dedicated bearers; | +| | | +| | Default values exist. | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI-NFV-TST001 | +| | | +| | 3GPP TS 32.455 | +| | | ++--------------+--------------------------------------------------------------+ +| pre-test | * All Spirent Landslide dependencies are installed (detailed | +| conditions | installation steps are described in Chapter 13- | +| | nsb-installation.rst and 14-nsb-operation.rst file for NSB | +| | Spirent Landslide vEPC tests; | +| | | +| | * The pod.yaml file contains all necessary information | +| | (TAS VM IP address, NICs, emulated SUTs and Test Nodes | +| | parameters (names, types, ip addresses, etc.). | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | Spirent Landslide components are running on the hosts | +| | specified in the pod file. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with Spirent Landslide Test | +| | Administration Server (TAS) by TCL and REST API. The test | +| | will resolve the topology and instantiate all emulated EPC | +| | network nodes. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Test scenarios run, which performs the following steps: | +| | | +| | * Start emulated EPC network nodes; | +| | * Establish subscribers connections to EPC network (default | +| | bearers); | +| | * Establish the number of dedicated bearer as specified in | +| | the test case as per default bearer for each subscriber; | +| | * start run users traffic through EPC network nodes; | +| | * During traffic is running, send bearer modification request| +| | after specified in test case timeout; | +| | * Disconnect dedicated bearers; | +| | * Disconnect subscribers at the end of the test. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | During test run, all the metrics provided by Spirent | +| | Landslide are stored in the yardstick dispatcher. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | The test case will create the test session in Spirent | +| | Landslide with the test case parameters and store the | +| | results in the database for benchmarking purposes. The aim | +| | is only to collect all the metrics that are provided by | +| | Spirent Landslide product for each test specific scenario. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/nsb/tc_vfw_rfc2544.rst b/docs/testing/user/userguide/nsb/tc_vfw_rfc2544.rst new file mode 100644 index 000000000..139990bc3 --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_vfw_rfc2544.rst @@ -0,0 +1,189 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2018 Intel Corporation. + +************************************************ +Yardstick Test Case Description: NSB vFW RFC2544 +************************************************ + ++------------------------------------------------------------------------------+ +| NSB vFW test for VNF characterization | +| | ++---------------+--------------------------------------------------------------+ +| test case id | tc_{context}_rfc2544_ipv4_1rule_1flow_{pkt_size}_{tg_type} | +| | | +| | * context = baremetal, heat, heat_external, ovs, sriov | +| | heat_sriov_external contexts; | +| | * tg_type = ixia (context != heat,heat_sriov_external), | +| | trex; | +| | * pkt_size = 64B - all contexts; | +| | 128B, 256B, 512B, 1024B, 1280B, 1518B - | +| | (context = heat, tg_type = ixia) | +| | | ++---------------+--------------------------------------------------------------+ +| metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * TG Latency; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * VNF Packets Fwd; | +| | * Dropped packets; | +| | | ++---------------+--------------------------------------------------------------+ +| test purpose | The VFW RFC2544 tests measure performance characteristics of | +| | the SUT (multiple ports) and sends UDP bidirectional traffic | +| | from all TG ports to SampleVNF vFW application. The | +| | application forwards received traffic based on rules | +| | provided by the user in the TC configuration and default | +| | rules created by vFW to send traffic from uplink ports to | +| | downlink and voice versa. | +| | | ++---------------+--------------------------------------------------------------+ +| configuration | The 2 ports RFC2544 test cases are listed below: | +| | | +| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml | +| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex.yaml | +| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1024B_ixia.yaml | +| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1280B_ixia.yaml | +| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_128B_ixia.yaml | +| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1518B_ixia.yaml | +| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_256B_ixia.yaml | +| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_512B_ixia.yaml | +| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml | +| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml | +| | * tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex. | +| | yaml | +| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml | +| | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml | +| | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml | +| | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml | +| | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml | +| | | +| | The 4 ports RFC2544 test cases are listed below: | +| | | +| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia_4port.yaml | +| | * tc_tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_4port. | +| | yaml | +| | * tc_tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex_4 | +| | port.yaml | +| | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_4port.yaml | +| | | +| | The scale-up RFC2544 test cases are listed below: | +| | | +| | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml | +| | | +| | The scale-out RFC2544 test cases are listed below: | +| | | +| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale_out.yaml | +| | | +| | Test duration is set as 30 sec for each test and default | +| | number of rules are applied. These can be configured | +| | | ++---------------+--------------------------------------------------------------+ +| test tool | The vFW is a DPDK application that performs basic filtering | +| | for malformed packets and dynamic packet filtering of | +| | incoming packets using the connection tracker library. | +| | | ++---------------+--------------------------------------------------------------+ +| applicability | The vFW RFC2544 test cases can be configured with different: | +| | | +| | * packet sizes; | +| | * test duration; | +| | * tolerated loss; | +| | * traffic flows; | +| | * rules; | +| | | +| | Default values exist. | +| | | ++---------------+--------------------------------------------------------------+ +| pre-test | For OpenStack test case image (yardstick-samplevnf) needs | +| conditions | to be installed into Glance with vFW and DPDK included in | +| | it (NSB install). | +| | | +| | For Baremetal tests cases vFW and DPDK must be installed on | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information. | +| | | +| | For standalone (SA) SRIOV/OvS test cases the | +| | yardstick-samplevnf image needs to be installed on hosts and | +| | pod.yaml file must be provided with necessary system, NIC | +| | information. | +| | | ++---------------+--------------------------------------------------------------+ +| test sequence | Description and expected result | +| | | ++---------------+--------------------------------------------------------------+ +| step 1 | For Baremetal test: The TG (except IXIA) and VNF are started | +| | on the hosts based on the pod file. | +| | | +| | For Heat test: Two host VMs are booted, as Traffic generator | +| | and VNF(vFW) based on the test flavor. In case of scale-out | +| | scenario the multiple VNF VMs will be started. | +| | | +| | For Heat external test: vFW VM is booted and TG (except IXIA)| +| | generator is started on the external host based on the pod | +| | file. In case of scale-out scenario the multiple VNF VMs | +| | will be deployed. | +| | | +| | For Heat SRIOV external test: vFW VM is booted with network | +| | interfaces of `direct` type which are mapped to VFs that are | +| | available to OpenStack. TG (except IXIA) is started on the | +| | external host based on the pod file. In case of scale-out | +| | scenario the multiple VNF VMs will be deployed. | +| | | +| | For SRIOV test: VF ports are created on host's PFs specified | +| | in the TC file and VM is booed using those ports and image | +| | provided in the configuration. TG (except IXIA) is started | +| | on other host connected to VNF machine based on the pod | +| | file. The vFW is started in the booted VM. In case of | +| | scale-out scenario the multiple VNF VMs will be created. | +| | | +| | For OvS-DPDK test: OvS DPDK switch is started and bridges | +| | are created with ports specified in the TC file. DPDK vHost | +| | ports are added to corresponding bridge and VM is booed | +| | using those ports and image provided in the configuration. | +| | TG (except IXIA) is started on other host connected to VNF | +| | machine based on the pod file. The vFW is started in the | +| | booted VM. In case of scale-out scenario the multiple VNF | +| | VMs will be deployed. | +| | | ++---------------+--------------------------------------------------------------+ +| step 2 | Yardstick is connected with the TG and VNF by using ssh (in | +| | case of IXIA TG is connected via TCL interface). The test | +| | will resolve the topology and instantiate all VNFs | +| | and TG and collect the KPI's/metrics. | +| | | ++---------------+--------------------------------------------------------------+ +| step 3 | The TG will send packets to the VNFs. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for different | +| | packet size with an accepted minimal packet loss for the | +| | default configuration. | +| | | ++---------------+--------------------------------------------------------------+ +| step 4 | In Baremetal test: The test quits the application and unbind | +| | the DPDK ports. | +| | | +| | In Heat test: All VNF VMs and TG are deleted on test | +| | completion. | +| | | +| | In SRIOV test: The deployed VM with vFW is destroyed on the | +| | host and TG (exclude IXIA) is stopped. | +| | | +| | In Heat SRIOV test: The deployed VM with vFW is destroyed, | +| | VFs are released and TG (exclude IXIA) is stopped. | +| | | +| | In OvS test: The deployed VM with vFW is destroyed on the | +| | host and OvS DPDK switch is stopped and ports are unbinded. | +| | The TG (exclude IXIA) is stopped. | +| | | ++---------------+--------------------------------------------------------------+ +| test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++---------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_vfw_rfc2544_correlated.rst b/docs/testing/user/userguide/nsb/tc_vfw_rfc2544_correlated.rst new file mode 100644 index 000000000..de490900d --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_vfw_rfc2544_correlated.rst @@ -0,0 +1,130 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2018 Intel Corporation. + +************************************************************* +Yardstick Test Case Description: NSB vFW RFC2544 (correlated) +************************************************************* + ++------------------------------------------------------------------------------+ +| NSB vFW test for VNF characterization using correlated traffic | +| | ++---------------+--------------------------------------------------------------+ +| test case id | tc_{context}_rfc2544_ipv4_1rule_1flow_64B_trex_corelated | +| | | +| | * context = baremetal, heat | +| | | ++---------------+--------------------------------------------------------------+ +| metric | * Network Throughput; | +| | * TG Packets Out; | +| | * TG Packets In; | +| | * TG Latency; | +| | * VNF Packets Out; | +| | * VNF Packets In; | +| | * VNF Packets Fwd; | +| | * Dropped packets; | +| | | +| | NOTE: For correlated TCs the TG metrics are available on | +| | uplink ports. | +| | | ++---------------+--------------------------------------------------------------+ +| test purpose | The VFW RFC2544 correlated tests measure performance | +| | characteristics of the SUT (multiple ports) and sends UDP | +| | traffic from uplink TG ports to SampleVNF vFW application. | +| | The application forwards received traffic from uplink ports | +| | to downlink ports based on rules provided by the user in the | +| | TC configuration and default rules created by vFW. The VNF | +| | downlink traffic is received by another UDPReplay VNF and it | +| | is mirrored back to the VNF on the same port. Finally, the | +| | traffic is received back to the TG uplink port. | +| | | ++---------------+--------------------------------------------------------------+ +| configuration | The 2 ports RFC2544 correlated test cases are listed below: | +| | | +| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_corelated | +| | _traffic.yaml | +| | | +| | Multiple VNF (2, 4, 10) RFC2544 correlated test cases are | +| | listed below: | +| | | +| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated | +| | _scale_10.yaml | +| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale | +| | _2.yaml | +| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale | +| | _4.yaml | +| | | +| | The scale-out RFC2544 test cases are listed below: | +| | | +| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale | +| | _out.yaml | +| | | +| | Test duration is set as 30 sec for each test and default | +| | number of rules are applied. These can be configured | +| | | ++---------------+--------------------------------------------------------------+ +| test tool | The vFW is a DPDK application that performs basic filtering | +| | for malformed packets and dynamic packet filtering of | +| | incoming packets using the connection tracker library. | +| | | ++---------------+--------------------------------------------------------------+ +| applicability | The vFW RFC2544 test cases can be configured with different: | +| | | +| | * packet sizes; | +| | * test duration; | +| | * tolerated loss; | +| | * traffic flows; | +| | * rules; | +| | | +| | Default values exist. | +| | | ++---------------+--------------------------------------------------------------+ +| pre-test | For OpenStack test case image (yardstick-samplevnf) needs | +| conditions | to be installed into Glance with vFW and DPDK included in | +| | it (NSB install). | +| | | +| | For Baremetal tests cases vFW and DPDK must be installed on | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information. | +| | | ++---------------+--------------------------------------------------------------+ +| test sequence | Description and expected result | +| | | ++---------------+--------------------------------------------------------------+ +| step 1 | For Baremetal test: The TG (except IXIA), vFW and UDPReplay | +| | VNFs are started on the hosts based on the pod file. | +| | | +| | For Heat test: Three host VMs are booted, as Traffic | +| | generator, vFW and UDPReplay VNF(vFW) based on the test | +| | flavor. In case of scale-out scenario the multiple vFW VNF | +| | VMs will be started. | +| | | ++---------------+--------------------------------------------------------------+ +| step 2 | Yardstick is connected with the TG, vFW and UDPReplay VNF by | +| | using ssh (in case of IXIA TG is connected via TCL | +| | interface). The test will resolve the topology and | +| | instantiate all VNFs and TG and collect the KPI's/metrics. | +| | | ++---------------+--------------------------------------------------------------+ +| step 3 | The TG will send packets to the VNFs. If the number of | +| | dropped packets is more than the tolerated loss the line | +| | rate or throughput is halved. This is done until the dropped | +| | packets are within an acceptable tolerated loss. | +| | | +| | The KPI is the number of packets per second for 64B packet | +| | size with an accepted minimal packet loss for the default | +| | configuration. | +| | | ++---------------+--------------------------------------------------------------+ +| step 4 | In Baremetal test: The test quits the application and unbind | +| | the DPDK ports. | +| | | +| | In Heat test: All VNF VMs and TG are deleted on test | +| | completion. | +| | | ++---------------+--------------------------------------------------------------+ +| test verdict | The test case will achieve a Throughput with an accepted | +| | minimal tolerated packet loss. | ++---------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/nsb/tc_vfw_rfc3511.rst b/docs/testing/user/userguide/nsb/tc_vfw_rfc3511.rst new file mode 100644 index 000000000..9051fc4df --- /dev/null +++ b/docs/testing/user/userguide/nsb/tc_vfw_rfc3511.rst @@ -0,0 +1,133 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2018 Intel Corporation. + +******************************************************* +Yardstick Test Case Description: NSB vFW RFC3511 (HTTP) +******************************************************* + ++------------------------------------------------------------------------------+ +| NSB vFW test for VNF characterization based on RFC3511 and IXIA | +| | ++---------------+--------------------------------------------------------------+ +| test case id | tc_{context}_http_ixload_{http_size}_Requests-65000_{type} | +| | | +| | * context = baremetal, heat_external | +| | * http_size = 1b, 4k, 64k, 256k, 512k, 1024k payload size | +| | * type = Concurrency, Connections, Throughput | +| | | ++---------------+--------------------------------------------------------------+ +| metric | * HTTP Total Throughput (Kbps); | +| | * HTTP Simulated Users; | +| | * HTTP Concurrent Connections; | +| | * HTTP Connection Rate; | +| | * HTTP Transaction Rate | +| | | ++---------------+--------------------------------------------------------------+ +| test purpose | The vFW RFC3511 tests measure performance characteristics of | +| | the SUT by sending the HTTP traffic from uplink to downlink | +| | TG ports through vFW VNF. The application forwards received | +| | traffic based on rules provided by the user in the TC | +| | configuration and default rules created by vFW to send | +| | traffic from uplink ports to downlink and voice versa. | +| | | ++---------------+--------------------------------------------------------------+ +| configuration | The 2 ports RFC3511 test cases are listed below: | +| | | +| | * tc_baremetal_http_ixload_1024k_Requests-65000 | +| | _Concurrency.yaml | +| | * tc_baremetal_http_ixload_1b_Requests-65000 | +| | _Concurrency.yaml | +| | * tc_baremetal_http_ixload_256k_Requests-65000 | +| | _Concurrency.yaml | +| | * tc_baremetal_http_ixload_4k_Requests-65000 | +| | _Concurrency.yaml | +| | * tc_baremetal_http_ixload_512k_Requests-65000 | +| | _Concurrency.yaml | +| | * tc_baremetal_http_ixload_64k_Requests-65000 | +| | _Concurrency.yaml | +| | * tc_heat_external_http_ixload_1b_Requests-10Gbps | +| | _Throughput.yaml | +| | * tc_heat_external_http_ixload_1b_Requests-65000 | +| | _Concurrency.yaml | +| | * tc_heat_external_http_ixload_1b_Requests-65000 | +| | _Connections.yaml | +| | | +| | The 4 ports RFC3511 test cases are listed below: | +| | | +| | * tc_baremetal_http_ixload_1b_Requests-65000 | +| | _Concurrency_4port.yaml | +| | | ++---------------+--------------------------------------------------------------+ +| test tool | The vFW is a DPDK application that performs basic filtering | +| | for malformed packets and dynamic packet filtering of | +| | incoming packets using the connection tracker library. | +| | | ++---------------+--------------------------------------------------------------+ +| applicability | The vFW RFC3511 test cases can be configured with different: | +| | | +| | * http payload sizes; | +| | * traffic flows; | +| | * rules; | +| | | +| | Default values exist. | +| | | ++---------------+--------------------------------------------------------------+ +| pre-test | For OpenStack test case image (yardstick-samplevnf) needs | +| conditions | to be installed into Glance with vFW and DPDK included in | +| | it (NSB install). | +| | | +| | For Baremetal tests cases vFW and DPDK must be installed on | +| | the hosts where the test is executed. The pod.yaml file must | +| | have the necessary system and NIC information. | +| | | ++---------------+--------------------------------------------------------------+ +| test sequence | Description and expected result | +| | | ++---------------+--------------------------------------------------------------+ +| step 1 | For Baremetal test: The vFW VNF is started on the hosts | +| | based on the pod file. | +| | | +| | For Heat external test: The vFW VM are deployed and booted. | +| | | ++---------------+--------------------------------------------------------------+ +| step 2 | Yardstick is connected with the TG (IxLoad) via IxLoad API | +| | and VNF by using ssh. The test will resolve the topology and | +| | instantiate all VNFs and TG and collect the KPI's/metrics. | +| | | ++---------------+--------------------------------------------------------------+ +| step 3 | The TG simulates HTTP traffic based on selected type of TC. | +| | | +| | Concurrency: | +| | The TC attempts to simulate some number of human users. | +| | The simulated users are gradually brought online until 64K | +| | users is met (the Ramp-Up phase), then taken offline (the | +| | Ramp Down phase). | +| | | +| | Connections: | +| | The TC creates some number of HTTP connections per second. | +| | It will attempt to generate the 64K of HTTP connections | +| | per second. | +| | | +| | Throughput: | +| | TC simultaneously transmits and receives TCP payload | +| | (bytes) at a certain rate measured in Megabits per second | +| | (Mbps), Kilobits per second (Kbps), or Gigabits per | +| | second. The 10 Gbits is default throughput. | +| | | +| | At the end of the TC, the KPIs are collected and stored | +| | (depends on the selected dispatcher). | +| | | ++---------------+--------------------------------------------------------------+ +| step 4 | In Baremetal test: The test quits the application and | +| | unbinds the DPDK ports. | +| | | +| | In Heat test: All VNF VMs are deleted and connections to TG | +| | are terminated. | +| | | ++---------------+--------------------------------------------------------------+ +| test verdict | The test case will try to achieve the configured HTTP | +| | Concurrency/Throughput/Connections. | ++---------------+--------------------------------------------------------------+ + |