diff options
Diffstat (limited to 'docs/testing/user')
10 files changed, 400 insertions, 19 deletions
diff --git a/docs/testing/user/userguide/overview.rst b/docs/testing/user/userguide/overview.rst index 5d76c76c..725a4b9d 100644 --- a/docs/testing/user/userguide/overview.rst +++ b/docs/testing/user/userguide/overview.rst @@ -55,20 +55,32 @@ Integration Description +-------------+----------------------+----------------------+ | Euphrates | Any | POSCA | +-------------+----------------------+----------------------+ +| Fraser | Any | POSCA | ++-------------+----------------------+----------------------+ Test suite & Test case Description ================================== -+--------+-------------------------------+ -|POSCA | posca_factor_ping | -| +-------------------------------+ -| | posca_factor_system_bandwidth | -| +-------------------------------+ -| | posca_facotor_througputs | -| +-------------------------------+ -| | posca_feature_scaleup | -| +-------------------------------+ -| | posca_feature_scaleout | -+--------+-------------------------------+ ++--------+----+-------------------------------------------+ +|POSCA | 1 | posca_factor_ping | +| +----+-------------------------------------------+ +| | 2 | posca_factor_system_bandwidth | +| +----+-------------------------------------------+ +| | 3 | posca_facotor_soak_througputs | +| +----+-------------------------------------------+ +| | 4 | posca_feature_vnf_scale_up | +| +----+-------------------------------------------+ +| | 5 | posca_feature_vnf_scale_out | +| +----+-------------------------------------------+ +| | 6 | posca_factor_storperf | +| +----+-------------------------------------------+ +| | 7 | posca_factor_multistack_storage_parallel | +| +----+-------------------------------------------+ +| | 8 | posca_factor_multistack_storage | +| +----+-------------------------------------------+ +| | 9 | posca_feature_moon_resources | +| +----+-------------------------------------------+ +| | 10 | posca_feature_moon_tenants | ++--------+----+-------------------------------------------+ As for the abandoned test suite in the previous Bottlenecks releases, please refer to http://docs.opnfv.org/en/stable-danube/submodules/bottlenecks/docs/testing/user/userguide/deprecated.html. diff --git a/docs/testing/user/userguide/posca/posca_factor_multistack_storage.rst b/docs/testing/user/userguide/posca/posca_factor_multistack_storage.rst new file mode 100644 index 00000000..70f555d1 --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_factor_multistack_storage.rst @@ -0,0 +1,59 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +*************************************************** +POSCA Stress (Factor) Test of Multistack Storage +*************************************************** + +Test Case +======== + ++--------------------------------------------------------------------------------------+ +|Bottlenecks POSCA Stress Test MultiStack Storage | +| | ++--------------+-----------------------------------------------------------------------+ +|test case name| posca_factor_multistack_storage | +| | | ++--------------+-----------------------------------------------------------------------+ +|description | Stress test regarding multistack storage using | +| | yardstick as a runner | ++--------------+-----------------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/posca_factor_multistack_storage.yaml | +| | | +| | stack number: 5, 10, 20, 50 ... | +| | | ++--------------+-----------------------------------------------------------------------+ +|test result | Read / Write IOPS, Throughput, latency | +| | | ++--------------+-----------------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: fio + test_times: 10 + rw: write, read, rw, rr, randomrw + bs: 4k + size: 50g + rwmixwrite: 50 + num_stack: 1, 3 + volume_num: 1 + numjobs: 1 + direct: 1 + + contexts: + stack_create: yardstick + flavor: + yardstick_test_ip: + yardstick_test_dir: "samples" + yardstick_testcase: "storage_bottlenecks" + + dashboard: + dashboard: "y" + dashboard_ip: diff --git a/docs/testing/user/userguide/posca/posca_factor_multistack_storage_parallel.rst b/docs/testing/user/userguide/posca/posca_factor_multistack_storage_parallel.rst new file mode 100644 index 00000000..f8343bde --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_factor_multistack_storage_parallel.rst @@ -0,0 +1,58 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +*************************************************** +POSCA Stress (Factor) Test of Multistack Storage +*************************************************** + +Test Case +======== + ++------------------------------------------------------------------------------------------------+ +|Bottlenecks POSCA Stress Test Storage (Multistack with Yardstick) | +| | ++--------------+---------------------------------------------------------------------------------+ +|test case name| posca_factor_multistack_storage_parallel | +| | | ++--------------+---------------------------------------------------------------------------------+ +|description | Stress test regarding storage while using yardstick | +| | for multistack as a runner | ++--------------+---------------------------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/posca_factor_multistack_storage_parallel.yaml | +| | | +| | | ++--------------+---------------------------------------------------------------------------------+ +|test result | Read / Write IOPS, Throughput, latency | +| | | ++--------------+---------------------------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: fio + test_times: 10 + rw: write, read, rw, rr, randomrw + bs: 4k + size: 50g + rwmixwrite: 50 + num_stack: 1, 3 + volume_num: 1 + numjobs: 1 + direct: 1 + + contexts: + stack_create: yardstick + flavor: + yardstick_test_ip: + yardstick_test_dir: "samples" + yardstick_testcase: "storage_bottlenecks" + + dashboard: + dashboard: "y" + dashboard_ip: diff --git a/docs/testing/user/userguide/posca/posca_factor_soak_throughputs.rst b/docs/testing/user/userguide/posca/posca_factor_soak_throughputs.rst new file mode 100644 index 00000000..9e427b15 --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_factor_soak_throughputs.rst @@ -0,0 +1,51 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +************************************* +POSCA Factor Test of Soak Throughputs +************************************* + +Test Case +======== + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Soak Test Throughputs | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_factor_soak_throughputs | +| | | ++--------------+--------------------------------------------------------------+ +|description | Long duration stability tests of data-plane traffic | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/... | +| | posca_factor_soak_throughputs.yaml | +| | | ++--------------+--------------------------------------------------------------+ +|test result | THROUGHPUT,THROUGHPUT_UNITS,MEAN_LATENCY,LOCAL_CPU_UTIL, | +| | REMOTE_CPU_UTIL,LOCAL_BYTES_SENT,REMOTE_BYTES_RECVD | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: netperf + test_duration_hours: 1 + vim_pair_ttl: 300 + vim_pair_lazy_cre_delay: 2 + package_size: + threshhold: + package_loss: 0% + latency: 300 + + runners: + stack_create: yardstick + flavor: + yardstick_test_dir: "samples" + yardstick_testcase: "netperf_soak" diff --git a/docs/testing/user/userguide/posca/posca_factor_storperf.rst b/docs/testing/user/userguide/posca/posca_factor_storperf.rst new file mode 100644 index 00000000..2dc9ecdc --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_factor_storperf.rst @@ -0,0 +1,37 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +*************************************************** +POSCA Stress Test of Storage Usage +*************************************************** + +Test Case +======== + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Stress Test Storage | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_factor_storperf | +| | | ++--------------+--------------------------------------------------------------+ +|description | Stress test regarding storage using Storperf | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/posca_posca_storperf.yaml | +| | | +| | | ++--------------+--------------------------------------------------------------+ +|test result | Read / Write IOPS, Throughput, latency | +| | | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: storperf diff --git a/docs/testing/user/userguide/posca/posca_feature_moon_resources.rst b/docs/testing/user/userguide/posca/posca_feature_moon_resources.rst new file mode 100644 index 00000000..5f89bca8 --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_feature_moon_resources.rst @@ -0,0 +1,52 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +************************************************************ +POSCA feature Test of Moon Security for resources per tenant +************************************************************ + +Test Case +========= + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Soak Test Throughputs | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_feature_moon_resources | +| | | ++--------------+--------------------------------------------------------------+ +|description | Moon authentication capability test for maximum number of | +| | authentication operations per tenant | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/... | +| | posca_feature_moon_resources.yaml | +| | | ++--------------+--------------------------------------------------------------+ +|test result | number of tenants, max number of users | +| | | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: https request + # info that the cpus and memes have the same number of data. + pdp_name: pdp + policy_name: "MLS Policy example" + model_name: MLS + tenants: 1,5,10,20 + subject_number: 10 + object_number: 10 + timeout: 0.2 + + runners: + stack_create: yardstick + Debug: False + yardstick_test_dir: "samples" + yardstick_testcase: "moon_resource" diff --git a/docs/testing/user/userguide/posca/posca_feature_moon_tenants.rst b/docs/testing/user/userguide/posca/posca_feature_moon_tenants.rst new file mode 100644 index 00000000..1ac93f8d --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_feature_moon_tenants.rst @@ -0,0 +1,55 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +*********************************************** +POSCA feature Test of Moon Security for Tenants +*********************************************** + +Test Case +========= + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Soak Test Throughputs | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_feature_moon_tenants | +| | | ++--------------+--------------------------------------------------------------+ +|description | Moon authentication capability test for maximum tenants | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/... | +| | posca_feature_moon_tenants.yaml | +| | | ++--------------+--------------------------------------------------------------+ +|test result | Max number of tenants | +| | | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: https request + # info that the cpus and memes have the same number of data. + pdp_name: pdp + policy_name: "MLS Policy example" + model_name: MLS + subject_number: 20 + object_number: 20 + timeout: 0.003 + initial_tenants: 0 + steps_tenants: 1 + tolerate_time: 20 + SLA: 5 + + runners: + stack_create: yardstick + Debug: False + yardstick_test_dir: "samples" + yardstick_testcase: "moon_tenant" diff --git a/docs/testing/user/userguide/posca/posca_feature_vnf_scale_out.rst b/docs/testing/user/userguide/posca/posca_feature_vnf_scale_out.rst new file mode 100644 index 00000000..baf30578 --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_feature_vnf_scale_out.rst @@ -0,0 +1,46 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +*********************************** +POSCA feature Test of VNF Scale Out +*********************************** + +Test Case +========= + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Soak Test Throughputs | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_feature_nfv_scale_out | +| | | ++--------------+--------------------------------------------------------------+ +|description | SampleVNF Scale Out Test | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/... | +| | posca_feature_nfv_scale_out.yaml | +| | | ++--------------+--------------------------------------------------------------+ +|test result | throughputs, latency, loss rate | +| | | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + number_vnfs: 1, 2, 4 + iterations: 10 + interval: 35 + + runners: + stack_create: yardstick + flavor: + yardstick_test_dir: "samples/vnf_samples/nsut/acl" + yardstick_testcase: "tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_traffic_scale_out" diff --git a/docs/testing/user/userguide/posca_guide.rst b/docs/testing/user/userguide/posca_guide.rst index 009aedb1..934415f0 100644 --- a/docs/testing/user/userguide/posca_guide.rst +++ b/docs/testing/user/userguide/posca_guide.rst @@ -30,7 +30,7 @@ Scopes of the POSCA testsuite: b) Parameters choosing and Algorithms. Test stories of POSCA testsuite: - a) Factor test (Stress test): base test cases that Feature test and Optimization will be dependant on. + a) Factor test (Stress test): base test cases that Feature test and Optimization will be dependant on or stress test to validate system. b) Feature test: test cases for features/scenarios. c) Optimization test: test to tune the system parameter. @@ -45,6 +45,8 @@ Preinstall Packages * Docker: https://docs.docker.com/engine/installation/ * For Ubuntu, please refer to https://docs.docker.com/engine/installation/linux/ubuntu/ +[Since Euphrates release, the docker-compose package is not required.] + * Docker-Compose: https://docs.docker.com/compose/ .. code-block:: bash @@ -97,14 +99,16 @@ Edit admin_rc.sh and add the following line export OS_CACERT=/tmp/os_cacert -If you are using compass, fuel, apex or joid to deploy your openstack -environment, you could use the following command to get the required files. +If you have deployed your openstack environment by compass, +you could use the following command to get the required files. As to Fuel, Apex and JOID installers, we only provide limited support now +for retrieving the configuration/description files. If you find that the following command can not do the magic, you should put the +required files in /tmp manually. .. code-block:: bash - bash /utils/env_prepare/config_prepare.sh -i <installer> [--debug] + bash ./utils/env_prepare/config_prepare.sh -i <installer> [--debug] -Note that if we execute the command above, then admin_rc.sh and pod.yml gets created automatically in /tmp folder along with the line `export OS_CACERT=/tmp/os_cacert` added in admin_rc.sh file. +Note that if we execute the command above, then admin_rc.sh and pod.yml will be created automatically in /tmp folder along with the line `export OS_CACERT=/tmp/os_cacert` added in admin_rc.sh file. Executing Specified Testcase @@ -116,9 +120,9 @@ Executing Specified Testcase bottlenecks testcase|teststory run <testname> - For the *testcase* command, testname should be as the same name of the test case configuration file located in testsuites/posca/testcase_cfg. - For stress tests in Danube/Euphrates, *testcase* should be replaced by either *posca_factor_ping* or *posca_factor_system_bandwidth*. - For the *teststory* command, a user can specify the test cases to be executed by defining it in a teststory configuration file located in testsuites/posca/testsuite_story. There is also an example there named *posca_factor_test*. +For the *testcase* command, testname should be as the same name of the test case configuration file located in testsuites/posca/testcase_cfg. +For stress tests in Danube/Euphrates, *testcase* should be replaced by either *posca_factor_ping* or *posca_factor_system_bandwidth*. +For the *teststory* command, a user can specify the test cases to be executed by defining it in a teststory configuration file located in testsuites/posca/testsuite_story. There is also an example there named *posca_factor_test*. 2. There are also other 2 ways to run test cases and test stories. diff --git a/docs/testing/user/userguide/test_cases.rst b/docs/testing/user/userguide/test_cases.rst index 57e338e9..b6b1d17f 100644 --- a/docs/testing/user/userguide/test_cases.rst +++ b/docs/testing/user/userguide/test_cases.rst @@ -11,3 +11,10 @@ Bottlenecks - Test Cases ./posca/posca_factor_system_bandwidth.rst ./posca/posca_factor_ping.rst + ./posca/posca_factor_storperf.rst + ./posca/posca_factor_multistack_storage.rst + ./posca/posca_factor_multistack_storage_parallel.rst + ./posca/posca_factor_soak_throughputs.rst + ./posca/posca_feature_moon_resources.rst + ./posca/posca_feature_moon_tenants.rst + ./posca/posca_feature_vnf_scale_out.rst |