diff options
9 files changed, 288 insertions, 24 deletions
diff --git a/docs/release/release-notes/release_notes.rst b/docs/release/release-notes/release_notes.rst index b52830ec..d26891f9 100644 --- a/docs/release/release-notes/release_notes.rst +++ b/docs/release/release-notes/release_notes.rst @@ -59,6 +59,9 @@ Version History | Nov 15th, 2017 | 1.6.1 | Bottlenecks Euphrates release 5.1 | | | | | +----------------+--------------------+-----------------------------------+ +| Apr 10th, 2018 | 1.7 | Bottlenecks Fraser release 6.0 | +| | | | ++----------------+--------------------+-----------------------------------+ Summary ======= @@ -81,6 +84,33 @@ Bottlenecks_ team. Release Data ============ +Fraser Release Data +----------------------- + ++--------------------------------------+--------------------------------+ +| **Project** | Bottlenecks | +| | | ++--------------------------------------+--------------------------------+ +| **Repo/tag** | * Bottlenecks/6.0.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Bottlenecks Docker image tag** | * 6.0.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Release designation** | * 6.0.0 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Release date** | * Apr 10th 2018 | +| | | +| | | ++--------------------------------------+--------------------------------+ +| **Purpose of the delivery** | Fraser stable release | +| | | ++--------------------------------------+--------------------------------+ + Euphrates Release Data ----------------------- @@ -206,8 +236,8 @@ Bramaputra Release Data +--------------------------------------+--------------------------------+ -Danube Deliverables -=================== +Release Deliverables +==================== Software Deliverables --------------------- @@ -220,6 +250,12 @@ Software Deliverables Documentatiion Deliverables --------------------------- +**Bottlenecks documentation <fraser>** + +* Release Notes: http://docs.opnfv.org/en/stable-fraser/submodules/bottlenecks/docs/release/release-notes/release_notes.html +* User Guide: http://docs.opnfv.org/en/stable-fraser/submodules/bottlenecks/docs/testing/user/userguide/index.html +* Developer Guide: http://docs.opnfv.org/en/stable-fraser/submodules/bottlenecks/docs/testing/developer/devguide/index.html + **Bottlenecks documentation <euphrates>** * Release Notes: http://docs.opnfv.org/en/stable-euphrates/submodules/bottlenecks/docs/release/release-notes/release_notes.html @@ -240,6 +276,16 @@ Documentatiion Deliverables * http://artifacts.opnfv.org/bottlenecks/colorado/1.0/releasenotes/index.html * User Guide: http://artifacts.opnfv.org/bottlenecks/colorado/1.0/docs/userguide/index.html +Reason for Fraser +================= + +* Introduction of data-plane soak tests aiming at benchmarking long duration stability of OPNFV platform +* Introduction of security tests by cooperating with Moon project to validate the software security based authentication schemes +* Introduction of multi-stack storage tests and Storperf storage test +* Testing framework provides local supports for preparing images for openstack resulting in offline running supports for Bottlenecks + +These tests cases and refactoring further enhance the capability of Bottlenecks project and let it be adaptable to different usages. + Reason for Euphrates ==================== diff --git a/docs/release/release-notes/requirements_notes.rst b/docs/release/release-notes/requirements_notes.rst index eb5cd743..7476f75a 100644 --- a/docs/release/release-notes/requirements_notes.rst +++ b/docs/release/release-notes/requirements_notes.rst @@ -11,9 +11,10 @@ Bottlenecks Requirements Notes for OPNFV Euphrates Release Installer Requirements ====================== -Bottlenecks Euphrates release is installer-agnostic which means that +Bottlenecks releases after Euphrates are installer-agnostic which means that the test cases could be executed over different deployments. -However, the pod description file and openstack rc file +However, the pod description file, +openstack rc file and the admin credentials are required to automatically get the SUT informations. ===================== @@ -30,7 +31,7 @@ Software Requirements Operation System ---------------- -Ubuntu 14.04 is the default system. However, we do not do any linux core operation or call any system related api directly. The software could be runned in higher versions of Ubuntu with high probability. Thorough tests are not performed. +Ubuntu 14.04 is the default system. However, we do not do any linux core operation or call any system related api directly. The software could be runned in higher versions of Ubuntu with high probability. Cloud OS -------- diff --git a/docs/testing/user/userguide/overview.rst b/docs/testing/user/userguide/overview.rst index 697191b0..725a4b9d 100644 --- a/docs/testing/user/userguide/overview.rst +++ b/docs/testing/user/userguide/overview.rst @@ -55,26 +55,32 @@ Integration Description +-------------+----------------------+----------------------+ | Euphrates | Any | POSCA | +-------------+----------------------+----------------------+ +| Fraser | Any | POSCA | ++-------------+----------------------+----------------------+ Test suite & Test case Description ================================== -+--------+-------------------------------------------+ -|POSCA | posca_factor_ping | -| +-------------------------------------------+ -| | posca_factor_system_bandwidth | -| +-------------------------------------------+ -| | posca_facotor_soak_througputs | -| +-------------------------------------------+ -| | posca_feature_vnf_scale_up | -| +-------------------------------------------+ -| | posca_feature_vnf_scale_out | -| +-------------------------------------------+ -| | posca_factor_storperf | -| +-------------------------------------------+ -| | posca_factor_multistack_storage_parallel | -| +-------------------------------------------+ -| | posca_factor_multistack_storage | -+--------+-------------------------------------------+ ++--------+----+-------------------------------------------+ +|POSCA | 1 | posca_factor_ping | +| +----+-------------------------------------------+ +| | 2 | posca_factor_system_bandwidth | +| +----+-------------------------------------------+ +| | 3 | posca_facotor_soak_througputs | +| +----+-------------------------------------------+ +| | 4 | posca_feature_vnf_scale_up | +| +----+-------------------------------------------+ +| | 5 | posca_feature_vnf_scale_out | +| +----+-------------------------------------------+ +| | 6 | posca_factor_storperf | +| +----+-------------------------------------------+ +| | 7 | posca_factor_multistack_storage_parallel | +| +----+-------------------------------------------+ +| | 8 | posca_factor_multistack_storage | +| +----+-------------------------------------------+ +| | 9 | posca_feature_moon_resources | +| +----+-------------------------------------------+ +| | 10 | posca_feature_moon_tenants | ++--------+----+-------------------------------------------+ As for the abandoned test suite in the previous Bottlenecks releases, please refer to http://docs.opnfv.org/en/stable-danube/submodules/bottlenecks/docs/testing/user/userguide/deprecated.html. diff --git a/docs/testing/user/userguide/posca/posca_factor_soak_throughputs.rst b/docs/testing/user/userguide/posca/posca_factor_soak_throughputs.rst new file mode 100644 index 00000000..9e427b15 --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_factor_soak_throughputs.rst @@ -0,0 +1,51 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +************************************* +POSCA Factor Test of Soak Throughputs +************************************* + +Test Case +======== + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Soak Test Throughputs | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_factor_soak_throughputs | +| | | ++--------------+--------------------------------------------------------------+ +|description | Long duration stability tests of data-plane traffic | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/... | +| | posca_factor_soak_throughputs.yaml | +| | | ++--------------+--------------------------------------------------------------+ +|test result | THROUGHPUT,THROUGHPUT_UNITS,MEAN_LATENCY,LOCAL_CPU_UTIL, | +| | REMOTE_CPU_UTIL,LOCAL_BYTES_SENT,REMOTE_BYTES_RECVD | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: netperf + test_duration_hours: 1 + vim_pair_ttl: 300 + vim_pair_lazy_cre_delay: 2 + package_size: + threshhold: + package_loss: 0% + latency: 300 + + runners: + stack_create: yardstick + flavor: + yardstick_test_dir: "samples" + yardstick_testcase: "netperf_soak" diff --git a/docs/testing/user/userguide/posca/posca_feature_moon_resources.rst b/docs/testing/user/userguide/posca/posca_feature_moon_resources.rst new file mode 100644 index 00000000..5f89bca8 --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_feature_moon_resources.rst @@ -0,0 +1,52 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +************************************************************ +POSCA feature Test of Moon Security for resources per tenant +************************************************************ + +Test Case +========= + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Soak Test Throughputs | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_feature_moon_resources | +| | | ++--------------+--------------------------------------------------------------+ +|description | Moon authentication capability test for maximum number of | +| | authentication operations per tenant | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/... | +| | posca_feature_moon_resources.yaml | +| | | ++--------------+--------------------------------------------------------------+ +|test result | number of tenants, max number of users | +| | | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: https request + # info that the cpus and memes have the same number of data. + pdp_name: pdp + policy_name: "MLS Policy example" + model_name: MLS + tenants: 1,5,10,20 + subject_number: 10 + object_number: 10 + timeout: 0.2 + + runners: + stack_create: yardstick + Debug: False + yardstick_test_dir: "samples" + yardstick_testcase: "moon_resource" diff --git a/docs/testing/user/userguide/posca/posca_feature_moon_tenants.rst b/docs/testing/user/userguide/posca/posca_feature_moon_tenants.rst new file mode 100644 index 00000000..1ac93f8d --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_feature_moon_tenants.rst @@ -0,0 +1,55 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +*********************************************** +POSCA feature Test of Moon Security for Tenants +*********************************************** + +Test Case +========= + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Soak Test Throughputs | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_feature_moon_tenants | +| | | ++--------------+--------------------------------------------------------------+ +|description | Moon authentication capability test for maximum tenants | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/... | +| | posca_feature_moon_tenants.yaml | +| | | ++--------------+--------------------------------------------------------------+ +|test result | Max number of tenants | +| | | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + tool: https request + # info that the cpus and memes have the same number of data. + pdp_name: pdp + policy_name: "MLS Policy example" + model_name: MLS + subject_number: 20 + object_number: 20 + timeout: 0.003 + initial_tenants: 0 + steps_tenants: 1 + tolerate_time: 20 + SLA: 5 + + runners: + stack_create: yardstick + Debug: False + yardstick_test_dir: "samples" + yardstick_testcase: "moon_tenant" diff --git a/docs/testing/user/userguide/posca/posca_feature_vnf_scale_out.rst b/docs/testing/user/userguide/posca/posca_feature_vnf_scale_out.rst new file mode 100644 index 00000000..baf30578 --- /dev/null +++ b/docs/testing/user/userguide/posca/posca_feature_vnf_scale_out.rst @@ -0,0 +1,46 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Tech and others. + +*********************************** +POSCA feature Test of VNF Scale Out +*********************************** + +Test Case +========= + ++-----------------------------------------------------------------------------+ +|Bottlenecks POSCA Soak Test Throughputs | +| | ++--------------+--------------------------------------------------------------+ +|test case name| posca_feature_nfv_scale_out | +| | | ++--------------+--------------------------------------------------------------+ +|description | SampleVNF Scale Out Test | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | config file: | +| | /testsuite/posca/testcase_cfg/... | +| | posca_feature_nfv_scale_out.yaml | +| | | ++--------------+--------------------------------------------------------------+ +|test result | throughputs, latency, loss rate | +| | | ++--------------+--------------------------------------------------------------+ + +Configuration +============ +:: + + load_manager: + scenarios: + number_vnfs: 1, 2, 4 + iterations: 10 + interval: 35 + + runners: + stack_create: yardstick + flavor: + yardstick_test_dir: "samples/vnf_samples/nsut/acl" + yardstick_testcase: "tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_traffic_scale_out" diff --git a/docs/testing/user/userguide/posca_guide.rst b/docs/testing/user/userguide/posca_guide.rst index ec623d03..934415f0 100644 --- a/docs/testing/user/userguide/posca_guide.rst +++ b/docs/testing/user/userguide/posca_guide.rst @@ -100,7 +100,7 @@ Edit admin_rc.sh and add the following line export OS_CACERT=/tmp/os_cacert If you have deployed your openstack environment by compass, -you could use the following command to get the required files. For Fuel, Apex and JOID installer, we only provide limited support now +you could use the following command to get the required files. As to Fuel, Apex and JOID installers, we only provide limited support now for retrieving the configuration/description files. If you find that the following command can not do the magic, you should put the required files in /tmp manually. @@ -108,7 +108,7 @@ required files in /tmp manually. bash ./utils/env_prepare/config_prepare.sh -i <installer> [--debug] -Note that if we execute the command above, then admin_rc.sh and pod.yml gets created automatically in /tmp folder along with the line `export OS_CACERT=/tmp/os_cacert` added in admin_rc.sh file. +Note that if we execute the command above, then admin_rc.sh and pod.yml will be created automatically in /tmp folder along with the line `export OS_CACERT=/tmp/os_cacert` added in admin_rc.sh file. Executing Specified Testcase diff --git a/docs/testing/user/userguide/test_cases.rst b/docs/testing/user/userguide/test_cases.rst index 57e338e9..b6b1d17f 100644 --- a/docs/testing/user/userguide/test_cases.rst +++ b/docs/testing/user/userguide/test_cases.rst @@ -11,3 +11,10 @@ Bottlenecks - Test Cases ./posca/posca_factor_system_bandwidth.rst ./posca/posca_factor_ping.rst + ./posca/posca_factor_storperf.rst + ./posca/posca_factor_multistack_storage.rst + ./posca/posca_factor_multistack_storage_parallel.rst + ./posca/posca_factor_soak_throughputs.rst + ./posca/posca_feature_moon_resources.rst + ./posca/posca_feature_moon_tenants.rst + ./posca/posca_feature_vnf_scale_out.rst |