summaryrefslogtreecommitdiffstats
path: root/docs/testing
diff options
context:
space:
mode:
Diffstat (limited to 'docs/testing')
-rw-r--r--docs/testing/developer/dashboard_guide.rst21
-rw-r--r--docs/testing/developer/index.rst16
-rw-r--r--docs/testing/developer/overview.rst94
-rw-r--r--docs/testing/developer/posca_guide.rst56
-rw-r--r--docs/testing/developer/rubbos_guide.rst128
-rw-r--r--docs/testing/developer/vstf_guide.rst241
-rw-r--r--docs/testing/index.rst46
-rw-r--r--docs/testing/user/index.rst29
-rw-r--r--docs/testing/user/posca/posca_stress_ping.rst52
-rw-r--r--docs/testing/user/posca/posca_stress_traffic.rst49
-rw-r--r--docs/testing/user/rubbos/rubbos_basic.rst30
-rw-r--r--docs/testing/user/rubbos/rubbos_heavy_tc1101.rst30
-rw-r--r--docs/testing/user/rubbos/rubbos_tc1101.rst30
-rw-r--r--docs/testing/user/rubbos/rubbos_tc1201.rst30
-rw-r--r--docs/testing/user/rubbos/rubbos_tc1301.rst30
-rw-r--r--docs/testing/user/rubbos/rubbos_tc1401.rst30
-rw-r--r--docs/testing/user/vstf/vstf_Ti1.rst28
-rw-r--r--docs/testing/user/vstf/vstf_Ti2.rst28
-rw-r--r--docs/testing/user/vstf/vstf_Ti3.rst28
-rw-r--r--docs/testing/user/vstf/vstf_Tn1.rst28
-rw-r--r--docs/testing/user/vstf/vstf_Tn2.rst28
-rw-r--r--docs/testing/user/vstf/vstf_Tu1.rst28
-rw-r--r--docs/testing/user/vstf/vstf_Tu2.rst28
-rw-r--r--docs/testing/user/vstf/vstf_Tu3.rst28
24 files changed, 1136 insertions, 0 deletions
diff --git a/docs/testing/developer/dashboard_guide.rst b/docs/testing/developer/dashboard_guide.rst
new file mode 100644
index 00000000..5679e508
--- /dev/null
+++ b/docs/testing/developer/dashboard_guide.rst
@@ -0,0 +1,21 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Huawei Technologies Co.,Ltd and others.
+
+**************
+Dashbard guide
+**************
+
+Scope
+=====
+This document provides an overview of the results of test cases developed by
+the OPNFV Bottlenecks Project, executed on OPNFV community labs.
+
+OPNFV CI(Continous Integration) system provides automated build, deploy and testing for
+the software developed in OPNFV. Unless stated, the reported tests are
+automated via Jenkins Jobs.
+
+Test results are visible in the following dashboard:
+
+* *Testing dashboard*: uses Mongo DB to store test results and Bitergia for
+ visualization, which includes the rubbos test result, vstf test result.
diff --git a/docs/testing/developer/index.rst b/docs/testing/developer/index.rst
new file mode 100644
index 00000000..20cc00eb
--- /dev/null
+++ b/docs/testing/developer/index.rst
@@ -0,0 +1,16 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Huawei Technologies Co.,Ltd and others.
+
+***************************
+Bottlenecks - Testing Guide
+***************************
+
+.. toctree::
+ :maxdepth: 2
+
+ ./overview.rst
+ ./posca_guide.rst
+ ./rubbos_guide.rst
+ ./vstf_guide.rst
+ ./dashboard_guide.rst
diff --git a/docs/testing/developer/overview.rst b/docs/testing/developer/overview.rst
new file mode 100644
index 00000000..8bc1b7c1
--- /dev/null
+++ b/docs/testing/developer/overview.rst
@@ -0,0 +1,94 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Huawei Technologies Co.,Ltd and others.
+
+*********************
+Project Testing Guide
+*********************
+
+For each *test suite*, you can either setup *test story* or *test case* to run
+certain test. *test story* could include several *test cases* as a set in one
+configuration file. You could then call the *test story* or *test case* by using
+Bottlencks CLI or Python build process.
+Details will be shown in the following section.
+
+Brief Introdcution of the Test suites in Project Releases
+=============================================================
+
+Brahmaputra: rubbos is introduced, which is an end2end NFVI perforamnce tool.
+Virtual switch test framework(VSTF) is also introduced,
+which is an test framework used for vswitch performance test.
+
+Colorado: rubbos is refactored by using puppet, which makes it quite flexible
+to configure with different number of load generator(Client), worker(tomcat).
+vstf is refactored by extracting the test case's configuration information.
+
+Danube: posca testsuite is introduced to implementing stress (factor), scenario and
+tuning test in parametric manner. Two testcases are developed and integrated into
+community CI pipeline.
+
+Integration Description
+=======================
++-------------+----------------------+----------------------+
+| Release | integrated installer | Supported Testsuite |
++-------------+----------------------+----------------------+
+| Brahmaputra | Fuel | Rubbos, VSTF |
++-------------+----------------------+----------------------+
+| Colorado | Compass | Rubbos, VSTF |
++-------------+----------------------+----------------------+
+| Danube | Compass | POSCA |
++-------------+----------------------+----------------------+
+
+Test suite & Test case Description
+==================================
++--------+-------------------------------+
+|Rubbos | rubbos_basic |
+| +-------------------------------+
+| | rubbos_TC1101 |
+| +-------------------------------+
+| | rubbos_TC1201 |
+| +-------------------------------+
+| | rubbos_TC1301 |
+| +-------------------------------+
+| | rubbos_TC1401 |
+| +-------------------------------+
+| | rubbos_heavy_TC1101 |
++--------+-------------------------------+
+|vstf | vstf_Ti1 |
+| +-------------------------------+
+| | vstf_Ti2 |
+| +-------------------------------+
+| | vstf_Ti3 |
+| +-------------------------------+
+| | vstf_Tn1 |
+| +-------------------------------+
+| | vstf_Tn2 |
+| +-------------------------------+
+| | vstf_Tu1 |
+| +-------------------------------+
+| | vstf_Tu2 |
+| +-------------------------------+
+| | vstf_Tu3 |
++--------+-------------------------------+
+|posca | posca_stress_ping |
+| +-------------------------------+
+| | posca_stress_traffic |
+| | |
+| | (posca_factor_sys_bandwidth) |
++--------+-------------------------------+
+
+Installation of Testsuites
+==========================
+
+* TODO
+
+Setting Up Configrations
+========================
+
+* TODO
+
+Run Tests Brief
+===============
+
+* TODO
+* More will be shown in the platform overview
diff --git a/docs/testing/developer/posca_guide.rst b/docs/testing/developer/posca_guide.rst
new file mode 100644
index 00000000..dbc7568c
--- /dev/null
+++ b/docs/testing/developer/posca_guide.rst
@@ -0,0 +1,56 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Huawei Technologies Co.,Ltd and others.
+
+*********************
+POSCA Testsuite Guide
+*********************
+
+
+POSCA Introduction
+====================
+The POSCA (Parametric Bottlenecks Testing Catalogue) testsuite
+classifies the bottlenecks test cases and results into 5 categories.
+Then the results will be analyzed and bottlenecks will be searched
+among these categories.
+
+The POSCA testsuite aims to locate the bottlenecks in parmetric
+manner and to decouple the bottlenecks regarding the deployment
+requirements.
+The POSCA testsuite provides an user friendly way to profile and
+understand the E2E system behavior and deployment requirements.
+
+Goals of the POSCA testsuite:
+ a) Automatically locate the bottlenecks in a iterative manner.
+ b) Automatically generate the testing report for bottlenecks in
+different categories.
+ c) Implementing Automated Staging.
+
+Scopes of the POSCA testsuite:
+ a) Modeling, Testing and Test Result analysis.
+ b) Parameters choosing and Algorithms.
+
+Test stories of POSCA testsuite:
+ a) Factor test (Stress test): base test cases that Feature test and Optimization will be
+dependant on.
+ b) Feature test: test cases for features/scenarios.
+ c) Optimization test: test to tune the system parameter.
+
+Detailed workflow is illutrated below.
+* TODO Add image here
+
+Preinstall Packages
+====================
+* TODO Description of dependent packages
+
+Run POSCA Locally
+=================
+* TO Description of POSCA testing steps
+
+Run POSCA through Community CI
+==============================
+* TODO Description of POSCA integrated into CI system
+
+Test Result Description
+=======================
+* TODO hwo to access the test result
diff --git a/docs/testing/developer/rubbos_guide.rst b/docs/testing/developer/rubbos_guide.rst
new file mode 100644
index 00000000..4d92d2e2
--- /dev/null
+++ b/docs/testing/developer/rubbos_guide.rst
@@ -0,0 +1,128 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Huawei Technologies Co.,Ltd and others.
+
+**********************
+Rubbos Testsuite Guide
+**********************
+
+
+Rubbos Introduction
+====================
+Rubbos is a bulletin board benchmark modeled after an online news forum like Slashdot.
+It is an open source Middleware and an n-tier system model which
+is used to be deployed on multiple physical node and
+to measure the whole performacne of OPNFV platform.
+Rubbos can deploy the Apache, tomcat, and DB.
+Based on the deployment, rubbos gives the pressure to the whole system.
+When the system reaches to the peak, the throughput will not grow more.
+This testcase can help to understand the bottlenecks of OPNFV plantform
+and improve the performance of OPNFV platform.
+
+Detailed workflow is illutrated below.
+
+.. image:: ../images/Framework_Setup.png
+ :width: 770px
+ :alt: Bottlenecks Framework Setup
+
+Preinstall Packages
+====================
+There is a need to install some packages before running the rubbos,
+gcc, gettext, g++, libaio1, libaio-dev, make and git are necessary.
+When the rubbos runs on the OPNFV community continuous integration(CI)
+system, the required packages are installed automately as shown in the
+code repository, which is /utils/infra_setup/vm_dev_setup/packages.conf,
+besides, the packages can be encapsulated in the images initially.
+If someone wants to use rubbos locally, he/she has to install them by
+hand, such as in ubuntu 14.04,
+
+.. code-block:: bash
+
+ apt-get update
+ apt-get install gettext
+
+How does Rubbos Integrate into Installers
+=========================================
+1.Community CI System
+
+Rubbos has been successfully integrated into fuel and compass with NOSDN scenario
+in OPNFV community CI system.
+
+Heat is used to create 9 instances, which is shown in
+/utils/infra_setup/heat_template/HOT_create_instance.sh, the 9 instances are used
+for installing Apache, Tomcat, Mysql, Control, Benchmark and 4 Clients. The tools,
+such as rubbos, sysstat, oprofile, etc, are installed in these instances to perform
+the test, the test results are stored in the Benchmark instance initially, then they
+are copied to the Rubbos_result instance, finally, the test results are transferred to
+the community dashboard.
+
+There's a need to store our pakages as large as M bytes or G bytes size, such as
+the images, jdk, apache-ant, apache-tomcat, etc, the OPNFV community storage system,
+Google Cloud Storage, is used, the pakages can be downloaded from
+https://artifacts.opnfv.org/bottlenecks/rubbos.
+
+2.Local Deployment
+
+If someone wants to run the rubbos in his own environment, he/she can keep to the following steps,
+
+2.1 Start up instances by using heat, nova or libvert. In Openstack Environemnt,
+the heat script can refer /utils/infra_setup/heat_template/HOT_create_instance.sh,
+if the openstack doesn't support heat module,
+the script /utils/infra_setup/create_instance.sh can be used.
+Without Openstack, there's a way to set up instances by using libvert, the scripts are shown under
+the directory /utils/rubbos_dev_env_setup.
+
+The image can be downloaded from the community cloud storage
+
+.. code-block:: bash
+
+ curl --connect-timeout 10 -o bottlenecks-trusty-server.img
+ http://artifacts.opnfv.org/bottlenecks/rubbos/bottlenecks-trusty-server.img
+
+2.2 Ssh into the control node and clone the bottlenecks codes to the root directory.
+
+.. code-block:: bash
+
+ git clone https://git.opnfv.org/bottlenecks /bottlenecks
+
+2.3 Download the packages and decompress them into the proper directory.
+
+.. code-block:: bash
+
+ curl --connect-timeout 10 -o app_tools.tar.gz
+ http://artifacts.opnfv.org/bottlenecks/rubbos/app_tools.tar.gz
+ curl --connect-timeout 10 -o rubbosMulini6.tar.gz
+ http://artifacts.opnfv.org/bottlenecks/rubbos/rubbosMulini6.tar.gz
+
+.. code-block:: bash
+
+ tar zxf app_tools.tar.gz -C /bottlenecks/rubbos
+ tar zxf rubbosMulini6.tar.gz -C /bottlenecks/rubbos/rubbos_scripts
+
+2.4 Ssh into the Control node and run the script
+
+.. code-block:: bash
+
+ source /bottlenecks/rubbos/rubbos_scripts/1-1-1/scripts/run.sh
+
+2.5 Check the test results under the directory /bottlenecks/rubbos/rubbos_results in
+Control node. The results are stored in the format of xml,
+move them to the brower chrome, then you can see the results.
+
+Test Result Description
+=======================
+In OPNFV community, the result is shown in the following format
+
+::
+
+ [{'client': 200, 'throughput': 27},
+ {'client': 700, 'throughput': 102},
+ {'client': 1200, 'throughput': 177},
+ {'client': 1700, 'throughput': 252},
+ {'client': 2200, 'throughput': 323},
+ {'client': 2700, 'throughput': 399},
+ {'client': 3200, 'throughput': 473}]
+
+The results are transferred to the community database and a map is drawed on the dashboard.
+Along with the growth of the number of the client, the throughput grows at first, then meets
+up with a point of inflexion, which is caused by the bottlenecks of the measured system.
diff --git a/docs/testing/developer/vstf_guide.rst b/docs/testing/developer/vstf_guide.rst
new file mode 100644
index 00000000..e1ad5f2b
--- /dev/null
+++ b/docs/testing/developer/vstf_guide.rst
@@ -0,0 +1,241 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Huawei Technologies Co.,Ltd and others.
+
+********************
+VSTF Testsuite Guide
+********************
+
+VSTF Introduction
+====================
+VSTF(Virtual Switch Test Framework) is a system-level testing framework in the
+area of network virtualization, and it could help you estimate the system switch
+ability and find out the network bottlenecks by main KPIs(bandwidth, latency,
+resource usage and so on), VSTF owns a methodology to define the test scenario and
+testcases, Now we could support Tu testcases in the Openstack environment, More
+scenarios and cases will be added.
+
+VSTF TestScenario
+-----------------
+1. Tu - VM to VM
+2. Tn - Physical Nic loopback
+3. TnV - VNF loopback
+4. Ti - VM to Physical Nic
+
+Pre-install Packages on the ubuntu 14.04 VM
+===========================================
+VSTF VM Preparation Steps
+-------------------------
+1. Create a ubuntu 14.04 VM
+2. Install dependency inside VM
+3. Install vstf python package inside VM
+
+VM preparation
+--------------
+Install python2.7 version and git
+
+::
+
+ sudo apt-get install python2.7
+ sudo apt-get install git
+
+Download Bottlenecks package
+
+::
+
+ sudo cd /home/
+ sudo git clone https://gerrit.opnfv.org/gerrit/bottlenecks
+
+Install the dependency
+
+::
+
+ sudo apt-get install python-pip
+ sudo pip install --upgrade pip
+ sudo dpkg-reconfigure dash
+ sudo apt-get install libjpeg-dev
+ sudo apt-get install libpng-dev
+ sudo apt-get install python-dev
+ sudo apt-get install python-testrepository
+ sudo apt-get install git
+ sudo apt-get install python-pika
+ sudo apt-get install python-oslo.config
+ sudo pip install -r /home/bottlenecks/vstf/requirements.txt
+
+Install vstf package
+
+::
+
+ sudo mkdir -p /var/log/vstf/
+ sudo cp -r /home/bottlenecks/vstf/etc/vstf/ /etc/
+ sudo mkdir -p /opt/vstf/
+ sudo cd /home/bottlenecks;sudo rm -rf build/
+ sudo python setup.py install
+
+Image on the Cloud
+------------------
++-----------+-------------------------------------------------------------+
+| Name | vstf-image |
++===========+=============================================================+
+| URL | http://artifacts.opnfv.org/bottlenecks/vstf-manager-new.img |
++-----------+-------------------------------------------------------------+
+| Format | QCOW2 |
++-----------+-------------------------------------------------------------+
+| Size | 5G |
++-----------+-------------------------------------------------------------+
+| User | root |
++-----------+-------------------------------------------------------------+
+| Passwd | root |
++-----------+-------------------------------------------------------------+
+There is a complete vstf image on the cloud ,you could download it and use it to
+deploy and run cases ,but do not need VM preparation steps.
+
+
+How is VSTF Integrated into Installers
+========================================
+VM requirements
+---------------
++------------------+----------+--------------------+-----------------------------------------------------+
+| Name | FLAVOR | IMAGE_NAME | NETWORK |
++==================+==========+====================+=====================================================+
+| vstf-manager | m1.large | vstf-image | control-plane=XX.XX.XX.XX |
++------------------+----------+--------------------+-----------------------------------------------------+
+| vstf-tester | m1.large | vstf-image | control-plane(eth0)=XX.XX.XX.XX |
+| | | | test-plane(eth1)=XX.XX.XX.XX |
++------------------+----------+--------------------+-----------------------------------------------------+
+| vstf-target | m1.large | vstf-image | control-plane(eth0)=XX.XX.XX.XX |
+| | | | test-plane(eth1)=XX.XX.XX.XX |
++------------------+----------+--------------------+-----------------------------------------------------+
+m1.large means 4U4G for the target image Size 5GB
+For the network used by VMs,network need two plane ,one plane is control plane and the other plane is test plane.
+
+OPNFV community Usage in the CI system
+--------------------------------------
++---------------------------------------+---------------------------------------+
+| Project Name | Project Categoty |
++=======================================+=======================================+
+| bottlenecks-daily-fuel-vstf-lf-master | bottlenecks |
++---------------------------------------+---------------------------------------+
+OPNFV community jenkins Project info
+
+Main Entrance for the ci test:
+
+::
+
+ cd /home/bottlenecks/ci;
+ bash -x vstf_run.sh
+
+Test on local(Openstack Environment)
+------------------------------------
+download the image file
+
+::
+
+ curl --connect-timeout 10 -o /tmp/vstf-manager.img \
+ http://artifacts.opnfv.org/bottlenecks/vstf-manager-new.img -v
+
+create the image file by the glance
+
+::
+
+ glance image-create --name $MANAGER_IMAGE_NAME \
+ --disk-format qcow2 \
+ --container-format bare \
+ --file /tmp/vstf-manager.img
+
+create the keypair for the image(anyone will be ok)
+
+::
+
+ cd /home/bottlenecks/utils/infra_setup/bottlenecks_key
+ nova keypair-add --pub_key $KEY_PATH/bottlenecks_key.pub $KEY_NAME
+
+create the vstf three VMs in the openstack by heat
+
+::
+
+ cd /home/bottlenecks/utils/infra_setup/heat_template/vstf_heat_template
+ heat stack-create vstf -f bottleneck_vstf.yaml
+
+launch the vstf process inside the vstf-manager vstf-tester vstf-target VMs
+
+::
+
+ cd /home/bottlenecks/utils/infra_setup/heat_template/vstf_heat_template
+ bash -x launch_vstf.sh
+
+edit the test scenario and test packet list in the vstf_test.sh, now support the Tu-1/2/3
+
+::
+
+ function fn_testing_scenario(){
+ ...
+ local test_length_list="64 128 256 512 1024"
+ local test_scenario_list="Tu-1 Tu-3"
+ ...
+ }
+
+launch the vstf script
+
+::
+
+ cd /home/bottlenecks/utils/infra_setup/heat_template/vstf_heat_template
+ bash -x vstf_test.sh
+
+Test Result Description
+=======================
+Result Format
+-------------
+For example after the test, The result will display as the following format
+
+::
+
+ { u'64': { u'AverageLatency': 0.063,
+ u'Bandwidth': 0.239,
+ u'CPU': 0.0,
+ u'Duration': 20,
+ u'MaximumLatency': 0.063,
+ u'MinimumLatency': 0.063,
+ u'MppspGhz': 0,
+ u'OfferedLoad': 100.0,
+ u'PercentLoss': 22.42,
+ u'RxFrameCount': 4309750.0,
+ u'RxMbps': 198.28,
+ u'TxFrameCount': 5555436.0,
+ u'TxMbps': 230.03}}
+
+Option Description
+------------------
++---------------------+---------------------------------------------------+
+| Option Name | Description |
++=====================+===================================================+
+| AverageLatency | The average latency data during the packet |
+| | transmission (Unit:microsecond) |
++---------------------+---------------------------------------------------+
+| Bandwidth | Network bandwidth(Unit:Million packets per second)|
++---------------------+---------------------------------------------------+
+| CPU | Total Resource Cpu usage(Unit: Ghz) |
++---------------------+---------------------------------------------------+
+| Duration | Test time(Unit: second) |
++---------------------+---------------------------------------------------+
+| MaximumLatency | The maximum packet latency during the packet |
+| | transmission (Unit:microsecond) |
++---------------------+---------------------------------------------------+
+| MinimumLatency | The maximum packet latency during the packet |
+| | transmission (Unit:microsecond) |
++---------------------+---------------------------------------------------+
+| MppspGhz | Million Packets per second with per CPU |
+| | resource Ghz(Unit: Mpps/Ghz) |
++---------------------+---------------------------------------------------+
+| OfferedLoad | The load of network offered |
++---------------------+---------------------------------------------------+
+| PercentLoss | The percent of frame loss rate |
++---------------------+---------------------------------------------------+
+| RxFrameCount | The total frame on Nic rx |
++---------------------+---------------------------------------------------+
+| RxMbps | The received bandwidth per second |
++---------------------+---------------------------------------------------+
+| TxFrameCount | The total frame on Nic rx |
++---------------------+---------------------------------------------------+
+| TxMbps | The send bandwidth per second |
++---------------------+---------------------------------------------------+
diff --git a/docs/testing/index.rst b/docs/testing/index.rst
new file mode 100644
index 00000000..70d3983b
--- /dev/null
+++ b/docs/testing/index.rst
@@ -0,0 +1,46 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Huawei Technologies Co.,Ltd and others.
+
+************************
+Bottlenecks Introduction
+************************
+
+**Welcome to Bottlenecks' documentation !**
+
+.. _Bottlenecks: https://wiki.opnfv.org/bottlenecks
+.. _Proposal: https://wiki.opnfv.org/download/attachments/2926187/bottlenecks_project_for_test_meeting_20150820.pptx?version=1&modificationDate=1458849192000&api=v2
+
+Bottlenecks_ is an OPNFV testing Project.
+
+The project's goal is to find system bottlenecks by testing and verifying
+OPNFV infrastructure in a staging environment before committing it to a production environment.
+Instead of debugging a deployment in production environment,
+an automatic method for executing benchmarks which plans to
+validate the deployment during staging is adopted.
+This project forms a staging framework to find bottlenecks and to do analysis of the OPNFV infrastructure.
+
+The *Bottlenecks* framework is deployed in several OPNFV community labs. It is
+installer, infrastructure and application independent.
+
+.. toctree::
+ :glob:
+
+ developer/index.rst
+ user/index.rst
+ *
+
+.. _Pharos: https://wiki.opnfv.org/pharos
+
+.. seealso:: Pharos_ for information on OPNFV community labs.
+
+Proposal_ for more project information.
+
+Contact Bottlenecks
+===================
+
+Feedback `Contact us`_
+
+.. _Contact us: opnfv-users@lists.opnfv.org
+
+
diff --git a/docs/testing/user/index.rst b/docs/testing/user/index.rst
new file mode 100644
index 00000000..8afd57ab
--- /dev/null
+++ b/docs/testing/user/index.rst
@@ -0,0 +1,29 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+**************************
+Bottlenecks - User Guide
+**************************
+
+.. This index.rst file is used to structure and organise the content in these directories.
+.. The documentation toolchain will create web and pdf versions of the content of this file.
+
+.. toctree::
+ :maxdepth: 2
+
+ ./rubbos/rubbos_basic.rst
+ ./rubbos/rubbos_tc1101.rst
+ ./rubbos/rubbos_tc1201.rst
+ ./rubbos/rubbos_tc1301.rst
+ ./rubbos/rubbos_tc1401.rst
+ ./rubbos/rubbos_heavy_tc1101.rst
+ ./vstf/vstf_Ti1.rst
+ ./vstf/vstf_Ti2.rst
+ ./vstf/vstf_Ti3.rst
+ ./vstf/vstf_Tn1.rst
+ ./vstf/vstf_Tn2.rst
+ ./vstf/vstf_Tu1.rst
+ ./vstf/vstf_Tu2.rst
+ ./vstf/vstf_Tu3.rst
+ ./posca/posca_stress_ping.rst
+ ./posca/posca_stress_traffic.rst
diff --git a/docs/testing/user/posca/posca_stress_ping.rst b/docs/testing/user/posca/posca_stress_ping.rst
new file mode 100644
index 00000000..846b4686
--- /dev/null
+++ b/docs/testing/user/posca/posca_stress_ping.rst
@@ -0,0 +1,52 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+***************************************************
+POSCA Stress (Factor) Test of Perfomance Life-Cycle
+***************************************************
+
+Test Case
+========
+
++-----------------------------------------------------------------------------+
+|Bottlenecks POSCA Stress Test Ping |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| posca_stress_ping |
+| | |
++--------------+--------------------------------------------------------------+
+|description | Stress test regarding perfomance life-cycle while using ping |
+| | to validate the VM pairs constructions |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/posca/testcase_cfg/posca_stress_ping |
+| | |
+| | stack number: 5, 10, 20, 50 ... |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | PKT loss rate, success rate, test time, latency |
+| | |
++--------------+--------------------------------------------------------------+
+
+Configuration
+============
+load_manager:
+ scenarios:
+ tool: ping
+ test_times: 100
+ package_size:
+ num_stack: 2, 5, 10
+ package_loss: 10%
+
+ contexts:
+ stack_create: yardstick
+ flavor:
+ yardstick_test_ip:
+ yardstick_test_dir: "samples"
+ yardstick_testcase: "ping_bottlenecks"
+
+dashboard:
+ dashboard: "y"
+ dashboard_ip:
diff --git a/docs/testing/user/posca/posca_stress_traffic.rst b/docs/testing/user/posca/posca_stress_traffic.rst
new file mode 100644
index 00000000..a995996e
--- /dev/null
+++ b/docs/testing/user/posca/posca_stress_traffic.rst
@@ -0,0 +1,49 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+**********************************************
+POSCA Stress (Factor) Test of System bandwidth
+**********************************************
+
+
+Test Case
+========
++-----------------------------------------------------------------------------+
+|Bottlenecks POSCA Stress Test Traffic |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| posca_stress_ping |
+| | |
++--------------+--------------------------------------------------------------+
+|description | Stress test regarding baseline of the system for a single |
+| | user, i.e., a VM pair while increasing the package size |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/posca/testcase_cfg/posca_factor_system_bandwith |
+| | |
+| | stack number: 1 |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | PKT loss rate, latency, throupht, cpu usage |
+| | |
++--------------+--------------------------------------------------------------+
+
+Configration
+===========
+test_config:
+ tool: netperf
+ protocol: tcp
+ test_time: 20
+ tx_pkt_sizes: 64, 256, 1024, 4096, 8192, 16384, 32768, 65536
+ rx_pkt_sizes: 64, 256, 1024, 4096, 8192, 16384, 32768, 65536
+ cpu_load: 0.9
+ latency: 100000
+runner_config:
+ dashboard: "y"
+ dashboard_ip:
+ stack_create: yardstick
+ yardstick_test_ip:
+ yardstick_test_dir: "samples"
+ yardstick_testcase: "netperf_bottlenecks"
diff --git a/docs/testing/user/rubbos/rubbos_basic.rst b/docs/testing/user/rubbos/rubbos_basic.rst
new file mode 100644
index 00000000..ffef6c48
--- /dev/null
+++ b/docs/testing/user/rubbos/rubbos_basic.rst
@@ -0,0 +1,30 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+************************************************
+Bottlenecks Rubbos Test Case Description Basic
+************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks Rubbos Basic |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_rubbos_Basic |
+| | |
++--------------+--------------------------------------------------------------+
+|description | Rubbos platform for 1 tomcat, 1 Apache and 1 mysql. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/rubbos/testcase_cfg/rubbos_basic.yaml |
+| | |
+| | client number: 1 |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/rubbos/rubbos_heavy_tc1101.rst b/docs/testing/user/rubbos/rubbos_heavy_tc1101.rst
new file mode 100644
index 00000000..08251c7b
--- /dev/null
+++ b/docs/testing/user/rubbos/rubbos_heavy_tc1101.rst
@@ -0,0 +1,30 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+******************************************************
+Bottlenecks Rubbos Test Case Description Heavy TC1101
+******************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks Rubbos TC Heavy1101 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_rubbos_heavy_tc1101 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | Rubbos platform for 1 tomcat, 1 Apache and 1 mysql. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/rubbos/testcase_cfg/rubbos-heavy_1-1-0-1.yaml |
+| | |
+| | client number: 10 |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/rubbos/rubbos_tc1101.rst b/docs/testing/user/rubbos/rubbos_tc1101.rst
new file mode 100644
index 00000000..2a261313
--- /dev/null
+++ b/docs/testing/user/rubbos/rubbos_tc1101.rst
@@ -0,0 +1,30 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+************************************************
+Bottlenecks Rubbos Test Case Description TC1101
+************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks Rubbos TC1101 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_rubbos_tc1101 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | Rubbos platform for 1 tomcat, 1 Apache and 1 mysql. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/rubbos/testcase_cfg/rubbos_1-1-0-1.yaml |
+| | |
+| | client number: 5 |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/rubbos/rubbos_tc1201.rst b/docs/testing/user/rubbos/rubbos_tc1201.rst
new file mode 100644
index 00000000..9b6a5f99
--- /dev/null
+++ b/docs/testing/user/rubbos/rubbos_tc1201.rst
@@ -0,0 +1,30 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+************************************************
+Bottlenecks Rubbos Test Case Description TC1201
+************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks Rubbos TC1201 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_rubbos_tc1201 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | Rubbos platform for 1 Apache, 2 tomcat and 1 mysql. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/rubbos/testcase_cfg/rubbos_1-2-0-1.yaml |
+| | |
+| | client number: 5 |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/rubbos/rubbos_tc1301.rst b/docs/testing/user/rubbos/rubbos_tc1301.rst
new file mode 100644
index 00000000..db4e0b63
--- /dev/null
+++ b/docs/testing/user/rubbos/rubbos_tc1301.rst
@@ -0,0 +1,30 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+************************************************
+Bottlenecks Rubbos Test Case Description TC1301
+************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks Rubbos TC1301 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_rubbos_tc1301 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | Rubbos platform for 1 Apache, 3 tomcat and 1 mysql. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/rubbos/testcase_cfg/rubbos_1-3-0-1.yaml |
+| | |
+| | client number: 5 |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/rubbos/rubbos_tc1401.rst b/docs/testing/user/rubbos/rubbos_tc1401.rst
new file mode 100644
index 00000000..f195d237
--- /dev/null
+++ b/docs/testing/user/rubbos/rubbos_tc1401.rst
@@ -0,0 +1,30 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+************************************************
+Bottlenecks Rubbos Test Case Description TC1401
+************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks Rubbos TC1401 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_rubbos_tc1401 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | Rubbos platform for 1 Apache, 4 tomcat and 1 mysql. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/rubbos/testcase_cfg/rubbos_1-4-0-1.yaml |
+| | |
+| | client number: 5 |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/vstf/vstf_Ti1.rst b/docs/testing/user/vstf/vstf_Ti1.rst
new file mode 100644
index 00000000..ce1690b4
--- /dev/null
+++ b/docs/testing/user/vstf/vstf_Ti1.rst
@@ -0,0 +1,28 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+*******************************************************************
+Bottlenecks vSwitch Test Framework(VSTF) Test Case Description Ti1
+*******************************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks VSTF Ti1 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_vstf_Ti1 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | vSwitch test Ti1. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/vstf/testcase_cfg/vstf_Ti1.yaml |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput & latency |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/vstf/vstf_Ti2.rst b/docs/testing/user/vstf/vstf_Ti2.rst
new file mode 100644
index 00000000..58cb40f7
--- /dev/null
+++ b/docs/testing/user/vstf/vstf_Ti2.rst
@@ -0,0 +1,28 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+*******************************************************************
+Bottlenecks vSwitch Test Framework(VSTF) Test Case Description Ti2
+*******************************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks VSTF Ti2 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_vstf_Ti2 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | vSwitch test Ti2. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/vstf/testcase_cfg/vstf_Ti2.yaml |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput & latency |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/vstf/vstf_Ti3.rst b/docs/testing/user/vstf/vstf_Ti3.rst
new file mode 100644
index 00000000..3bb9dfce
--- /dev/null
+++ b/docs/testing/user/vstf/vstf_Ti3.rst
@@ -0,0 +1,28 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+*******************************************************************
+Bottlenecks vSwitch Test Framework(VSTF) Test Case Description Ti3
+*******************************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks VSTF Ti3 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_vstf_Ti3 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | vSwitch test Ti3. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/vstf/testcase_cfg/vstf_Ti3.yaml |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput & latency |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/vstf/vstf_Tn1.rst b/docs/testing/user/vstf/vstf_Tn1.rst
new file mode 100644
index 00000000..5c591a9b
--- /dev/null
+++ b/docs/testing/user/vstf/vstf_Tn1.rst
@@ -0,0 +1,28 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+*******************************************************************
+Bottlenecks vSwitch Test Framework(VSTF) Test Case Description Tn1
+*******************************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks VSTF Tn1 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_vstf_Tn1 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | vSwitch test Tn1. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/vstf/testcase_cfg/vstf_Tn1.yaml |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput & latency |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/vstf/vstf_Tn2.rst b/docs/testing/user/vstf/vstf_Tn2.rst
new file mode 100644
index 00000000..4b6d0c45
--- /dev/null
+++ b/docs/testing/user/vstf/vstf_Tn2.rst
@@ -0,0 +1,28 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+*******************************************************************
+Bottlenecks vSwitch Test Framework(VSTF) Test Case Description Tn2
+*******************************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks VSTF Tn2 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_vstf_Tn2 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | vSwitch test Tn2. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/vstf/testcase_cfg/vstf_Tn2.yaml |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput & latency |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/vstf/vstf_Tu1.rst b/docs/testing/user/vstf/vstf_Tu1.rst
new file mode 100644
index 00000000..cf27bb25
--- /dev/null
+++ b/docs/testing/user/vstf/vstf_Tu1.rst
@@ -0,0 +1,28 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+*******************************************************************
+Bottlenecks vSwitch Test Framework(VSTF) Test Case Description Tu1
+*******************************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks VSTF Tu1 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_vstf_Tu1 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | vSwitch test Tu1. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/vstf/testcase_cfg/vstf_Tu1.yaml |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput & latency |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/vstf/vstf_Tu2.rst b/docs/testing/user/vstf/vstf_Tu2.rst
new file mode 100644
index 00000000..2ae84467
--- /dev/null
+++ b/docs/testing/user/vstf/vstf_Tu2.rst
@@ -0,0 +1,28 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+*******************************************************************
+Bottlenecks vSwitch Test Framework(VSTF) Test Case Description Tu2
+*******************************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks VSTF Tu2 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_vstf_Tu2 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | vSwitch test Tu2. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/vstf/testcase_cfg/vstf_Tu2.yaml |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput & latency |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/vstf/vstf_Tu3.rst b/docs/testing/user/vstf/vstf_Tu3.rst
new file mode 100644
index 00000000..4e090f17
--- /dev/null
+++ b/docs/testing/user/vstf/vstf_Tu3.rst
@@ -0,0 +1,28 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Tech and others.
+
+*******************************************************************
+Bottlenecks vSwitch Test Framework(VSTF) Test Case Description Tu3
+*******************************************************************
+
+
++-----------------------------------------------------------------------------+
+|Bottlenecks VSTF Tu3 |
+| |
++--------------+--------------------------------------------------------------+
+|test case name| opnfv_bottlenecks_vstf_Tu3 |
+| | |
++--------------+--------------------------------------------------------------+
+|description | vSwitch test Tu3. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | config file: |
+| | /testsuite/vstf/testcase_cfg/vstf_Tu3.yaml |
+| | |
++--------------+--------------------------------------------------------------+
+|test result | throughput & latency |
+| | |
++--------------+--------------------------------------------------------------+
+