summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorPer Holmgren <per.q.holmgren@ericsson.com>2015-10-27 15:34:13 +0100
committerPer Holmgren <per.q.holmgren@ericsson.com>2015-12-09 15:35:44 +0100
commita53e33844e5236d80ee178ca18b7c352eec37286 (patch)
treeea283d45fe72315b3575f0896e66448eabe91dc3
parent70c27f24d07411103fb63090d17390f5ea249ebc (diff)
Adds new test case descriptions
Adds test case descriptions for JIRA issues number 6 (Network Performance) and number 8 (Network Latency) in accordance to version 2 of the test case description template. Jira:YARDSTICK-6 Jira:YARDSTICK-8 Signed-off-by: Per Holmgren <per.q.holmgren@ericsson.com> Change-Id: Ib6f4d08a1f7c6ba10f3f220d2bb3998311ecf203
-rw-r--r--docs/yardstick/opnfv_yardstick_tc001.rst71
-rw-r--r--docs/yardstick/opnfv_yardstick_tc002.rst71
2 files changed, 142 insertions, 0 deletions
diff --git a/docs/yardstick/opnfv_yardstick_tc001.rst b/docs/yardstick/opnfv_yardstick_tc001.rst
new file mode 100644
index 000000000..72ba68e9f
--- /dev/null
+++ b/docs/yardstick/opnfv_yardstick_tc001.rst
@@ -0,0 +1,71 @@
+.. image:: ../../etc/opnfv-logo.png
+ :height: 40
+ :width: 200
+ :alt: OPNFV
+ :align: left
+
+*************************************
+Yardstick Test Case Description TC001
+*************************************
++-----------------------------------------------------------------------------+
+|Network Performance |
++==============+==============================================================+
+|test case id | OPNFV_YARDSTICK_TC001_NW PERF |
++--------------+--------------------------------------------------------------+
+|metric | Number of flows and throughput |
++--------------+--------------------------------------------------------------+
+|test purpose | To evaluate the IaaS network performance with regards to |
+| | flows and throughput, such as if and how different amounts |
+| | of flows matter for the throughput between hosts on different|
+| | compute blades. Typically e.g. the performance of a vSwitch |
+| | depends on the number of flows running through it. Also |
+| | performance of other equipment or entities can depend |
+| | on the number of flows or the packet sizes used. |
+| | The purpose is also to be able to spot trends. Test results, |
+| | graphs ans similar shall be stored for comparison reasons and|
+| | product evolution understanding between different OPNFV |
+| | versions and/or configurations. |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc001.yaml |
+| | |
+| | Packet size: 60 bytes |
+| | Number of ports: 10, 50, 100, 500 and 1000, where each |
+| | runs for 20 seconds. The whole sequence is run |
+| | twice. The client and server are distributed on different HW.|
+| | For SLA max_ppm is set to 1000. The amount of configured |
+| | ports map to between 110 up to 1001000 flows, respectively. |
++--------------+--------------------------------------------------------------+
+|test tool | pktgen |
+| | |
+| | (Pktgen is not always part of a Linux distribution, hence it |
+| | needs to be installed. It is part of the Yardstick Docker |
+| | image. |
+| | As an example see the /yardstick/tools/ directory for how |
+| | to generate a Linux image with pktgen included.) |
++--------------+--------------------------------------------------------------+
+|references |https://www.kernel.org/doc/Documentation/networking/pktgen.txt|
+| | |
+| |ETSI-NFV-TST001 |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different packet sizes, amount |
+| | of flows and test duration. Default values exist. |
+| | |
+| |SLA (optional): |
+| | max_ppm: The number of packets per million packets sent |
+| | that are acceptable to lose, i.e. not received. |
++--------------+--------------------------------------------------------------+
+|pre-test | The test case image needs to be installed into Glance |
+|conditions | with pktgen included in it. |
+| | |
+| | No POD specific requirements have been identified. |
++--------------+------+----------------------------------+--------------------+
+|test sequence | step | description | result |
+| +------+----------------------------------+--------------------+
+| | 1 | The hosts are installed, as | Logs are stored |
+| | | server and client. pktgen is | |
+| | | invoked and logs are produced | |
+| | | and stored. | |
++--------------+------+----------------------------------+--------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
++--------------+--------------------------------------------------------------+ \ No newline at end of file
diff --git a/docs/yardstick/opnfv_yardstick_tc002.rst b/docs/yardstick/opnfv_yardstick_tc002.rst
new file mode 100644
index 000000000..fb1b4694f
--- /dev/null
+++ b/docs/yardstick/opnfv_yardstick_tc002.rst
@@ -0,0 +1,71 @@
+.. image:: ../../etc/opnfv-logo.png
+ :height: 40
+ :width: 200
+ :alt: OPNFV
+ :align: left
+
+*************************************
+Yardstick Test Case Description TC002
+*************************************
+
++-----------------------------------------------------------------------------+
+|Network Latency |
++==============+==============================================================+
+|test case id | OPNFV_YARDSTICK_TC002_NW LATENCY |
++--------------+--------------------------------------------------------------+
+|metric | RTT, Round Trip Time |
++--------------+--------------------------------------------------------------+
+|test purpose | To do a basic verification that network latency is within |
+| | acceptable boundaries when packets travel between hosts |
+| | located on same or different compute blades. |
+| | The purpose is also to be able to spot trends. Test results, |
+| | graphs and similar shall be stored for comparison reasons and|
+| | product evolution understanding between different OPNFV |
+| | versions and/or configurations. |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc002.yaml |
+| | |
+| | Packet size 100 bytes. Total test duration 600 seconds. |
+| | One ping each 10 seconds. SLA RTT is set to maximum 10 ms. |
++--------------+--------------------------------------------------------------+
+|test tool | ping |
+| | |
+| | Ping is normally part of any Linux distribution, hence it |
+| | doesn't need to be installed. It is also part of the |
+| | Yardstick Docker image. |
+| | (For example also a Cirros image can be downloaded from |
+| | https://download.cirros-cloud.net, it includes ping) |
++--------------+--------------------------------------------------------------+
+|references | Ping man page |
+| | |
+| | ETSI-NFV-TST001 |
++--------------+--------------------------------------------------------------+
+|applicability | Test case can be configured with different packet sizes, |
+| | burst sizes, ping intervals and test duration. |
+| | SLA is optional. The SLA in this test case serves as an |
+| | example. Considerably lower RTT is expected, and |
+| | also normal to achieve in balanced L2 environments. However, |
+| | to cover most configurations, both bare metal and fully |
+| | virtualized ones, this value should be possible to achieve |
+| | and acceptable for black box testing. Many real time |
+| | applications start to suffer badly if the RTT time is higher |
+| | than this. Some may suffer bad also close to this RTT, while |
+| | others may not suffer at all. It is a compromise that may |
+| | have to be tuned for different configuration purposes. |
++--------------+--------------------------------------------------------------+
+|pre-test | The test case image needs to be installed into Glance |
+|conditions | with ping included in it. |
+| | |
+| | No POD specific requirements have been identified. |
++--------------+------+----------------------------------+--------------------+
+|test sequence | step | description | result |
+| +------+----------------------------------+--------------------+
+| | 1 | The hosts are installed, as | Logs are stored |
+| | | server and client. Ping is | |
+| | | invoked and logs are produced | |
+| | | and stored. | |
++--------------+------+----------------------------------+--------------------+
+|test verdict | Test should not PASS if any RTT is above the optional SLA |
+| | value, or if there is a test case execution problem. |
++--------------+--------------------------------------------------------------+
+