Adds test scenario result report: fuel-os-odl_l2-nofeature-ha

Change-Id: I2574411fe5ac670851ea9d1be5ba5f2da5defb17
Signed-off-by: Per Holmgren <per.q.holmgren@ericsson.com>

1 files changed, 127 insertions, 9 deletions

diff --git a/docs/results/fuel-os-odl_l2-nofeature-ha.rst b/docs/results/fuel-os-odl_l2-nofeature-ha.rst
index 88f701de5..914781684 100644
--- a/docs/results/fuel-os-odl_l2-nofeature-ha.rst
+++ b/docs/results/fuel-os-odl_l2-nofeature-ha.rst
@@ -14,25 +14,143 @@ Test Results for fuel-os-odl_l2-nofeature-ha
 Details
 =======
 
-.. after this doc is filled, remove all comments and include the scenario in
-.. results.rst by removing the comment on the file name.
-
+.. _Grafana: http://130.211.154.108/grafana/dashboard/db/yardstick-main
+.. _POD2: https://wiki.opnfv.org/pharos?&#community_test_labs
 
 Overview of test results
 ------------------------
 
-.. general on metrics collected, number of iterations
+See Grafana_ for viewing test result metrics for each respective test case. It
+is possible to chose which specific scenarios to look at, and then to zoom in
+on the details of each run test scenario as well.
+
+All of the test case results below are based on 6 scenario test
+runs, each run on the Ericsson POD2_ between February 13 and 24 in 2016. Test
+case TC011_ is the greater exception for which there are only 2 test runs
+available, due to earlier problems with InfluxDB test result population.
+The best would be to have more runs to draw better conclusions from, but these
+are the only runs available at the time of OPNFV R2 release.
+
+TC002
+-----
+The round-trip-time (RTT) between 2 VMs on different blades is measured using
+ping. Most test run measurements result on average between 0.3 and 0.5 ms, but
+one date (Feb. 23) sticks out with an RTT average of 1 ms.
+A few runs start with a 1 - 2 ms RTT spike (This could be because of normal ARP
+handling). One test run has a greater RTT spike of 3.9 ms, which is the same
+one with the 0.9 ms average. The other runs have no similar spike at all.
+To be able to draw conclusions more runs should be made.
+SLA set to 10 ms. The SLA value is used as a reference, it has not
+been defined by OPNFV.
+
+TC005
+-----
+The IO read bandwidth looks similar between different dates, with an
+average between approx. 165 and 185 MB/s. Within each test run the results
+vary, with a minimum 2 MB/s and maximum 617 MB/s on the totality. Most runs
+have a minimum BW of 3 MB/s (two runs at 2 MB/s). The maximum BW varies more in
+absolute numbers between the dates, between 566 and 617 MB/s.
+SLA set to 400 MB/s. The SLA value is used as a reference, it has not been
+defined by OPNFV.
+
+TC010
+-----
+The measurements for memory latency are similar between test dates and result
+in approx. 1.2 ns. The variations within each test run are similar, between
+1.215 and 1.219 ns. One exception is February 16, where the average is 1.222
+and varies between 1.22 and 1.28 ns.
+SLA set to 30 ns. The SLA value is used as a reference, it has not been defined
+by OPNFV.
+
+TC011
+-----
+Only 2 test runs are available to report results on.
+
+Packet delay variation between 2 VMs on different blades is measured using
+Iperf3. On the first date the reported packet delay variation varies between
+0.0025 and 0.011 ms, with an average delay variation of 0.0067 ms.
+On the second date the delay variation varies between 0.002 and 0.006 ms, with
+an average delay variation of 0.004 ms.
+
+TC012
+-----
+Results are reported for 5 test runs. It is not known why the 6:th test run
+is missing.
+Between test dates the average measurements for memory bandwidth vary between
+17.4 and 17.9 GB/s. Within each test run the results vary more, with a minimal
+BW of 16.4 GB/s and maximum of 18.2 GB/s on the totality.
+SLA set to 15 GB/s. The SLA value is used as a reference, it has not been
+defined by OPNFV.
+
+TC014
+-----
+Results are reported for 5 test runs. It is not known why the 6:th test run
+is missing.
+The Unixbench processor test run results vary between scores 3080 and 3240,
+one result each date. The average score on the total is 3150.
+No SLA set.
+
+TC037
+-----
+Results are reported for 5 test runs. It is not currently known why the 6:th
+test run is missing.
+The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
+on different blades are measured when increasing the amount of UDP flows sent
+between the VMs using pktgen as packet generator tool.
+
+Round trip times and packet throughput between VMs can typically be affected by
+the amount of flows set up and result in higher RTT and less PPS throughput.
+
+The RTT results are similar throughout the different test dates and runs at
+approx. 15 ms. Some test runs show an increase with many flows, in the range
+towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
+RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
+RTT results.
+In some test runs when running with less than approx. 10000 flows the PPS
+throughput is normally flatter compared to when running with more flows, after
+which the PPS throughput decreases. Around 20 percent decrease in the worst
+case. For the other test runs there is however no significant change to the PPS
+throughput when the number of flows are increased. In some test runs the PPS
+is also greater with 1000000 flows compared to other test runs where the PPS
+result is less with only 2 flows.
+
+The average PPS throughput in the different runs varies between 414000 and
+452000 PPS. The total amount of packets in each test run is approx. 7500000 to
+8200000 packets. One test run Feb. 15 sticks out with a PPS average of
+558000 and approx. 1100000 packets in total (same as the on mentioned earlier
+for RTT results).
+
+There are lost packets reported in most of the test runs. There is no observed
+correlation between the amount of flows and the amount of lost packets.
+The lost amount of packets normally range between 100 and 1000 per test run,
+but there are spikes in the range of 10000 lost packets as well, and even
+more in a rare cases.
 
 Detailed test results
 ---------------------
-
-.. info on lab, installer, scenario
+The scenario was run on Ericsson POD2_ with:
+Fuel 8.0
+OpenStack Liberty
+OpenVirtualSwitch 2.3.1
+OpenDayLight Beryllium
 
 Rationale for decisions
 -----------------------
-.. result analysis, pass/fail
+Pass
+
+Tests were successfully executed and metrics collected.
+No SLA was verified. To be decided on in next release of OPNFV.
 
 Conclusions and recommendations
 -------------------------------
-
-.. did the expected behavior occured?
+The pktgen test configuration has a relatively large base effect on RTT in
+TC037 compared to TC002, where there is no background load at all. Approx.
+15 ms compared to approx. 0.5 ms, which is more than a 3000 percentage
+difference in RTT results.
+Especially RTT and throughput come out with better results than for instance
+the *fuel-os-nosdn-nofeature-ha* scenario does. The reason for this should
+probably be further analyzed and understood. Also of interest could be
+to make further analyzes to find patterns and reasons for lost traffic.
+Also of interest could be to see if there are continuous variations where
+some test cases stand out with better or worse results than the general test
+case.