Adds test results for scenario: fuel_os_onos_nofeature_ha

All scenario TCs included, except TC011 which has had an
InfluxDB population problem. This is now solved but too
late to run with scenario in R2 release time frame.

Change-Id: I341bd4109baf14069b6241f37da4c62da4cad394
Signed-off-by: Per Holmgren <per.q.holmgren@ericsson.com>

1 files changed, 116 insertions, 8 deletions

diff --git a/docs/results/fuel-os-onos-nofeature-ha.rst b/docs/results/fuel-os-onos-nofeature-ha.rst
index 5e94e9b6f..4e7345820 100644
--- a/docs/results/fuel-os-onos-nofeature-ha.rst
+++ b/docs/results/fuel-os-onos-nofeature-ha.rst
@@ -14,25 +14,133 @@ Test Results for fuel-os-onos-nofeature-ha
 Details
 =======
 
-.. after this doc is filled, remove all comments and include the scenario in
-.. results.rst by removing the comment on the file name.
-
+.. _Grafana: http://130.211.154.108/grafana/dashboard/db/yardstick-main
+.. _POD2: https://wiki.opnfv.org/pharos?&#community_test_labs
 
 Overview of test results
 ------------------------
 
-.. general on metrics collected, number of iterations
+See Grafana_ for viewing test result metrics for each respective test case. It
+is possible to chose which specific scenarios to look at, and then to zoom in
+on the details of each run test scenario as well.
+
+All of the test case results below are based on 7 scenario test
+runs, each run on the Ericsson POD2_ between February 13 and 21 in 2016. Test
+case TC011_ is not reported on due to an InfluxDB issue.
+The best would be to have more runs to draw better conclusions from, but these
+are the only runs available at the time of OPNFV R2 release.
+
+TC002
+-----
+The round-trip-time (RTT) between 2 VMs on different blades is measured using
+ping. The majority (5) of the test run measurements result in an average
+between 0.4 and 0.5 ms. The other 2 dates stick out with an RTT average of 0.9
+to 1 ms.
+The majority of the runs start with a 1 - 1.5 ms RTT spike (This could be
+because of normal ARP handling). One test run has a greater RTT spike of 4 ms,
+which is the same one with the 1 ms RTT average. The other runs have no similar
+spike at all. To be able to draw conclusions more runs should be made.
+SLA set to 10 ms. The SLA value is used as a reference, it has not
+been defined by OPNFV.
+
+TC005
+-----
+The IO read bandwidth looks similar between different dates, with an
+average between approx. 170 and 185 MB/s. Within each test run the results
+vary, with a minimum of 2 MB/s and maximum of 690MB/s on the totality. Most
+runs have a minimum BW of 3 MB/s (one run at 2 MB/s). The maximum BW varies
+more in absolute numbers between the dates, between 560 and 690 MB/s.
+SLA set to 400 MB/s. The SLA value is used as a reference, it has not been
+defined by OPNFV.
+
+TC010
+-----
+The measurements for memory latency are similar between test dates and result
+in a little less average than 1.22 ns. The variations within each test run are
+similar, between 1.213 and 1.226 ns. One exception is the first date, where the
+average is 1.223 and varies between 1.215 and 1.275 ns.
+SLA set to 30 ns. The SLA value is used as a reference, it has not been defined
+by OPNFV.
+
+TC011
+-----
+For this scenario no results are available to report on. Reason is an
+integer/floating point issue regarding how InfluxDB is populated with
+result data from the test runs. The issue was fixed but not in time to produce
+input for this report.
+
+TC012
+-----
+Between test dates the average measurements for memory bandwidth vary between
+17.1 and 18.1 GB/s. Within each test run the results vary more, with a minimal
+BW of 15.5 GB/s and maximum of 18.2 GB/s on the totality.
+SLA set to 15 GB/s. The SLA value is used as a reference, it has not been
+defined by OPNFV.
+
+TC014
+-----
+The Unixbench processor test run results vary between scores 3100 and 3260,
+one result each date. The average score on the total is 3170.
+No SLA set.
+
+TC037
+-----
+The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
+on different blades are measured when increasing the amount of UDP flows sent
+between the VMs using pktgen as packet generator tool.
+
+Round trip times and packet throughput between VMs can typically be affected by
+the amount of flows set up and result in higher RTT and less PPS throughput.
+
+There seems to be mainly two result types. One type a high and flatter
+PPS throughput not very much affected by the number of flows. Here also the
+average RTT is stable around 13 ms throughout all the test runs.
+
+The second type starts with a slightly lower PPS in the beginning than type
+one, and decreases even further when passing approx. 10000 flows. Here also the
+average RTT tends to start at approx. 15 ms ending with an average of 17 to 18
+ms with the maximum amount of flows running.
+
+Result type one can with the maximum amount of flows have a greater PPS than
+the second type with the minimum amount of flows.
+
+For result type one the average PPS throughput in the different runs varies
+between 399000 and 447000 PPS. The total amount of packets in each test run
+is between approx. 7000000 and 10200000 packets.
+The second result type has a PPS average of between 602000 and 621000 PPS and a
+total packet amount between 10900000 and 13500000 packets.
+
+There are lost packets reported in many of the test runs. There is no observed
+correlation between the amount of flows and the amount of lost packets.
+The lost amount of packets normally range between 100 and 1000 per test run,
+but there are spikes in the range of 10000 lost packets as well, and even
+more in a rare cases. Some case is in the range of one million lost packets.
 
 Detailed test results
 ---------------------
-
-.. info on lab, installer, scenario
+The scenario was run on Ericsson POD2_ with:
+Fuel 8.0
+OpenStack Liberty
+OpenVirtualSwitch 2.3.1
+OpenNetworkOperatingSystem Drake
 
 Rationale for decisions
 -----------------------
-.. result analysis, pass/fail
+Pass
+
+Tests were successfully executed and metrics collected.
+No SLA was verified. To be decided on in next release of OPNFV.
 
 Conclusions and recommendations
 -------------------------------
+The pktgen test configuration has a relatively large base effect on RTT in
+TC037 compared to TC002, where there is no background load at all. Approx.
+15 ms compared to approx. 0.5 ms, which is more than a 3000 percentage
+difference in RTT results.
+Especially RTT and throughput come out with better results than for instance
+the *fuel-os-nosdn-nofeature-ha* scenario does. The reason for this should
+probably be further analyzed and understood. Also of interest could be
+to make further analyzes to find patterns and reasons for lost traffic.
+Also of interest could be to see why there are variations in some test cases,
+especially visible in TC037.
 
-.. did the expected behavior occured?