aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick
AgeCommit message (Collapse)AuthorFilesLines
2018-07-02Merge "Cleanup TestProxBinSearchProfile unit tests"Rodolfo Alonso Hernandez1-13/+13
2018-07-02Merge "Cleanup ProxDuration unit tests"Rodolfo Alonso Hernandez1-142/+133
2018-06-29Cleanup TestProxBinSearchProfile unit testsRodolfo Alonso Hernandez1-13/+13
Removed unneeded output and mocks. JIRA: YARDSTICK-1266 Change-Id: I04fd1ba32ea46c7896548faebc5af1a43b0f750e Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-29NFVI support for standalone, baremetal and heat contextsChornyi, TarasX26-151/+474
JIRA: YARDSTICK-1257 Change-Id: I6733bd49ac91985e8f3d7722e6990e8733bb430e Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
2018-06-29Merge "Removed unused ACL config file."Rodolfo Alonso Hernandez1-47/+0
2018-06-28Merge "Cleanup process test cases"Rodolfo Alonso Hernandez1-4/+6
2018-06-28Merge "Cleanup OpenStack utils test cases"Rodolfo Alonso Hernandez1-32/+35
2018-06-28Decrease Sampling interval - Invalid Port resultsDanielMartinBuckley1-4/+6
JIRA: YARDSTICK-1219 The computed in/fwd throughputs displayed on the L2FWD, L3FWD and VPE dashboards. The computed value is likely 4x the expected value. It is required to return sampling information form Generator AND VNF at least every 1 second. Change-Id: I4435fd05ba3116ead836843a4c2fce133b767a28 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-28Cleanup ProxDuration unit testsRodolfo Alonso Hernandez1-142/+133
Removed unneeded output, removed access to files, fixed indentation, bumped deprecated asserts, reduced time execution and fixed file header. JIRA: YARDSTICK-1262 Change-Id: I4803980748d4731cc27608e386707a858e1a8630 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-27Removed unused ACL config file.Mytnyk, Volodymyr1-47/+0
Change-Id: Ibdb1dd12587df0182b5e8d3b7bc6901be0aac3ec Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-06-27Update to Grafana Dashboard for NSB PROX NFVIDanielMartinBuckley1-0/+2
JIRA: YARDSTICK-1252 CPU grouping in grafana for better representation Throughput table is difficult to read as the tests run longer the values can overlap Default time in dashboard should be always last 1hour in dashboard (auto refresh can be also checked) Add one column in table with Gbps throughput (Rx Throughput) plus code change in yardstick Boxes are not aligned and big and small, need to be fixed boxes are now aligned, Added Seperate graphe for LINE Rate Change-Id: Ief31777c2e7a18aab4bf21fb6ccc0e93a3bb9178 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-27Fix NSB NfVi support for 25 and 40GbpsXavier Simonart1-4/+27
When setting interface_speed option in the test case, NSB/PROX was still considering 100% as 10 Gbps. With this fix, when interface_speed is set to for instance 25 Gbps then NSB/PROX will start generate at 25 Gbps. Yardstick logging info printed on the screen will show performance result as percentage of interface_speed (e.g. 100% will be 25 Gbps). This is done by sending the command "speed 250" to PROX (PROX cannot know the underlying interface speed in all cases e.g. in virtio, and hence considers 100% as 10Gbps). JIRA: YARDSTICK-1217 Change-Id: I4989161b04a7afa5e347935ee7ccef7dfe5a504d Signed-off-by: Xavier Simonart <xavier.simonart@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-27NSB NFVi PROX VPE losing many packetsDanielMartinBuckley1-0/+4
JIRA: YARDSTICK-1105 Running a vPE test ends up with up to 25% drooped packets. This is due to tolerated loss being incorrectly configured, test duration was too short and test interval was too short. Note This is due to the fact that vPE use case have by default a tolerated loss of 100%, Now changed to 0.001, same as L2FWD Also: Grafan dashboard has been updated to show all SUT CPUS Utilization in 1 panel. And LINE rate MIN, MAX & TEST Rate have been added Change-Id: I7adae2199b3f656fe460705b6aeb3aa69c767d50 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-27Merge "Define several containers per pod in a replication controller"Rodolfo Alonso Hernandez2-50/+102
2018-06-27Merge "Decrease Sampling interval"Abhijit Sinha4-39/+123
2018-06-26NSB NFVi BNG test fails to run - stops after one stepDanielMartinBuckley2-4/+17
JIRA: YARDSTICK-1244 When sending the start or stop command to PROX. We sould not send the same core. We should not send "start 1,1" instead we should send "start 1" Change-Id: Ie600dc3ed808fd00341e92f13bd52199e68dc51f Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-26Decrease Sampling intervalDanielMartinBuckley5-39/+124
JIRA: YARDSTICK-1219 Currently NSB for NFVI using PROX returns sampling information every 13 seconds. This is too slow. It is required to return sampling information form Generator AND VNF at least every 1 second. This change is depandant on JIRA: YARDSTICK-1212 and YARDSTICK-1220 Change-Id: Ica7ab795a2919d191d8cd846d028e15739e33fb7 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-26Merge "Addition of Configurable Sampling and Configurable Confirmation Retry"Abhijit Sinha6-43/+663
2018-06-26Merge "Extended Context class with get_physical_nodes functionality"Rodolfo Alonso Hernandez20-70/+464
2018-06-26Addition of Configurable Sampling and Configurable Confirmation RetryDanielMartinBuckley6-43/+663
JIRA: YARDSTICK-1221 In order to increase accuracy of result the following are done :- - Improve Measurement Accuracy (YARDSTICK-1212) - Improve Sampling Interval (YARDSTICK-1219) - Allow 4 PROX ports to be read simultaneously (YARDSTICK-1220) This change does the following :- - Stores LINE Rate statistics of Sample - Requires a confirmation retry before deciding to increase or decrease this is configurable. - Allows the user to disable Sampling or specify a sample interval - Added Code Coverage of ProxDurationRunner based on YARDSTICK-1199 Change-Id: I27242ac1849c9a2712866385b5fbc05977c71516 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-26Merge "Bugfix: HA kill process recovery has a conflict"Abhijit Sinha2-8/+33
2018-06-26Merge "Refactor "utils.parse_ini_file" testing"Rodolfo Alonso Hernandez1-17/+93
2018-06-26Merge "Refactor remote command execution in pktgen_dpdk"Emma Foley2-5/+18
2018-06-26Extended Context class with get_physical_nodes functionalityChornyi, TarasX20-70/+464
JIRA: YARDSTICK-1255 Change-Id: I446d715dc6cf716a4fcbc1b809c1b1d6303071e0 Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
2018-06-26Merge "Add unit test file for DurationRunner"Rodolfo Alonso Hernandez1-0/+39
2018-06-26Merge "Refactor remote command execution in pktgen"Rex Lee4-133/+107
2018-06-26Bugfix: HA kill process recovery has a conflictrexlee87762-8/+33
It happens in Nokia SUT when running in the Plugfest. The problem happens when the start_process start to recover the killed process (like nova-api), but the self-cured mechanism already recovered. And somehow it lead to a conflict and has problems. So the recover of the HA attack-recover should be improved to only recover it when it needs to. JIRA: YARDSTICK-1222 Change-Id: I1acb5a7d59d6fe4e0de0b0c5942fa89e051dd1ff Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-06-25Add unit test file for DurationRunnerMiikka Koistinen1-0/+39
JIRA: YARDSTICK-1199 Change-Id: I5526935207ae3661a4cb8b8d25e93bc26adc3ef2 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-25Merge "Cleanup CGNAPT unit tests"Rodolfo Alonso Hernandez1-123/+70
2018-06-25Merge "Add "volumeMounts" parameter in Kubernetes context"Rodolfo Alonso Hernandez2-28/+89
2018-06-25Merge "Add "volumes" parameter in Kubernetes context"Rodolfo Alonso Hernandez6-40/+118
2018-06-25Change IxNextgen library loadRodolfo Alonso Hernandez2-23/+8
Changed IxNextgen library load in IXIA RFC2544 traffic generator call. JIRA: YARDSTICK-1116 Change-Id: Icbdc5f1255dd2676584f3a1be767287e55e71f4a Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-22Merge "Configure ACL via static file"Abhijit Sinha14-436/+403
2018-06-21Configure ACL via static fileMytnyk, Volodymyr14-436/+403
This patch allows user to configure ACL/vFW SampleVNF ACL via configuration file provided in TC definition. The Yardstick applies the rules to SampleVNF from specified config file + rules generated by Yardstick (default rules). The example of SampleVNF ACL CLI commands generated/applied by Yardstick can be found at (using default ACL config file): ACL VNF ACL CLI commands: acl/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml http://paste.openstack.org/show/723303/ vFW VNF ACL CLI commands: vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml http://paste.openstack.org/show/723304/ Change-Id: I76a630261a982083b628e3985fc3bec14ca495db Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-06-21Merge "Cleanup EnvCommand test cases"Rodolfo Alonso Hernandez1-33/+37
2018-06-19Refactor remote command execution in pktgen_dpdkMiikka Koistinen2-5/+18
PktgenDPDKLatency.run will raise a RuntimeError if a remote command fails. This commit makes it use the already existing exception raising mechanism in yardstick's ssh client. JIRA: YARDSTICK-1166 Change-Id: I3a3c7691399044b174f5d040c015c9b907b2fe5d Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-19Refactor remote command execution in pktgenMiikka Koistinen4-133/+107
Some remote commands in Pktgen are executed without checking the exit status. This patch makes all remote commands check exit value, and removes unused variables that are captured from remote command executions. JIRA: YARDSTICK-1166 Change-Id: I42a667ebd22d086887d61e1671bc569b03c59d33 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-18Cleanup EnvCommand test casesRodolfo Alonso Hernandez1-33/+37
Refactor exception testing and remove console output. JIRA: YARDSTICK-1235 Change-Id: I6376c3637ffb6476a2578ef963cd2420b63bab52 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-18Refactor RFC2455 TRex traffic profile injectionRodolfo Alonso Hernandez17-1042/+855
The TRex RFC2455 traffic profile configures the duration of the traffic injection. Once the traffic is started, the traffic profile should poll the traffic injector client for the injection status. When the injection is finished, the traffic execution should end. Instead of this, the traffic profile waits a fixed time (using time.sleep method) and then stops the traffic. This approach is not accurate and may cause shorter injection periods. JIRA: YARDSTICK-1197 Change-Id: I5090df81ef4ec7945ff6c1aff070656b48e0fe77 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-18Add "duration" parameter to test case definitionRodolfo Alonso Hernandez7-92/+107
Add "duration" parameter to test case definition, in scenario:options section. This parameter will be rendered in the traffic profile. If the parameter is not present in the test case scenario options, the default time written in the traffic profile options will be 30 seconds (TrafficProfile.DEFAULT_DURATION = 30). If the traffic profile injection time is not defined, the default injection time will be 30 seconds. testcase:scenario:options:duration (default = 30) render --> traffic_profile:duration parse --> TrafficProfile.duration (default = 30) Target traffic profiles (RFC2544): - RFC2544Profile - IXIARFC2544Profile JIRA: YARDSTICK-1194 Change-Id: I968922e6bb882d7ee15aa1c4db4037face7a3492 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-18Merge changes from topic 'convert-sla-asserts-to-raises'Abhijit Sinha3-87/+200
* changes: Replace MagicMocks with Mocks from SearchRunner UTs Add vsperf_dpdk SLA validation unit test cases Cleanup vsperf unit tests
2018-06-18Merge "Convert SLA asserts to raises"Abhijit Sinha46-156/+220
2018-06-18Stop running tests from tests/unitEmma Foley2-3/+3
Since all tests have been moved into yardstick/tests, temperary code to run tests in both tests and yardstick/tests should be removed. JIRA: YARDSTICK-837 Change-Id: I3bf3c9ece9bc8457d992e53ee8cc5cadb1dec8f0 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-18Merge "Cleanup BaseMonitor unit tests"Rodolfo Alonso Hernandez1-7/+11
2018-06-18Cleanup BaseMonitor unit testsRodolfo Alonso Hernandez1-7/+11
Stop multiprocessing queues to avoid BrokenPipe exception. Assert correctly the raise of exceptions. JIRA: YARDSTICK-1229 Change-Id: I1a1c92c49a439283a35c9a69b59d6866f35ec09a Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-15Replace MagicMocks with Mocks from SearchRunner UTsMiikka Koistinen1-27/+16
This patch replaces unneeded MagicMocks with Mocks from TestSearchRunner and TestSearchRunnerHelper. JIRA: YARDSTICK-1169 Change-Id: I30a7bbc71c0b1062e6c0dfe634a5a8f1ecb7c684 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Convert SLA asserts to raisesMiikka Koistinen46-156/+220
This commit converts Python assertions to a custom exception in all places where SLA validation is checked with an assertion. This commit also fixes all emerged pylint errors. JIRA: YARDSTICK-966 Change-Id: If771ed03b2cbc0a43a57fcfb9293f18740b3ff80 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Add vsperf_dpdk SLA validation unit test casesMiikka Koistinen1-0/+45
JIRA: YARDSTICK-1228 Change-Id: Ic32f2bcd7f8bdf718c9c266666409d32ecab4924 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Cleanup vsperf unit testsMiikka Koistinen1-60/+139
Remove unneeded mock return values and improve the test execution. Add unit test cases about SLA validation. JIRA: YARDSTICK-1227 Change-Id: I90b8b9c4cb6a57ae4b0869e84af0da43ee22a670 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Merge "assert[Greater,Equal] -> assert_{,not_}called"Rodolfo Alonso Hernandez10-51/+52