aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/tests/unit/benchmark/runner
AgeCommit message (Collapse)AuthorFilesLines
2018-10-02NSB PROX NFVi Test does not stop after reaching expected precisionDanielMartinBuckley2-10/+10
JIRA: YARDSTICK-1419 When using prox_binsearch algorithm, a binary search is performed, increasing lower bound when step was successful and decreasing upper bound when the step was a failure. This runs until the test_precision (as specified in the traffic profile) is reached. When the test precision is reached, the test is not completed until the runner duration is reached. As runner duration is usually high (e.g. 1800sec), the tests take much too long to execute. This makes it difficult to create test suites. Change-Id: I6cc503a09fb534a556c61c805e6df4786bb8cc4b Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-09-04Add UT: ArithmeticRunner exception handlingMiikka Koistinen1-1/+227
* Use time.sleep to avoid multiprocessing.Queue broken pipe error. JIRA: YARDSTICK-1245 Change-Id: I1ef1f8e68a553278996b38c226314c13dd974faa Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-08-09Merge "Add UT: DurationRunner worker exception handling"Abhijit Sinha1-1/+131
2018-08-09Merge "Add UT: DurationRunner worker normal operation"Abhijit Sinha1-0/+146
2018-07-27Add UT: ArithmeticRunner worker normal operationMiikka Koistinen1-0/+182
* Use time.sleep to avoid multiprocessing.Queue broken pipe error. JIRA: YARDSTICK-1245 Change-Id: I0176c062f44830804ddbeebeb012afc1f9e20d41 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-07-27Add unit test file for ArithmeticRunnerMiikka Koistinen1-0/+38
* Initial test: test ArithmeticRunner._run_benchmark called arguments JIRA: YARDSTICK-1245 Change-Id: Id18c36bd1345052863ab777bf899c00fe66fd713 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-07-27Add UT: DurationRunner worker exception handlingMiikka Koistinen1-1/+131
* Use time.sleep to avoid multiprocessing.Queue broken pipe error. JIRA: YARDSTICK-1199 Change-Id: Icd35dad547bfa1ac4fab564c00628ef0d30ac338 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-07-27Add UT: DurationRunner worker normal operationMiikka Koistinen1-0/+146
* Use time.sleep to avoid multiprocessing.Queue broken pipe error. * Use a real duration in one of the test cases to test that the worker has successfully completed multiple iterations. JIRA: YARDSTICK-1199 Change-Id: I0e14682d9993b69e2aa9fd9340a5a7620e9c0d8e Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-07-05Make "IterationIPC" MQ producer for VNF control messagesRodolfo Alonso Hernandez1-0/+56
"IterationIPC" runner class is a consumer for MQ aware VNFs. A MQ aware traffic generator can send "started", "finished" and "iteration" messages. This feature implements a MQ producer in the runner in order to send messages to the VNFs. The messages implemented are: - "start_iteration" - "stop_iteration" JIRA: YARDSTICK-1286 Change-Id: I706f9a9dda5e5beed52231be7d71452945a7dbed Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-05Add "IterationIPC" runnerRodolfo Alonso Hernandez1-0/+136
"IterationIPC" is a runner that runs a configurable number of times before it returns. Each iteration has a configurable timeout. The loop control depends on the feedback received from the running VNFs. The context PIDs from the VNFs to listen the messages from are given in the scenario "setup" method. The MQ consumer, "RunnerIterationIPCConsumer", is subscribed to messages sent to topic "messaging.TOPIC_TG". The endpoints are defined in the class "RunnerIterationIPCEndpoint". Three are implemented: "messaging.TG_METHOD_STARTED", "messaging.TG_METHOD_FINISHED" and "messaging.TG_METHOD_ITERATION". JIRA: YARDSTICK-1127 Change-Id: I4b2f11a15ef41e6f3c70bd64188d5c7fbcdb7eed Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-04Cleanup ActionTestCase unit testsRodolfo Alonso Hernandez1-19/+23
Removed unneeded output and applied the correct mocks. JIRA: YARDSTICK-1290 Change-Id: Ib8dbf582d81f7de0fadce7ddb215360abb2d16a4 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-28Cleanup ProxDuration unit testsRodolfo Alonso Hernandez1-142/+133
Removed unneeded output, removed access to files, fixed indentation, bumped deprecated asserts, reduced time execution and fixed file header. JIRA: YARDSTICK-1262 Change-Id: I4803980748d4731cc27608e386707a858e1a8630 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-26Merge "Addition of Configurable Sampling and Configurable Confirmation Retry"Abhijit Sinha1-0/+295
2018-06-26Addition of Configurable Sampling and Configurable Confirmation RetryDanielMartinBuckley1-0/+295
JIRA: YARDSTICK-1221 In order to increase accuracy of result the following are done :- - Improve Measurement Accuracy (YARDSTICK-1212) - Improve Sampling Interval (YARDSTICK-1219) - Allow 4 PROX ports to be read simultaneously (YARDSTICK-1220) This change does the following :- - Stores LINE Rate statistics of Sample - Requires a confirmation retry before deciding to increase or decrease this is configurable. - Allows the user to disable Sampling or specify a sample interval - Added Code Coverage of ProxDurationRunner based on YARDSTICK-1199 Change-Id: I27242ac1849c9a2712866385b5fbc05977c71516 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-25Add unit test file for DurationRunnerMiikka Koistinen1-0/+39
JIRA: YARDSTICK-1199 Change-Id: I5526935207ae3661a4cb8b8d25e93bc26adc3ef2 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Replace MagicMocks with Mocks from SearchRunner UTsMiikka Koistinen1-27/+16
This patch replaces unneeded MagicMocks with Mocks from TestSearchRunner and TestSearchRunnerHelper. JIRA: YARDSTICK-1169 Change-Id: I30a7bbc71c0b1062e6c0dfe634a5a8f1ecb7c684 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Convert SLA asserts to raisesMiikka Koistinen1-3/+4
This commit converts Python assertions to a custom exception in all places where SLA validation is checked with an assertion. This commit also fixes all emerged pylint errors. JIRA: YARDSTICK-966 Change-Id: If771ed03b2cbc0a43a57fcfb9293f18740b3ff80 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-13assert[Greater,Equal] -> assert_{,not_}calledEmma Foley1-2/+2
assertEqual(mock_xxx.call_count, 1) -> mock_xxx.assert_called_once assertEqual(mock_xxx.call_count, 0) -> mock_xxx.assert_not_called assertGreater(mock.call_count, 0) -> mock.assert_called() assertGreaterEqual(mock.call_count, 1) -> mock.assert_called() JIRA: YARDSTICK-1069 Change-Id: I890084d120c8e78304e169e2a0e5d30011a41525 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-03-02Remove main() and __main__ from tests.Emma Foley1-8/+0
Removes the unnecessary main() functions from tests. Also removes shebang (#!) where it appears at the top of files. JIRA: YARDSTICK-861 Change-Id: I79180d1eb9c5bce640142dd62ce28c7437c1b210 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-02-28Mock TRex STL libraries globallyRodolfo Alonso Hernandez1-9/+2
TRex Python libraries are not going to be available during unit testing execution. Those modules calling TRex libraries should be mocked individually. This patch will remove the need of mocking those libraries per module. JIRA: YARDSTICK-1010 Change-Id: I4aa11d43ecf32a3dad78f869541b4afea4ec1d28 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-02-23Yardstick real-time influxdb KPI reporting regressionschenjiankun1-4/+13
JIRA: YARDSTICK-989 We used to have real-time influxdb reporting of test KPIs. The user could monitor using grafana and see the real-time output. The record format was changed to now only report KPIs at the end of the test. This is a problem for test cases which run for a long duration, we need to wait until the end of the test execution to get any results from influxdb. If the test fails in between or doesn't exit cleanly for some reason, we are left with no results stored in influxdb which gives the user no information. This is also a regression from the previous behaviour. Change-Id: I0f476dff9162a359f0286fb421f2e9c4befaa5cc Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2018-02-12Move tests: unit/benchmarkEmma Foley3-0/+300
* Fix pylint errors * Add TODOs Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged with a TODO, and should be fixed later. Change-Id: I30eb4b0aafe0575d0cddbc946108291f21a98ed8 Jira: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>