aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2019-12-10Add energy base scenario and test filessungn2-0/+139
Folder of energy contains scripts of energy measurement. File __init__.py make it a module that can be imported. File energy.py contains the main function of energy measurement through redfish API. File test_energy.py test function of energy.py File energy_sample_chassis_output.txt provide a sample output of chassis query. File energy_sample_power_metrics.txt provide sample metrics of power query. This will allow users to collect data of energy consumption of baremetal machines. JIRA: YARDSTICK-1624 Co-authored by: Roland Picard <roland.picard@orange.com> Signed-off-by: sungn <sungn2@lenovo.com> Change-Id: Ied86894f3dcc2536f0cb1a044f4591351e5b3270
2019-10-08Fix output parsing of UnixbenchDeepanshu Bhatia1-2/+2
Pick the last word in the awk as score for handling partial index score and index score Change-Id: Ida56b704b14c7c27156f57590bb1ea8532650f90 JIRA: YARDSTICK-1622 Signed-off-by: Deepanshu Bhatia <deepanshu@voereir.com>
2019-05-13Ping: Fix TypeError without SLA in timeout scenarioDeepanshu Bhatia1-6/+8
In case of no SLA association with scenario, when the commands timeout, stdout is empty and it gives TypeError. Reverted the condition to the old one which was used before introduction of verifySLA() JIRA: YARDSTICK-1618 Change-Id: Ibd86ba7168f26ae5b15c21fd6129cc737a7038db Signed-off-by: Deepanshu Bhatia <deepanshu@voereir.com>
2019-04-26Merge changes from topic 'feat/baro_nfvi_metrics'Volodymyr Mytnyk1-37/+156
* changes: Use baro and yardstick metrics in dynamic HTML report benchmark.core.report: Add _combine_times benchmark.core.report: Add _get_baro_metrics Add ability to get data from different DBs in influx Refactor: add _format_datasets
2019-04-19Log VM OS version, Sample VNF branch/commit IDStepan Andrushko1-0/+11
Added debug logs to track VM, Sample VNF details during testing: - Virtual machine OS, kernel version; - Sample VNF branch, commit ID. JIRA: YARDSTICK-1499 Change-Id: I243c435809d4541dfdb8c7c3466f50c5d524ac00 Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
2019-04-08Merge "Update to support using external heat template"Volodymyr Mytnyk2-1/+27
2019-04-04Update to support using external heat templatetreyad2-1/+27
Modify the _get_server() function when using external heat template. The server attrs should be read from stack.outputs. JIRA: YARDSTICK-1453 Change-Id: I00eda7476e77fc0a5c15ef8db7d057fa1cf267e6 Signed-off-by: treyad <treyad@viosoft.com>
2019-04-03Use baro and yardstick metrics in dynamic HTML reportEmma Foley1-6/+24
_combine_metrics combines metrics from different sources. This is for use with the ``yardstick report generate-nsb`` command, which will combine yardstick and barometer metrics in the dynamic HTML report. JIRA: YARDSTICK-1593 Change-Id: I87002948ebb4cc88fb0932380bcb9920eb53db58 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2019-03-07Merge "Support using existing private key when using external heat template ↵Volodymyr Mytnyk1-16/+43
file"
2019-02-28Support using existing private key when using external heat template filetreyad1-16/+43
Currently, the private key is auto-generated from Yardstick. This patch will allow user use their existing private key when they use external heat template file. JIRA: YARDSTICK-1447 Change-Id: I45a0ab4ac08e7ccedd770867ed43de92040b6a10 Signed-off-by: treyad <treyad@viosoft.com>
2019-02-22Add new scenario NSPerf-RFC3511Chornyi, TarasX1-0/+55
New scenarion executes complete run per runner's iteration and designed to work with IxLoad TG. JIRA: YARDSTICK-1592 Change-Id: I43f37ae4ab211a29d75a9c9edd93141767a4c57d Signed-off-by: Chornyi Taras <tarasx.chornyi@intel.com>
2019-02-21Unify TG and VNF names in infrastructure filesOrest Voznyy1-1/+1
Update naming in related files, i.e. pod, docs, docstrings. Rename TGs and VNFs to follow a single naming convention, for all VNFs: name: trafficgen_0 role: tg__0 name: vnf_0 role: vnf__0 JIRA: YARDSTICK-1578 Change-Id: I7596b915939f242349a99721b6088ae738dd277f Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2019-02-11benchmark.core.report: Add _combine_timesEmma Foley1-0/+31
_combine_times() combines and interleves separate lists of timestamps. This will allow the Yardstick and Barometer timestamps to be combined for the dynamic HTML report created with ``yardstick report generate-nsb`` command. JIRA: YARDSTICK-1593 Change-Id: I0f5ea4d001775495fb9b9b4de2d2360c9c61cc51 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2019-02-11benchmark.core.report: Add _get_baro_metricsEmma Foley1-0/+60
Retrieve and parse Barometer metrics relevant to a Yardstick testcase. _get_barometer_metrics() retrieves NFVi metrics from collectd for metrics collected during a Yardstick test case. JIRA: YARDSTICK-1593 Change-Id: I1ef1ccd999e1cbee546db39fb8ad6de0d97e6576 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2019-02-11Refactor: add _format_datasetsEmma Foley1-31/+41
JIRA: YARDSTICK-1593 Change-Id: I8d2c8665d767c92da20db8f97690f20da4a68908 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2019-02-07Update Intel Copyright for files edited in 2019John O Loughlin1-1/+1
JIRA: YARDSTICK-1591 Change-Id: I3ea9039d25bfce578681adb9e27e1598e84a1f56 Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
2019-02-06Merge "Update Sequence runner to use ScenarioOutput class"Volodymyr Mytnyk1-19/+14
2019-02-05Update Sequence runner to use ScenarioOutput classSerhiy Pshyk1-19/+14
List of changes: - update Sequence runner to use new mechanism for reporting data (kpi) - add new rfc2544 example test case that executes the same test with different framesizes sequentially JIRA: YARDSTICK-1592 Change-Id: I731cf1ca40f7232b26ea68f81d22248ebc53eb62 Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
2019-02-01Add vBNG test cases stats processing functionalityOleksandr Naumets1-0/+8
JIRA: YARDSTICK-1546 Change-Id: I3575223ef8231fb4d574e961360b0f2d90a71749 Signed-off-by: Oleksandr Naumets <oleksandrx.naumets@intel.com>
2019-01-31Merge "Refactor: add _get_timestamps()"Volodymyr Mytnyk1-10/+21
2019-01-31Add new scenario NSPerf-RFC2544Serhiy Pshyk3-52/+150
List of changes: - Added new scenario NSPerf-RFC2544 that runs complete test per runner's iteration - NSPerf is still present for backward compatibility - Added ScenarioOutput class which allows scenario to report data (kpi,...) at any moment of time (not only once per run) - New output mechanism is used only by new NSPerf-RFC2544 scenario JIRA: YARDSTICK-1592 Change-Id: Ica8078f61d895f74df90e8eaad7a675b28bd19c6 Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
2019-01-30Refactor: add _get_timestamps()Emma Foley1-10/+21
JIRA: YARDSTICK-1593 Change-Id: I3ec352dd577c6030fa86a2817e264ba7c80773f1 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2019-01-29NSB sync: clean-up draft IPC implementation (part 1)Mytnyk, Volodymyr4-238/+1
The existing implementation of IPC is not finished and isn't used by any of the VNFs/TG implementation. It is used in the code but does nothing from functionality perspective. New syncronization mechanism is going to be implemented by using different approach than it was designed before. Thus, the current IPC mechanism is not going to be re-used. So, removing it. The IPC consumer/producer implementation is left as it may be required for other purposes. - Remove SampleVNF MQ consumer class - Remove IterationIPC MQ producer for VNF control messages - Remove MQ producer from SampleVNFTrafficGen class - Remove TrafficGeneratorProducer class - Remove IterationIPC runner - Remove unused task_id form VNF Generic initialization as it is not required for synchronization of VNFs/TGs anymore. - Fix UT JIRA: YARDSTICK-1592 Change-Id: I65fe51bcbd1bfeea0c43eb79ca6fb2aab5b65ae7 Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2019-01-24Merge changes from topic 'report/html_table'Volodymyr Mytnyk1-42/+80
* changes: Additional rework of NSB report Only show metrics for selected nodes in data table Fix conversion to JS for HTML reports
2019-01-24Merge "[dovetail] split the sla check results into process recovery and ↵Rex Lee2-20/+19
service recovery for HA tcs"
2019-01-22[dovetail] split the sla check results into process recovery and servicerexlee87762-20/+19
recovery for HA tcs make sla check for process recovery optional: only check when sla is set. JIRA: YARDSTICK-1512 Change-Id: I717d39614a1af2ee9e62dec3b8a32d7d611f557e Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2019-01-21Merge "Pass the path of ssh public key to scenario method"Emma Foley1-1/+3
2018-12-24Merge "Compatible with python2 and 3 string check"Jing Lu1-1/+2
2018-12-21Additional rework of NSB reportPatrice Buriez1-12/+17
- Make format_for_jstree expect a list of metric names - Avoid displaying timestamps twice in initial data table - Sort metrics in initial data table - Display testcase name - Fix styling - Make better use of JS and jQuery features - Move event handler to JS file - Avoid adding multiple tbody elements into data table - Adjust unit tests and functional tests accordingly JIRA: YARDSTICK-1367 Topic: report/html_table (12 of 12) Change-Id: I85d853f8e392953cace67e94fa0af2e2492a2b86 Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-12-21Fix conversion to JS for HTML reportsPatrice Buriez1-30/+63
Some new fields in InfluxDB have plain text contents that AST cannot parse as valid expressions, resulting in exceptions being raised by yardstick CLI commands "report generate" and "report generate-nsb". Reworked _generate_common() to properly handle and convert: - unicode, str and missing keys - None, unicode, str, float, long and int values - float, long and int stored as unicode or str values Added test__generate_common() unit test, to verify all possible conversion flavors. Also renamed incorrect "tasks" to "metrics". Also fixed warning in Python3 for some unit tests, caused by deprecation of unittest.TestCase.assertRaisesRegexp(). JIRA: YARDSTICK-1367 Topic: report/html_table (10 of 12) Change-Id: Iff75bfd2c1dcaf3806f67f52f0ea594f10aceb5b Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-12-19Add report generate-nsb commandPatrice Buriez1-13/+50
JIRA: YARDSTICK-1367 Topic: report/html_table (8 of 12) Change-Id: I8a2f96224a745334b67dd71875bebb3b69b9adc3 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-12-19Use Chart.js for graphs in HTML reportsPatrice Buriez1-7/+4
This JavaScript library is available under MIT license. Also adjusted version for jQuery and jsTree, and added fallback font-family names. JIRA: YARDSTICK-1367 Topic: report/html_table (6 of 12) Change-Id: Ibe8b7e3d1f1365d2cbc019bfc22762aaa365a4e1 Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-12-13Add raise for iteration exception in IterationRunnercope.li1-0/+1
When we run the testcase, if there was a exception in iteration _work_process, the process will still run on ,and the test result will not affected by the exception. JIRA: YARDSTICK-1555 Change-Id: I03803c0473015379bb0eb20ed7d474a2d923a4e3 Signed-off-by: cope.li <cope.li@huawei.com>
2018-12-11Pass the path of ssh public key to scenario methodcope.li1-1/+3
In our testcase, we need the key_file for our VM login. JIRA:YARDSTICK-1558 Change-Id: If99704c4f91c87d79e156f2dd73795a059b7ce6c Signed-off-by: cope.li <cope.li@huawei.com>
2018-12-07Compatible with python2 and 3 string checkcope.li1-1/+2
JIRA:YARDSTICK-1557 Change-Id: I54ee4d6ef835e8e6ecd55cfa305151e842f88b03 Signed-off-by: cope.li <cope.li@huawei.com>
2018-11-29Merge "Distribute dpdk/vhostuser between CPUs for SA OVS"Volodymyr Mytnyk1-10/+25
2018-11-22Add parser to support jsTree in reportPatrice Buriez1-3/+63
Allow the user to select what data to show in the report using jsTree to navigate a hierarchical metrics list. JIRA: YARDSTICK-1367 Topic: report/html_table (4 of 11) Change-Id: I86d782a0a70b80a1cdfaab2f41afb7668066cbf7 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-11-22Generate report from a file instead of stringPatrice Buriez1-14/+17
``$ yardstick report generate ...`` now renders a template stored in a file instead of from an imported string Instead of using Django, templating is done by jinja2, as this is used in other parts of Yardstick. JIRA: YARDSTICK-1367 Topic: report/html_table (2 of 11) Change-Id: Iaff53e7e28903e46164ce0977f6b8adbe04d23d7 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-11-22Fix OvS-DPDK context mem allocation problemMytnyk, Volodymyr1-5/+11
The hugepages allocated for OvS-DPDK context is not correct, thus it is causing an issue to allocate the required amount of memory for VM. JIRA: YARDSTICK-1538 Change-Id: I1a34199d4f78c999476603fa86d79ffc1fbf27c8 Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-11-21Update unit tests for yardstick/benchmark/core/report.pyPatrice Buriez1-6/+2
* Remove redundant imports * Rename some constants * Add additional unit tests * Reword KeyError message JIRA: YARDSTICK-1367 Topic: report/html_table (1 of 11) Change-Id: I3d89c1ed9acaeb7421a0734e33075ebae252c035 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-11-20Merge "Make VM machine type as a user configuration"Volodymyr Mytnyk1-2/+4
2018-11-20Make VM machine type as a user configurationMytnyk, Volodymyr1-2/+4
For backward compatibility, the default value of this option has been left unchanged. JIRA: YARDSTICK-1528 Change-Id: I42ceca96501f1136cba27d85c2d82240bd45c10c Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-11-18Run testcase 074 result overridden by job statusAmeed Ashour1-2/+2
Change-Id: Icb24727fd7ca8a340ea424284b510b007055ec9c JIRA: YARDSTICK-1526 Signed-off-by: Ameed Ashour <Ameed.Ashour.Ext@nokia.com>
2018-11-13Distribute dpdk/vhostuser between CPUs for SA OVSStepan Andrushko1-10/+25
Update Yardstick with ability to distribute DPDK/vhost-user ports/queues between CPU cores. Extended OVS add vhost-user interface command with option "n_rxq", provided in the test case. Ordered ports added to OVS. JIRA: YARDSTICK-1490 Change-Id: I71f5d878a9ee3a11f636762fc16b04ee6bc79e14 Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
2018-11-12Merge "Implement functionality for updating http client"Volodymyr Mytnyk1-2/+23
2018-11-12Implement functionality for updating http clientMyron Sosyak1-2/+23
Implemented functionality for updating http client configuration. Extended http_ixload.py with functions for update page object and count of simulated users in http client on netTraffic. JIRA: YARDSTICK-1435 Change-Id: I9a0be226d4201d861d3a764864b42e5d87a4a305 Signed-off-by: Myron Sosyak <myronx.sosyak@intel.com> Signed-off-by: Taras Chornyi <tarasx.chornyi@intel.com>
2018-11-08Merge "add error logs to storperf"Abhijit Sinha1-16/+21
2018-11-07Merge "Setup hugepages on SA host(sriov, ovs_dpdk)"Abhijit Sinha2-1/+10
2018-11-07Merge "Make OvS max_idle & queues configuration optional"Abhijit Sinha2-5/+14
2018-11-07Merge "Update: assign static IP to VM for standalone"Abhijit Sinha1-1/+3