aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick
AgeCommit message (Collapse)AuthorFilesLines
2018-02-28Addition of storage of extra counters for GrafanaDanielMartinBuckley3-21/+98
JIRA: YARDSTICK-1036 This stores a number of extra counters in influxdb for Prox test cases. It also stores existing counters with a "succcess_" tag. Previously throughput where stored without success or failure indication. Also "Result_" counters are also stored. These can now be used by Grafana to graph output. Change-Id: Ie5636c14ecbab1b53a988bdfbd47ddd1fcdbd695 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-02-15NSB: Ixia remove unused traffic_generator argRoss Brattain2-15/+14
also fix pylint unittest warnings JIRA: YARDSTICK-1014 Change-Id: I4ff9769fdca89c6fc1fb9ca62e9f8ea6b174681e Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-02-14NSB traffic_profile: fix pylint warnings and return partialRoss Brattain1-0/+6
also fix unittests Change-Id: Ie3026f5a0df8e690271afb71fe36ac531cbae2a0 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-02-14constants: fix pylint warnings for OSErrorRoss Brattain1-6/+4
IOError and OSError are the same in Python 3? Anyway we want to ignore ENOENT, not EEXIST JIRA: YARDSTICK-1012 Change-Id: Ia75e41fecce505685745e0099669e45f3e82a1be Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-02-12Move tests: unit/benchmarkEmma Foley145-0/+13893
* Fix pylint errors * Add TODOs Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged with a TODO, and should be fixed later. Change-Id: I30eb4b0aafe0575d0cddbc946108291f21a98ed8 Jira: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-02-12Merge "Heatstack create: fix str/bytes error in NamedTemporaryFile.write()"Emma Foley2-2/+2
2018-02-12Merge "Replace neutron get network id with shade."Emma Foley3-99/+102
2018-02-12Heatstack create: fix str/bytes error in NamedTemporaryFile.write()Ross Brattain2-2/+2
we need to use jsonutils.dump_as_bytes for NamedTemporaryFile.write() 2018-02-11 19:15:01,366 [INFO] yardstick.orchestrator.heat heat.py:563 Creating stack 'yardstick-4359f33e' START 2018-02-11 19:15:01,935 [ERROR] yardstick.benchmark.contexts.heat heat.py:317 stack failed Traceback (most recent call last): File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/heat.py", line 313, in deploy timeout=self.heat_timeout) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/orchestrator/heat.py", line 567, in create stack.create(self._template, self.heat_parameters, block, timeout) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/orchestrator/heat.py", line 56, in create template_file.write(jsonutils.dumps(template)) File "/home/rbbratta/yardstick_venv3/lib/python3.5/tempfile.py", line 622, in func_wrapper return func(*args, **kwargs) TypeError: a bytes-like object is required, not 'str' JIRA: YARDSTICK-1005 Change-Id: I504f23b86119b62f3aea5b83a445b97bf810220c Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-02-12Merge "vpe: convert DPKD_PORT to LINK ID"Emma Foley1-10/+29
2018-02-11Merge "Remove references to "dpdk_nic_bind" utility"Emma Foley4-34/+32
2018-02-11Merge "Replace "python-heatclient" with "shade" client"Rodolfo Alonso Hernandez3-544/+245
2018-02-09Replace neutron get network id with shade.Shobhi Jain3-99/+102
Function get_network_id now uses shade client instead of shade client. Removed redundant function: get_port_id. JIRA: YARDSTICK-890 Change-Id: I6081477fee39fa78131187e65daf3d7d3a9e439f Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>
2018-02-09Merge "Make 'Scenario' classes plugable"Emma Foley1-13/+23
2018-02-09Replace "python-heatclient" with "shade" clientRodolfo Alonso Hernandez3-544/+245
Replaced "python-heatclient" with "shade" client. The "python-heatclient" library is removed from requirements file. "shade" client version 1.22.2 is the required one for OpenStack Pike release. shade is the recommended client to interact with OpenStack clouds [1] [1] https://github.com/openstack-infra/shade JIRA: YARDSTICK-880 Change-Id: Ibbdfc42a543d2d9ed4cf70d78de3de8cbecd0076 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-02-07Merge "Improve ovs_deploy.bash script"Ross Brattain1-16/+7
2018-02-06vpe: convert DPKD_PORT to LINK IDRoss Brattain1-10/+29
http://dpdk.org/doc/guides/sample_app_ug/ip_pipeline.html#application-resources-present-in-the-configuration-file LINK instances are created implicitly based on the PORT_MASK application startup argument. LINK0 is the first port enabled in the PORT_MASK, port 1 is the next one, etc. The LINK ID is different than the DPDK PMD-level NIC port ID, which is the actual position in the bitmask mentioned above. For example, if bit 5 is the first bit set in the bitmask, then LINK0 is having the PMD ID of 5. This mechanism creates a contiguous LINK ID space and isolates the configuration file against changes in the board PCIe slots where NICs are plugged in. Change-Id: I6e449272cfcfb2b2a75c246f7f569e3f923da245 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-02-05Remove references to "dpdk_nic_bind" utilityRodolfo Alonso Hernandez4-34/+32
In DPDK 16.07 [1], "dpdk_nic_bind" utility was renamed to "dpdk-devbind". This patch removes all references to this previous naming to avoid confusions. [1] https://dpdk-guide.gitlab.io/dpdk-guide/setup/binding.html JIRA: YARDSTICK-995 Change-Id: I827c4c11c9e9e519f33f21b80459e7d65e51545e Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-01-30Make 'Scenario' classes plugableRodolfo Alonso Hernandez1-13/+23
This patch makes yardstick.benchmark.scenario.base:Scenario classes plugable. A new entry point is added to the setup. This entry point could be extended in other plugin projects to add new Scenario classes. E.g.: take a look at [1]. This is a Yardstick plugin example project. Clone the project and execute, from the project directory: $ sudo -EH python setup.py install This will add a new module to python ('yardstick-new-plugin') and a new Scenario class. Now list the scenarios in Yardstick: $ yardstick scenario list ... | SpecCPU2006_for_VM | Spec CPU2006 benchmark for Virtual Machine | | SpecCPU2006 | Spec CPU2006 benchmark | | Dummy2 | Execute Dummy (v2!) echo | +-----------------------+--------------------------------------------+ [1] https://github.com/ralonsoh/yardstick_new_plugin JIRA: YARDSTICK-910 Change-Id: Ib70ee9bf4dc7ff91d1dd6377317b313288e36bff Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-01-29Merge "Fix up tests for Parser scenario"Ross Brattain1-3/+3
2018-01-29Merge "Move tests: unit/cmd/"Ross Brattain5-0/+222
2018-01-29Improve ovs_deploy.bash scriptRodolfo Alonso Hernandez1-16/+7
Improved ovs_deploy.bash script [1]: - Check if there is an error during the execution. - Removed "prerequisites" function because is not called. - Modified DPDK package installation: execute "make config" and then "make" - Added comments. [1]https://github.com/opnfv/yardstick/blob/master/yardstick/resources/scripts/install/ovs_deploy.bash JIRA: YARDSTICK-916 Change-Id: I3029f983dde72aa46f6fdd61dd64d086ab3c2af6 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-01-27Fix up tests for Parser scenarioEmma Foley1-3/+3
* Check the results at the test methods * Remove print statement from Parser scenario * Replace assertEquals(x, True) with assertTrue(x) Change-Id: I5c2612692e625fc888c7ce7637b9e1625440724a Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-25Replace assertEquals with assertEqualEmma Foley1-1/+1
unittest.assertEquals is deprecated[1], and has been replaced with unittest.assertEqual. [1] https://docs.python.org/2/library/unittest.html#deprecated-aliases Change-Id: I9c6320e3a9ec5528036b529a9c32fc48b0bcfd62 JIRA: YARDSTICK-864 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-25Move tests: unit/cmd/Emma Foley5-0/+222
* Fix pylint errors * Add TODOs Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged with a TODO, and should be fixed later. Change-Id: I505d72e1c646fea1976a0bf854369c6b6308bbb1 Jira: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-24Merge "'libvirt' guest console in Standalone model"Ross Brattain1-3/+10
2018-01-23Merge "Use assertIn(x, y) instead of other variations"Rodolfo Alonso Hernandez1-1/+1
2018-01-23Rename ResourceProfile.check_if_sa_running()Emma Foley3-32/+25
* Update log message and rename method to make debugging easier * Fix pep8 violations in touched files JIRA: YARDSTICK-859 Change-Id: Id138d72cd8557ffacd5dcc81d06feeb484e13625 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-22Merge "Clean unit tests in "benchmark.scenarios.lib.test_create_volume.py"Ross Brattain1-5/+4
2018-01-22Merge "Move tests: unit/dispatcher"Ross Brattain3-0/+177
2018-01-22Merge "Move tests: unit/orchestrator/"Ross Brattain3-0/+600
2018-01-22Clean unit tests in "benchmark.scenarios.lib.test_create_volume.pyEmma Foley1-5/+4
* Use mock_object.assert_called_with instead of self.assertTrue(mock_object.called) * Remove an unneeded if clause in CreateVolume * Add individual tests for each method in CreateVolume Change-Id: I54cc56434d316f059b3822879145e6a299866fcb Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-19Replace "oslo_utils.importutils" with standard library "importlib"Rodolfo Alonso Hernandez6-29/+73
The current implementation of dynamic library importation is prone to failure [1]: - "sys.modules" is modified manually, which is something not recommended [2]. - When a module is imported is added to "sys.modules"; that means there is no need to manually create an entry in this object. - "importlib" library is part of the standard library and is now available in PY3 and PY2 (backported). This library contains a function called "import_module" to import a module in runtime. [1]https://github.com/opnfv/yardstick/blob/d2c7cc4e9768ed003257a95c92cdb278d516761b/yardstick/common/utils.py#L72-L93 [2]http://justus.science/blog/2015/04/19/sys.modules-is-dangerous.html JIRA: YARDSTICK-949 Change-Id: Ide3b74f98858d06fa275fb6c9b78ceeaa64feed5 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-01-17Move tests: unit/orchestrator/Emma Foley3-0/+600
* Fix pylint errors * Add notes where additional action is needed. Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged for follow-up, and should be fixed later. JIRA: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Change-Id: Idbf56d0cd3b29b70691acd6af4d5ac3c400d2ccc
2018-01-16bugfix: Fix duplicate 'address' in standlone SRIOVDino Madarang1-2/+1
* Add unit test per Rodolfo's comment Creating a VM using generted xml file /tmp/vm_sriov_0.xml returns an XML error: missing source address type. This fix modifies generted xml from: <source> <address> <address bus="0x1a" domain="0x0000" function="0x0" slot="0x10" type="pci" /> </address> </source> to: <source> <address bus="0x1a" domain="0x0000" function="0x0" slot="0x10" type="pci" /> </source> Change-Id: I694153e7468986bacb19ba3e09e761993aad7184 Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Reviewed-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> Reviewed-by: Alain Jebara <alain.jebara@intel.com> Reviewed-by: Emma Foley <emma.l.foley@intel.com>
2018-01-12Use assertIn(x, y) instead of other variationsEmma Foley1-1/+1
unittest.assertIn(x,y) should be used instead of: * unittest.assertTrue(x in y) * unittest.assertEqual(x in y, True) Also fixes pep8 violations existing in the file. Change-Id: I705c1bd4af74757b5c928995894aede436d66817 JIRA: YARDSTICK-901 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-12Move tests: unit/dispatcherEmma Foley3-0/+177
* Fix pylint errors * Add TODOs Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged with a TODO, and should be fixed later. JIRA: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Change-Id: I1f8a6041424c4c69f46ef30bd5c66d2d51d24347
2018-01-11Move tests: unit/commonEmma Foley9-0/+1668
* Fix pylint errors * Add TODOs Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged with a TODO, and should be fixed later. JIRA: YARDSTICK-837 Change-Id: I2847c380556985fae2c92bb8a7da9c47e91e1338 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-11Merge "Move tests: functional/"Ross Brattain3-0/+146
2018-01-11Merge "Move tests: unit/apiserver/"Ross Brattain4-0/+161
2018-01-11Move tests: unit/apiserver/Emma Foley4-0/+161
* Fix pylint errors * Add TODOs Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged with a TODO, and should be fixed later. JIRA: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Change-Id: I30d3f61e5ea479758f9a2f39cf415da18e49b9d6
2018-01-11Move tests: unit/test_ssh.pyEmma Foley1-0/+572
Change-Id: I0eabdd5d7e918dcf0df747119a104dedf125606f Jira: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-11Move tests: functional/Emma Foley3-0/+146
All tests in tests/functional have been moved to yardstick/tests/functional JIRA: YARDSTICK-837 Change-Id: I89276b36635de74dd7b8f70e961c71aa0d2c494e Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-11Run tests in yardstick/tests/Emma Foley3-0/+76
* Edit .testr.conf and tools/run_tests.sh * Run functional and unit tests in tests/ and yardstick/tests/ * Add yardstick/tests/unit/__init__.py and yardstick/tests/__init__.py * Update pylintrc rules to disable: * missing-docstring * fixme * protected-access Test will run in both tests/ and yardstick/tests/ initially Change-Id: Iccd0df0d1b85b5d08f7faa56669bc32687925f7e JIRA: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-01-08Remove unnecessary function definition in IxLoadTrafficGenRodolfo Alonso Hernandez1-5/+2
Removed call to 'instantiate' base class in IxLoadTrafficGen. JIRA: YARDSTICK-894 Change-Id: I0ebee1f5e0f22cfd3a67eaac8accc67ecd35ad0c Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-01-04Merge "SPEC cpu2006 test case for VM"Jing Lu1-0/+170
2018-01-04Remove tool provisioning in PROX helperRodolfo Alonso Hernandez1-30/+24
Tool provisioning in PROX setup environment helper is not needed [1]. The tool (PROX traffic injector) is already provided during the VNF building: ./ansible/nsb_setup.yml --> ./ansible/build_yardstick_image.yml --> ./ansible/ubuntu_server_cloudimg_modify_samplevnfs.yml --> ./ansible/roles/install_samplevnf [1]https://github.com/opnfv/yardstick/blob/master/yardstick/network_services/vnf_generic/vnf/prox_helpers.py#L641 JIRA: YARDSTICK-872 Change-Id: I0f925a7967a35a97901fbe5053793a791a7b1b01 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-01-03Make GenericVNF an abstract classRodolfo Alonso Hernandez7-60/+94
GenericVNF class is now an abstract class. Only optional methods are implemented. Mandatory methods: - instantiate - wait_for_instantiate - terminate - scale - collect_kpi. JIRA: YARDSTICK-866 Change-Id: Ia8766f9f98816e11894d1e72b0f3bd573d091d99 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-12-25SPEC cpu2006 test case for VMJingLu51-0/+170
JIRA: YARDSTICK-906 This test case is for benchmarking VM(VNF) compute performance using SPEC CPU2006. Change-Id: Ida87aa4d394e38b398acb0af3176dd1b4c858c35 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-12-25Add get function in ssh.pyJingLu51-0/+6
JIRA: YARDSTICK-904 This task is about to implement a get function in ssh.py to fetch desired files/folder in a remote VM to a local path. In some test cases, test tools will produce results in files. We want retain the original result files and archive them in Yardstick docker for future usage. Change-Id: Ifd333eb044ce31cffa6f5b2a8f6a46648a8858d1 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-12-22Replace subprocess "check_output" with "Popen"Rodolfo Alonso Hernandez3-5/+125
"check_output" is a blocking wrapper for "Popen" which returns the output of the command execution or raises an exception in case of error. "Popen" is a non-blocking function that allows to create asynchronous tasks. It returns any possible execution error but doesn't raise an exception; this is delegated to the developer. This code is used in the Yardstick CLI base test class. Change-Id: Ie3e1228b2d40cb306354447653678bf581bc9697 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>