aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-04-18Merge "Replace neutron create security group full with shade."Rodolfo Alonso Hernandez5-110/+170
2018-04-18Update the dummy-scenario-heat-context testcaseEmma Foley1-3/+4
The testcase is included in the docs to demonstrate definining a heat context. It previously used cirros, but this is not included by default. Set-up instructions/scripts add yardstick-image and yardstick-flavor, so these have been added to the testcase instead. A default value for context_name was also added. Change-Id: I5f642086971487b334ae48a5cf5c9b71e9b99e2e Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-04-17remove apex lake tc007 from test suitesRoss Brattain4-52/+0
fixes opnfv_os-nosdn-fdio-noha_daily.yaml failures due to missing testcase Change-Id: I5f050d72334da46b94de5fa3662047bf694f7e3e Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-04-17fix unittest test_deploy_try_retrieve_context_does_not_existRoss Brattain1-1/+1
unittest is failing due to mock being called multiple times. I assume this is due to parallelization of tests in Jenkins somehow ====================================================================== FAIL: test_deploy_try_retrieve_context_does_not_exist (benchmark.contexts.test_heat.HeatContextTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/mock/mock.py", line 1305, in patched return func(*args, **keywargs) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/tests/unit/benchmark/contexts/test_heat.py", line 309, in test_deploy_try_retrieve_context_does_not_exist mock_path_exists.assert_called_once_with(key_filename) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/mock/mock.py", line 947, in assert_called_once_with raise AssertionError(msg) AssertionError: Expected 'exists' to be called once. Called 3 times. ---------------------------------------------------------------------- Change-Id: I696c63ef9f7943b841e2d3d533c92f18aa7044da Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-04-17OpenStack deployment: delete RS or all VMsStepan Andrushko5-24/+56
New parameter is added to shell script to delete data for VMs either from input file or all. Added disk images removal as per new input parameter. JIRA: YARDSTICK-1123 Change-Id: I8d2cee4a3a7ad7147f4d59303bab656d80370221 Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
2018-04-17Remove unused methods in SampleVNFEmma Foley2-90/+0
The following private methods are unused in yardstick/network_services/vnf_generic/vnf/sample_vnf.py:SampleVNF * _build_ports(self) * _get_route_data(self, route_index, route_type) * _get_port0localip6(self) * _get_port1localip6(self) * _get_port0prefixlen6(self) * _get_port1prefixlen6(self) * _get_port0gateway6(self) * _get_port1gateway6(self) They are being removed as: * _get_port* functions are already implemented and used in MultiPortConfig * _get_route_data is an auxillary functions used by the previous ones * _build_ports is implemented in the helpers JIRA: YARDSTICK-1122 Change-Id: Ie7a706cd87b610ea56f89214118a02e612a70dc0 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-04-16Improve "Libvirt.create_snapshot_qemu" functionRodolfo Alonso Hernandez3-18/+111
Improved "Libvirt.create_snapshot_qemu" function: - Check if the base image is present in the remote host. If not, try to copy from the execution host. - Check the execution status. In case the command fails, raise an exception. JIRA: YARDSTICK-944 Change-Id: I78bd0d3ed6a1f35ed772c0d192bb240009a580ed Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-04-16Update neutron delete_neutron_net function.Shobhi Jain4-24/+34
Update the changes done in patch [1]. JIRA: YARDSTICK-890 [1] https://gerrit.opnfv.org/gerrit/#/c/52361/ Change-Id: Iccbf16d0a2e72919d5092366241ff41686f01a58 Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>
2018-04-16Replace neutron create security group full with shade.Shobhi Jain5-110/+170
Function create_security_group_full now uses shade client. Remove openstack util's private 'get_security_groups', 'get_security_group_id' & 'create_security_group' functions as 'create_security_group_full' now uses 'get_security_group' & 'create_security_group' function defined in shade client library. JIRA: YARDSTICK-890 Change-Id: Ibd5b9ca9b7ce99b3e40c5dcc545f2af672ce2203 Signed-off-by: Shobhi Jain <shobhi.jain@intel.com>
2018-04-16Merge "Remove tool provisioning in Standalone contexts"Emma Foley2-9/+5
2018-04-16Get HA test case results on failureMiikka Koistinen5-75/+55
Now, yardstick HA test cases (which have 'sla action: monitor') do not output results if the test case SLA validation fails. This patch modifies the task runner and the serviceHA scenario so that during an SLA validation failure the test result data is captured correctly. Do similar changes in the general scenario. JIRA: YARDSTICK-1115 Change-Id: I648a8b229600c0ad089320ac3f803698f73aa800 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-04-16Merge "Bugfix: interface down HA tc050 attack and monitor logistic wrong"Rex Lee2-91/+42
2018-04-13Add MQ consumer, producer and payload base classesRodolfo Alonso Hernandez12-0/+490
Added MessagingProducer base class. A class implementing this base class can send a cast message using the MQ service installed in the Yardstick jumphost (RabbitMQ by default, other MQ could be implemented). The producer will send messages to an specific topic. Added MessagingConsumer base class. A class implementing this base class will be able to receive any message sent by a MessagingPorducer class publishing messages in the topic subscribed. By default both Producer and Consumer "fanout" is True. That means every Consumer will create a fanout Queue attached to the MQ Exchange topic. All Consumers attached to this topic will receive the message sent by the Producer [1]. Added Payload base class. To send data through the message queue service, a Payload derived object should be created. This base class allows to define the attributes container in the class, convert the object to a dict and retrieve the object from a dict. Added a new library, "oslo.messaging", to implement the RPC client and server. [1]http://blog.thedigitalcatonline.com/blog/2013/08/21/some-tips-about-amqp-direct-exchanges/ JIRA: YARDSTICK-1074 Change-Id: I63932b5fb3de2bdc1270fc83295630a2a349e2a6 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-04-13Bugfix: interface down HA tc050 attack and monitor logistic wrongrexlee87762-91/+42
1. simplify the attack to be one interface 2. move monitor step before attack 3. update rst doc to match JIRA: YARDSTICK-1120 Change-Id: Icfa21d58e7ccdbeaca8a9a1a3098cceb21c981c3 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-04-13Merge "Bugfix: to list all used loop device"Rodolfo Alonso Hernandez1-1/+2
2018-04-13Merge "Bugfix: Can't get image list in API"Jing Lu6-47/+105
2018-04-12Bugfix: to list all used loop deviceting wu1-1/+2
Bug fix in script "build_yardstick_image.yml", to - Use option "-a" to list all used loop devices, to be more compatible. - Ignore the error and continue, if the debug command failed. JIRA: YARDSTICK-1121 Change-Id: I24aaebb2ffe3107ca5c619401725aa8c8e1af295 Signed-off-by: ting wu <ting.wu@enea.com>
2018-04-12Bugfix: Can't get image list in APIchenjiankun6-47/+105
JIRA: YARDSTICK-1110 To match OpenStack Pike release, we upgrade novaclient from 7.1.1 to 9.1.1. And the client interface has changed, it no longer has nova_client.images attribute. So I use shade instead. Later will change all same issue. Change-Id: Ie4f54069d4346e44e2ad925439930504b945cbad Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2018-04-11Merge "Bug fix in using Unixbench"Rodolfo Alonso Hernandez1-1/+1
2018-04-11Add docker support for Ubuntu 17.10 with YardstickJohn O Loughlin2-14/+2
JIRA: YARDSTICK-1119 Change-Id: I6d9b21667605f1c0a157c2a72e708c36b90e8c3a Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
2018-04-11Upgrade collectd to version 5.8John O Loughlin1-1/+1
Update collectd version to 5.8 as it's the latest stable and supports GCC 7 JIRA: YARDSTICK-1114 Change-Id: I9147b7277d87047eda729664d5ee3e8ab2c145af Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
2018-04-11Merge "Add support for dpdk 17.05"Emma Foley7-21/+23
2018-04-11Add support for dpdk 17.05John O Loughlin7-21/+23
Add support for dpdk 17.05. Add option to dpdk scripts to download dpdk 17.08 17.11 and 18.02. These are currently unsupported due to prox build issues. JIRA: YARDSTICK-1113 Change-Id: I585566364a4d1f04a696439af96c50c0e9d38033 Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
2018-04-11Bug fix in using Unixbenchliyin1-1/+1
JIRA: YARDSTICK-1117 This bug will cause the error in running unixbench. "no such file named ./run" So we have to fix it. Change-Id: I141fcb5567fbb0e3df97aee677cb946a25a9788e Signed-off-by: liyin <liyin11@huawei.com>
2018-04-10Bugfix: k8 scenario fixrexlee87762-45/+19
JIRA: YARDSTICK-1109 Change-Id: Ic61a63099c3d2d4af5fa5634299e4b2946d53eaf Signed-off-by: rexlee8776 <limingjiang@huawei.com> (cherry picked from commit 3b1316647b50bc3bbb1eb6216aa07fd4dff34edf)
2018-04-07docker/supervisor.sh: Fix condition block endAlexandru Avadanii1-3/+3
Fixes: 18546b4f Change-Id: I51fcb48e790ff7c57273a3f6712ccb2b9bc4aa32 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit ead90064d12be552bd334828d41ffffa75fad01b)
2018-04-07docker/supervisor.sh: Fix condition block endAlexandru Avadanii1-3/+3
Fixes: 18546b4f Change-Id: I51fcb48e790ff7c57273a3f6712ccb2b9bc4aa32 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-04-06[AArch64] Dockerfile: Fix malformed patchAlexandru Avadanii1-2/+2
Change-Id: Ia2dfc6c4def8e71e36747b8f27cbf22700f676b1 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 161f7d676b4404a5445681a7871e086b751fdafc)
2018-04-06[AArch64] Dockerfile: Fix malformed patchAlexandru Avadanii1-2/+2
Change-Id: Ia2dfc6c4def8e71e36747b8f27cbf22700f676b1 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-04-06Merge "Bugfix: k8 scenario fix"Abhijit Sinha2-45/+19
2018-04-06[docs] Updated baremetal instalation instructionsChornyi, TarasX1-0/+9
JIRA: YARDSTICK-1111 Change-Id: Iefab24f74c87a0855c2d70090ac34256ffa9200c Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
2018-04-03[userguide] Use literalinclude directive in 05-operationEmma Foley1-75/+9
Uses literalinclude directive instead of having a copy of the file content in documentation. This means that any update to this file is detected, and the most up-to-date content is used. If a testcase is updated, the updates wil be picked up. The examples used are ones that are tested reqularly. JIRA: YARDSTICK-1066 Change-Id: I15d967eb04d1e6f4348683d8746fa9989dceda94 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-04-03[docs] update the user guide for "yardstick-operation"Emma Foley4-1/+367
* Add a new chapter: operation which shows how to use Yardstick and how to create test cases * Include details on how to reuse and keep VNFs between testcases JIRA: YARDSTICK-885 JIRA: YARDSTICK-886 Change-Id: I365152e8812427a3c54259adbc18d5ddb216d775 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-04-03[docs] Move user guide chapters to make room for new chapter.Emma Foley15-551/+437
* Increment the numbering on chapters * Fix some of the changes flagged by doc8 Change-Id: Ie03d165297671dfd50b9de339612cdc8bf00b988 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-04-03Bugfix: k8 scenario fixrexlee87762-45/+19
JIRA: YARDSTICK-1109 Change-Id: Ic61a63099c3d2d4af5fa5634299e4b2946d53eaf Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-03-31ADD tc042 into CIliyin3-0/+31
JIRA: YARDSTICK-1108 We add this test case in ovs_dpdk-ha and ovs_dpdk-noha testsuites also add image build into load_images script Change-Id: I2b0c6b106dd98c3693df18dba46259ff8ef0a76e Signed-off-by: liyin <liyin11@huawei.com>
2018-03-29add verbose to debug losetup deleteRoss Brattain1-1/+1
JIRA: YARDSTICK-1096 JIRA: YARDSTICK-1054 Change-Id: I5a11dbd2638782c5e499cb94dd5882b8bf696e8d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-03-29Merge "Fix RabbitMQ service installation and initialization"Ross Brattain2-3/+7
2018-03-29Merge "don't print exceptions in atexit handler"Ross Brattain1-1/+1
2018-03-29Merge "task: don't hide exceptions in Task"Ross Brattain1-6/+9
2018-03-29don't print exceptions in atexit handlerRoss Brattain1-1/+1
atexit handler calls terminate_all after the regular python execution path, and it looks like the traceback stack is None somehow. In this context log.debug("", exc_info=True) doesn't work it prints out NoneType on Python3 and causes other problems on Python2.7 remove the exc_info=True from the logging call JIRA: YARDSTICK-1107 Change-Id: Ida0a0ced7ff5e017e2f8608880e3bb531724af95 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-03-28add new Fraser scenariosRoss Brattain10-0/+304
New scenarios from wiki: https://wiki.opnfv.org/display/SWREL/Fraser+Scenario+Statu Change-Id: Ifd6e45e73be2bbb99743aa3f4981d22899aab92a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-03-28Merge "restore Heat stack failure logs for CI"Ross Brattain2-1/+27
2018-03-28Add dashboard for TC058Miikka Koistinen1-0/+265
This dashboard is for Yardstick test case TC058. JIRA: YARDSTICK-1091 Change-Id: Ib76d917f52cd0fc7d968ac300f8e7e01de3dfd0f Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-03-28Remove tool provisioning in Standalone contextsRodolfo Alonso Hernandez2-9/+5
In both Standalone contexts, SR-IOV and DPDK, the tool provisioning must be removed. If the tool, "dpdk-devbind.py", is not in the VNF, the test should fail because this tool should be provided during the provisioning, not during the test execution. JIRA: YARDSTICK-1092 Change-Id: I212c1303fd655f151e9c0efc29aad97fe6d71d1c Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-03-28Fix RabbitMQ service installation and initializationRodolfo Alonso Hernandez2-3/+7
RabbitMQ service added in [1] is not correctly installed and initialized: - There is an error during the installation process ("\" character missing). - In the installation script, the service needs to be started first. - In the container installation, the service needs to be started via supervisor. [1]https://gerrit.opnfv.org/gerrit/#/c/53597/ JIRA: YARDSTICK-1103 Change-Id: Iade3d6ce4b522e6f576af71b7afe5559081f7929 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-03-28Merge "Replace neutron create security group rule with shade."Rex Lee2-38/+76
2018-03-28Merge "Dashboard for TC056"Rex Lee1-0/+301
2018-03-27task: don't hide exceptions in TaskRoss Brattain1-6/+9
If the Task raised an exception we currently hide it and replace it with RuntimeError. This is bad. If an exception occured, then we don't have a result so re-raise the original exception. Or we could log the traceback and raise RuntimeError, but that doesn't seem to be a good idea. Sample traceback after re-raising original. Without this patch the ValueError is only written to _write_error_data 2018-03-25 22:57:56,511 yardstick.benchmark.contexts.node node.py:85 DEBUG BareMetals: [] 2018-03-25 22:57:56,511 yardstick.benchmark.contexts.node node.py:89 DEBUG Env: {} 2018-03-25 22:57:56,511 yardstick.cmd.commands.task task.py:57 INFO Task FAILED Traceback (most recent call last): File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/cmd/commands/task.py", line 54, in do_start result = Task().start(param, **kwargs) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 103, in start task_args_fnames) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 321, in _parse_tasks task_args_fnames[i] File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 558, in parse_task context.init(cfg_attrs) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/heat.py", line 131, in init server = Server(name, self, server_attrs) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/model.py", line 210, in __init__ (name, p)) ValueError: server 'trafficgen_1', placement 'pgrp2' is invalid 2018-03-25 22:57:56,512 yardstick.cmd.commands.task task.py:62 INFO Task FAILED 2018-03-25 22:57:56,662 yardstick.benchmark.runners.base base.py:124 DEBUG Terminating all runners NoneType JIRA: YARDSTICK-1102 Change-Id: I7e6fa41fc1d36f6d438a1602ab60cb41ffbee1e9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-03-27restore Heat stack failure logs for CIRoss Brattain2-1/+27
We need to see what Heat failures occured so that we can debug CI failures. The only way to do this seems to be to use the private event_utils.get_events function to query for all failures Also since we had a failure go ahead and dump the Heat template that failed. Sample output: 2018-03-25 23:50:08,765 [INFO] yardstick.orchestrator.heat heat.py:629 Creating stack 'yardstick-460ed969' START 2018-03-25 23:50:21,932 [ERROR] yardstick.orchestrator.heat heat.py:644 Resource CREATE failed: BadRequest: resources.yardstick-460ed969-xe1: Invalid input for operation: physical_network 'nosuch' unknown for flat provider network. Neutron server returns request_ids: ['req-6f981f1e-a9e2-4114-af84-1ee528aed51b'] 2018-03-25 23:50:21,933 [ERROR] yardstick.orchestrator.heat heat.py:644 BadRequest: resources.yardstick-460ed969-xe1: Invalid input for operation: physical_network 'nosuch' unknown for flat provider network. Neutron server returns request_ids: ['req-6f981f1e-a9e2-4114-af84-1ee528aed51b'] 2018-03-25 23:50:21,972 [ERROR] yardstick.orchestrator.heat heat.py:645 {'description': '\n' 'All referred generated resources are prefixed with the ' 'template\n' 'name (i.e. yardstick-460ed969).\n', 'heat_template_version': '2013-05-23', 'outputs': {'trafficgen_1.yardstick-460ed969': {'description': 'VM UUID', 'value': {'get_resource': 'trafficgen_1.yardstick-460ed969'}}, 'trafficgen_1.yardstick-460ed969-fip': {'description': 'floating ' 'ip ' JIRA: YARDSTICK-998 Change-Id: Ia8f4e5ba7e280fb9086519680d5ee90a2b442e6b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>