aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-07-06move flatten dict key to common utilsrexlee877612-79/+97
So it can easily be used by other testcase to unify result JIRA: YARDSTICK-702 Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-07-05Merge "unify pod keywork so api can easily used"Kubi53-127/+173
2017-07-05Merge "Use "OS_INSECURE" variable as the insecure mode indicator"Kubi7-8/+9
2017-07-04support segmentation_id for vlan provider networkJingLu53-3/+9
If a network specified in the test case is a vlan provider network, new attribute "segmentation_id" can be used to specify the desired vlan tag in the test case network section: networks: test-net: cidr: '192.168.1.0/24' provider: "vlan" physical_network: 'physnet1' segmentation_id: "1000" If the "segmentation_id" attribute is absent, a random vlan tag will be allocated to the network. Change-Id: Ic53852447a3c1bd8feb9ebd42d35f1ade3684be1 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-04Bugfix: background scenario can't not update resultchenjiankun1-2/+4
JIRA: YARDSTICK-700 In this patch: https://gerrit.opnfv.org/gerrit/#/c/35257/ In yardstick/benchmark/core/task.py We update background result only when it be stopped abnormally. We need move it out of 'if' statement. Change-Id: I25b76e160273a87da66098565a16e4a143cec11f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-03Yardstick: Aarch64 jenkins slave supportAlexandru Nemes4-16/+86
Change-Id: I2d58c38d75fc1fe036769340e9fb02932f543510 Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
2017-07-03Bugfix: amend the CLI to build yardstick-image manuallyJingLu51-0/+2
Change-Id: I8fbf0d6383375660c05e55d36fb0d2f91e1d8de8 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-03Merge "Set TEST_DB_URL for storperf"Jing Lu2-1/+14
2017-06-30unify pod keywork so api can easily usedrexlee877653-127/+173
JIRA: YARDSTICK-695 Change-Id: I826cb2ea07949b1eaae4df4cb77175ec8f5df89b Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-30Merge "Change prepareYardstickEnv to prepare_env and add log info"Jing Lu9-120/+140
2017-06-30Use "OS_INSECURE" variable as the insecure mode indicatorJingLu57-8/+9
Now we use "OS_CACERT" as the insecure mode indicator, it is better to use "OS_INSECURE". Change-Id: I1406193e27510390b4b8fd8f4751d8361560172f Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-30Change prepareYardstickEnv to prepare_env and add log infochenjiankun9-120/+140
JIRA: YARDSTICK-697 Currently the action in env like prepareYardstickEnv, createGrafanaContainer, createInfluxDBContainer, this is not python style. So I do some change: prepareYardstickEnv->prepare_env createGrafanaContainer->create_grafana createInfluxDBContainer->create_influxdb And also add some log info in env_action. Change-Id: Ia91576b975f3de76b96e312779fda4911e7cac24 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-30Add API to show test case documentationchenjiankun5-1/+40
JIRA: YARDSTICK-694 We need API to show test case documentation. API: /yardstick/testcases/<testcase_name>/docs method: GET example: http://192.168.131.2:8888/yardstick/testcases/opnfv_yardstick_tc002/docs Change-Id: Ib8d591f0ff5f91c4d0a740539727ec73ddd86249 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-30Merge "Bugfix: remove test.dbf file in recovery script"Jing Lu1-1/+1
2017-06-30Bugfix: remove test.dbf file in recovery scriptzshi1-1/+1
JIRA: YARDSTICK-696 Change-Id: I09b73079ff828c43040723794115bb07f74242e8 Signed-off-by: zshi <zshi@redhat.com>
2017-06-29Merge "Call core code directly in the API of run test case"Rex Lee25-262/+185
2017-06-29Call core code directly in the API of run test casechenjiankun25-262/+185
JIRA: YARDSTICK-688 We need to call core code directly in the API of runTestCase. It would be more stable. Change-Id: I431a85ded7cd3b20da0462f947c25d91bb99decd Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-29Add API to update pod yaml filechenjiankun1-0/+17
JIRA: YARDSTICK-693 Apart from API to upload pod yaml file, we also need API to update pod file. API: /yardstick/env/action method: POST param: { 'action': 'update_pod_file', 'args': { 'pod': { pod content } } } Change-Id: I0ba168612ccc2e43c531e4b9253cf72e5c745297 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-27Merge "Add API to update hosts info about SUT"Rex Lee2-0/+14
2017-06-27Set TEST_DB_URL for storperfJingLu52-1/+14
JIRA: YARDSTICK-692 add the URL for the test results DB to the storperd_admin-rc TEST_DB_URL=http://testresults.opnfv.org/test/api/v1 Also, add metadata JSON to the POST to /job API when creating the job: Change-Id: Ib950f759d80e707bc49d9015bc120d985051f131 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-26Merge "increase line length to 99"Ross Brattain1-0/+1
2017-06-26Bugfix: remove double quote around SECURE variableJingLu52-17/+17
JIRA: YARDSTICK-689 This patch remove the double quote around the SECURE varibale as it may cause the fowllowing error: $ openstack "${SECURE}" image list openstack: ' image list' is not an openstack command. See 'openstack --help'. Did you mean one of these? access token create address scope create address scope delete address scope list address scope set address scope show aggregate add host aggregate create ... Change-Id: Ibb22e85b3dd89b0e3b62821bd5ebe4f155886ffc Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-25Add API to update hosts info about SUTrexlee87762-0/+14
JIRA: YARDSTICK-674 For some SUT which use domain name as endpoint and AUTH_URL, yardstick restAPI should support to add "ip domain_name" info into /etc/hosts so that it can use the ip_address to access the SUT. api: /yardstick/env/action description: update hosts info for domain name method: POST parameters: { "action": "update_hosts", "args": {"opnfv.org": "1.1.1.1", "openstack.org": "2.2.2.2" } } Change-Id: Iaca9c846f02b1d53e2408d6a21f9201b599717d4 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-24Merge "Yardstick output format unified"Jing Lu15-298/+299
2017-06-24Add API to upload pod.yaml filechenjiankun3-3/+30
JIRA: YARDSTICK-687 We need a API to upload a pod.yaml file to /etc/yardstick/pod.yaml. API: /yardstick/env/action method: POST param: { 'action': 'upload_pod_file', 'file': file object } Change-Id: I3d25df364da10aaf34f995e948e1704235a40f6f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-24Merge "Improve Yardstick user-guide"Jing Lu1-1/+7
2017-06-24Add API to update openrc variablechenjiankun2-4/+38
JIRA: YARDSTICK-651 Currently we source openrc variable manually or use yardstick env prepare to get openrc file. We need API to update the openrc variable. api: /yardstick/env/action description: source environment variable method: POST parameters: { 'action': 'update_openrc', 'args':{ 'openrc': { 'OS_USERNAME': 'admin', 'OS_PASSWORD': 'console', 'OS_TENANT_NAME': 'admin', 'OS_AUTH_URL': 'http://192.168.131.222:5000/v2.0', 'EXTERNAL_NETWORK': 'ext-net' } } } Change-Id: I680a7249116c8ff0c1a9e7d5089538d935240c80 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-23Merge "improve git code with devstack functions"Ross Brattain1-19/+67
2017-06-23Improve Yardstick user-guideJingLu51-1/+7
This patch adds a short description on how to use the "yardstick env prepare" command in a Non-OPNFV installer environtment. Change-Id: Idf54bf5318499d2b129a1445e0758e351945e755 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-22Merge "auto restart influxdb and grafana support"Ross Brattain1-2/+7
2017-06-22improve git code with devstack functionsRoss Brattain1-19/+67
if we aren't re-cloning we still need to make sure we update the branch/tag/commit to the version from the remote so we need to fetch and then checkout again it is more complicated. Dockerfile does a shallow clone, so we don't get all the remote ref information. To clone a random remote commit id we have to unshallow Change-Id: If15f504b6ff2cfdfc8894fd3b37e687a19616714 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-22Merge "Acquire NSB specific data from Heat."Ross Brattain11-151/+780
2017-06-22Merge "HA testcase improvement"Kubi11-25/+83
2017-06-22Merge "bugfix: storperf timeout in ci"Kubi2-1/+7
2017-06-22HA testcase improvementJingLu511-25/+83
This patch improve HA test case in the following aspects: 1. the "GeneralHA" type now will check if the target service process in the controller node. 2. support ignore server certificate 3. add debug log for recovering service failed 4. improve method to kill keystone process Change-Id: I9ae7ab54391fe41d5d7f3e4951a7ac2e3ba75968 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-22auto restart influxdb and grafana supportrexlee87761-2/+7
JIRA: YARDSTICK-686 Change-Id: If486df4d825f9d0b03eeaa77716cc2855e2f1bd0 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-22bugfix: storperf timeout in cirexlee87762-1/+7
JIRA: YARDSTICK-680 Change-Id: Ie405187b8ab085a9e4f40a7e7e7e661c94e83630 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-22Yardstick output format unifiedchenjiankun15-298/+299
JIRA: YARDSTICK-658 Currently the yardstick have three dispatcher: file, influxdb, mongodb. (influxdb using API to get result and mongodb using testAPI to get result) But their output format is different. It is hard to use. In this patch, make all dispatchers using the same data source. And make the output format of file and influxdb unified. As for mongodb, since it is related to testAPI, so I make it push data every test case. The unified output format is: http://paste.openstack.org/show/610125/ Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-22Merge "update requirements.txt for OpenStack Ocata"Ross Brattain1-54/+53
2017-06-22Merge "ping: don't split if target_vm is a dict"Ross Brattain1-1/+4
2017-06-22Merge "cleanup: remove fuel plugin since fuel@opnfv uses mcp"Ross Brattain13-556/+0
2017-06-22Merge "temp freeze releng version for stable/danube workaround"Jing Lu1-1/+2
2017-06-21increase line length to 99Ross Brattain1-0/+1
When line length is 79 I have to spend too much time manually wrapping my code into unreadable junk I can't use expressive tuple unpacking and I start shortening variables and other bad tricks to decrease the line length. We have too many levels of indent to manage with 79 class: def: if: for: etc. Change-Id: Ib2ce9cf887f9880c46815822c8d0e1a7806e53ca Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-21Merge "heat: use dict literal in init_template"Ross Brattain1-11/+18
2017-06-20temp freeze releng version for stable/danube workaroundRoss Brattain1-1/+2
releng doesn't have a branch, so we need to freeze the releng version. There is an issue with releng fetch_os_creds.sh change /home/opnfv/repos/releng/utils/fetch_os_creds.sh: line 75: BRANCH: unbound variable So we need to revert back to commit before this change Change-Id: Ie8b0f68879a873aab584aa32b576d8b5ba602a13 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-20ping: don't split if target_vm is a dictRoss Brattain1-1/+4
If we run sample/ping-hot.yaml, it will encounter an AttributeError, log see below: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/ping.py", line 94, in run target_vm_name = target_vm.split('.')[0] AttributeError: 'dict' object has no attribute 'split' Because here host and target will be a dict JIRA: YARDSTICK-561 Change-Id: I4b7628bf20050d6d516a80efe3785f750d27c05e Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-20Acquire NSB specific data from Heat.Edward MacGillivray11-162/+798
First we add mac_address, subnet_cidr to Heat template outputs Then we convert those into a form for NSB and add vld_id. NSB also requires PCI Bus ID, kernel driver and dpdk_port_num. We get this by ssh-ing into instance and dumping sysfs We also need to fix allow for ssh key auth, and implement relative path file loading so NSB can find all its YAML files JIRA: YARDSTICK-580 Change history: don't hide heat create tracebacks we need tracebacks for debug vnf_generic: add task_path to scenario so we can load relative paths for vnf_generic we want to be able to load yaml relative to the task path For example: traffic_profile: ../../traffic_profiles/fixed.yaml topology: ping_tg_topology.yaml # TODO: look in relative path where the tc.yaml is found These need to be relative to samples/vnf_samples/nsut/ping/tc_ping_heat_context.yaml Add a scenario["task_path"] entry heat: log actual exception vnf_generic: replace list with set and iterate over values() some general refactors to remove redundact lookups and type conversions heat: provide mac_address, device_id and network_id from outputs We may need more information to dynamically determine test topology. Towards this end return more info in the heat template. We can return mac_address, device_id and network_id. Once we have this info we can add it to the context_cfg as an interfaces dict. add sample vnf ping multi-network test this test requires 3 network, one for mgmt and the other two for NSB traffic tests We have to make sure we don't use DPDK on mgmt interface because DPDK unbinds the driver heat: convert networks to OrderedDict so we can lookups networks as well as iterate over them in consisitent order heat: and vld_id to networks for vnf_generic vnf_generic uses vld_id Virtual Link Descriptor ID to identify interfaces Add the key to the networks dict and store in Networks object implement relative path file loading in vnf_generic in multiple places we need to load a file relative to the task path, so add open_relative_file_path and modify load_vnf_model to include the scenario_cfg parameter so we have access to task_path DRAFT: heat timeout support Heat stack in CI job failed due to some Nova issue. But then apparently yardstick kept running and took 180mins to timeout https://build.opnfv.org/ci/view/bottlenecks/job/bottlenecks-compass-posca_stress_ping-baremetal-daily-master/16/console We can add a Heat create timeout and fail faster if there is an error. The question is how long should we wait for a Heat stack to deploy. We can set a default and allow override in the heat context config, if users make complicated stacks heat: get netmask and gateway from heat outputs we have do some tricky business with finding the subnet cidr and converting it into netmask vnf_generic: get vpci, driver and dpdk_port_num use a big old find command to dump all the sysfs netdev info nicely. This was re-used from autotest FCoE tests. r"""find /sys/devices/pci* -type d -name net -exec sh -c '{ grep -sH ^ \ +$1/ifindex $1/address $1/operstate $1/device/vendor $1/device/device \ +$1/device/subsystem_vendor $1/device/subsystem_device ; \ +printf "%s/driver:" $1 ; basename $(readlink -s $1/device/driver); } \ +' sh \{\}/* \; This finds all PCI devices that are network devices, then dumps all the relevant info using /bin/sh. Then we parse this into a 'netdevs' dict inside the node_dict and also convert into VNF fields we need. vnf_generic: set node name for kpis node is a dict, so we have to use node_name vnfdgen: we CANNOT use TaskTemplate.render because it does not allow for missing variables, we need to allow password for key_filename to be undefined remove default ssh password hack, once rendering is fixed add new example tc_external_ping_heat_context Change-Id: If1fe0c1a2ab0a5be17e40790a66f28f706fa44d6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-06-20Configure Yardstick flavor to give more RAM to ARM instancesAlexandru Nemes1-1/+1
Yardstick TC005 fails with errors that indicate low memory conditions: "fio: error while loading shared libraries: librbd.so.1: cannot map zero-fill pages" Changing the yardstick flavor to give 1024MB of RAM instead of 512MB made this test case work. JIRA: ARMBAND-275 Change-Id: I2193802082c28edb20004964d2df3780eb5eccb7 Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
2017-06-19heat: use dict literal in init_templateRoss Brattain1-11/+18
prefer dict literals when possible over increment assignment. Change-Id: Ia40bbc04eed61feb036a1e9ec3b3110b03c095e1 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-15cleanup: remove fuel plugin since fuel@opnfv uses mcpGuo Ruijing13-556/+0
Change-Id: Ic9fb9db5c5af517cec1e60740ce5a8fe0572c5bf Signed-off-by: Guo Ruijing <ruijing.guo@intel.com>