aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/common
AgeCommit message (Collapse)AuthorFilesLines
2017-07-17Add API(v2) to create influxdbchenjiankun1-0/+11
JIRA: YARDSTICK-726 API: /api/v2/yardstick/containers/action METHOD: POST PARAMS: { 'action': 'create_influxdb', 'args': { 'environment_id': environment_id } } Change-Id: I20961fc6ae9918c894de7fabd3f81ca966bd24cd Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-17Kubernetes (k8s) supportchenjiankun3-0/+159
JIRA: YARDSTICK-682 We decide to support k8s in E release. We need to discuss with openretriver team and then rewrite the ping test case under k8s as the first step. Change-Id: I3f81ebca8de5c1f3a8b7d42581cd7342dc320239 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-14Add API(v2) to upload openrcchenjiankun1-0/+2
JIRA: YARDSTICK-719 API: /api/v2/yardstick/environments/openrcs/action METHOD: POST PARAMS: { 'action': 'upload_openrc', 'args': { 'file': file, 'environment_id': environment_id } } Change-Id: If367904a2d0c2d4a192fdc87a8da21ac6549269e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-13Add API to get environmentschenjiankun1-0/+10
JIRA: YARDSTICK-716 API: /api/v2/yardstick/environments METHOD: GET Change-Id: I46b7fb2b143fe76b6a0edbf1ecc8281187b85918 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-12Change endpoint to url in APIchenjiankun1-0/+1
There are come conflict with new architecture. Also it is more proper to use url as endpoint. Change-Id: Idce331e36d85d0720bbf3255dca29ba6022e18b2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-12constants: default SERVER_IP to 127.0.0.1Ross Brattain1-1/+9
When running unittest while connected to P2P VPN, their is no default route, so we get None. Instead of using None, default to 127.0.0.1 Not sure why this was renamed to SERVER_IP either, what do we use this value for? Still getting py.test unittest failures all the time. The problem is we lookup influxdb.ip but SERVER_IP is None, so the lookup fails. INFLUXDB_IP = get_param('influxdb.ip', SERVER_IP) SERVER_IP can't ever be None Change-Id: Iedb09dc541137f7cdc6ef8d26dd312807985bfa8 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-11Yardstick API architecture improvementchenjiankun1-2/+22
JIRA: YARDSTICK-710 Since we have the plan to upload api v2 and gui. We need to add put and delete method. So the architecture need to be improved. Change-Id: Ie20a79c26ef6c581897ce4e63980fa2895b162d2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-06move flatten dict key to common utilsrexlee87761-0/+23
So it can easily be used by other testcase to unify result JIRA: YARDSTICK-702 Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-07-05Merge "Use "OS_INSECURE" variable as the insecure mode indicator"Kubi1-1/+2
2017-06-30Use "OS_INSECURE" variable as the insecure mode indicatorJingLu51-1/+2
Now we use "OS_CACERT" as the insecure mode indicator, it is better to use "OS_INSECURE". Change-Id: I1406193e27510390b4b8fd8f4751d8361560172f Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-30Add API to show test case documentationchenjiankun1-0/+1
JIRA: YARDSTICK-694 We need API to show test case documentation. API: /yardstick/testcases/<testcase_name>/docs method: GET example: http://192.168.131.2:8888/yardstick/testcases/opnfv_yardstick_tc002/docs Change-Id: Ib8d591f0ff5f91c4d0a740539727ec73ddd86249 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-29Merge "Call core code directly in the API of run test case"Rex Lee1-0/+3
2017-06-29Call core code directly in the API of run test casechenjiankun1-0/+3
JIRA: YARDSTICK-688 We need to call core code directly in the API of runTestCase. It would be more stable. Change-Id: I431a85ded7cd3b20da0462f947c25d91bb99decd Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-25Add API to update hosts info about SUTrexlee87761-0/+1
JIRA: YARDSTICK-674 For some SUT which use domain name as endpoint and AUTH_URL, yardstick restAPI should support to add "ip domain_name" info into /etc/hosts so that it can use the ip_address to access the SUT. api: /yardstick/env/action description: update hosts info for domain name method: POST parameters: { "action": "update_hosts", "args": {"opnfv.org": "1.1.1.1", "openstack.org": "2.2.2.2" } } Change-Id: Iaca9c846f02b1d53e2408d6a21f9201b599717d4 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-24Add API to upload pod.yaml filechenjiankun1-0/+1
JIRA: YARDSTICK-687 We need a API to upload a pod.yaml file to /etc/yardstick/pod.yaml. API: /yardstick/env/action method: POST param: { 'action': 'upload_pod_file', 'file': file object } Change-Id: I3d25df364da10aaf34f995e948e1704235a40f6f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-24Add API to update openrc variablechenjiankun1-0/+3
JIRA: YARDSTICK-651 Currently we source openrc variable manually or use yardstick env prepare to get openrc file. We need API to update the openrc variable. api: /yardstick/env/action description: source environment variable method: POST parameters: { 'action': 'update_openrc', 'args':{ 'openrc': { 'OS_USERNAME': 'admin', 'OS_PASSWORD': 'console', 'OS_TENANT_NAME': 'admin', 'OS_AUTH_URL': 'http://192.168.131.222:5000/v2.0', 'EXTERNAL_NETWORK': 'ext-net' } } } Change-Id: I680a7249116c8ff0c1a9e7d5089538d935240c80 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-02utils: make module import saferRoss Brattain1-1/+4
If a module fails to import, then ignore it and try the next module. This can help if certain module depenencies aren't installed, e.g. TREX. Change-Id: I2bc4384429fabd61cd430817489cb2f3c86fe9b2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-05-11Merge "Extend TC008 to run pktgen-dpdk inside VM Need a fast path inside VM ↵Rex Lee1-0/+19
to verify full throughput of SRIOV and OVS-dpdk."
2017-05-01Extend TC008 to run pktgen-dpdk inside VMJing Zhang1-0/+19
Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk. Update 1: Change newly added file names to avoid conflict Update 2: Add more unit test cases Update 3: Fix default parameter typo for testpmd Update 4: Adapted to the pktgen-dpdk prompt change from "Pktgen>" to "Pktgen:/>", now just expect "Pktgen" Update 5: Per comment, merge common functions between latency and throughput tests to utils.py Update 6: Per comment, seperate the test case from TC008 to a new test case TC077 Change-Id: I1f7471d4ba77636a3a66c79c2652578321312185 JIRA: YARDSTICK-614 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-04-26Bugfix: fix multi-region supportRoss Brattain1-1/+5
When running with multiple regions we need to specify the region when we select the endpoint. the region is specified in the OS_REGION_NAME env variable Change-Id: I37853dd42f6d9013d4475b5e3b2b9f97b6bdc8d0 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-04-20Yardstick virtualenv supportchenjiankun1-1/+1
JIRA: YARDSTICK-620 In the patch, I separate python-setuptools from install.sh and separate appdirs from requirements.txt because there will be error when install them in virtualenv. Also I will update the documentation when this patch merged; Change-Id: I99a532d1851983775869e7c5e8e65fd0229e86df Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-04-01Yardstick virtualenv supportchenjiankun2-58/+79
JIRA: YARDSTICK-620 Currently we recommend using docker to run yardstick. And it is hard to use virtualenv to install yardstick. So I modify install.sh in yardstick root path. It will support using virtualenv to install yardstick(including API) in linux. In this patch, I make yardstick support read yardstick configuration have priority over constants. Change-Id: I9ea1241b228532a6497451e6c8f232173ddb783e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-30Bugfix: cacert should suppot insecure mode if neededrexlee87761-0/+1
currently, we use env variable "OS_CACERT" to specify the location of cert file. but if there's no cacert and use for personal verification, it's better to also support insecure mode by set OS_CACERT=False JIRA: YARDSTICK-616 Change-Id: I73b0860863bf6386f9114328c52594ef87e02fa3 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-03-17Yardstick: User interface for Yardstick.rajesh_4k2-0/+135
Currently Yardstick doesnt have any UI which gives detail analysis of the test-results. This commit generates a HTML page after the execution of a command "yardstick report generate <task-ID> <TC-name>" which intern can be executed after the execution of test-case. Used: Highcharts.js for the graphs. JIRA: YARDSTICK-280 Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
2017-03-10Bugfix: yardstick https supportchenjiankun1-15/+6
JIRA: YARDSTICK-587 When run in https environment, there is a bug: error: failed to deploy stack: '_init_() got an unexpected keyword argument 'ca_cert'' The reason is the key pass to Session() is cacert, but the key should be verify. Change-Id: Ia9fc1d7908c2fca9d827a5f64deac7cd333d5c07 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Merge "Bugfix: don't use jsonutils.load, use loads()"Rex Lee1-1/+3
2017-03-09Merge "Record test case names when run a task using API"Rex Lee1-0/+2
2017-03-08Bugfix: yardstick will create stacks with the same name when run using API ↵chenjiankun1-0/+17
in parallel JIRA: YARDSTICK-575 Currently yardstick will create stacks with the same name when run using API in parallel. The reason is there is a global variable in context base and the core will always deploy the first context in Context.list. When run in parallel, it will run in the one process. So yardstick will deploy stacks with the same name. The solution is do not use Context.list in yardstick core. And using a local variable instead. BTW, if we use API to call yardstick core, we can not config the output way. So I parse yardstick.conf when task start. And I think we can include scenario_cfg, context_cfg, yardstick_cfg in one config object later so that we can get all config in one object. Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-08Bugfix: don't use jsonutils.load, use loads()Ross Brattain1-1/+3
It looks like jsonutils.load uses a codec reader to read from the file, but we already are using Python 3.5 open() which should already do the UTF-8 decode itself. return json.load(codecs.getreader(encoding)(fp), **kwargs)C When we use jsonutils.load() we get a TypeErorr when concating str and bytes 2017-02-28 16:42:38,431 yardstick.cmd.commands.task task.py:61 ERROR Traceback (most recent call last): File "yardstick/yardstick/cmd/commands/task.py", line 58, in do_start self._finish() File "yardstick/yardstick/cmd/commands/task.py", line 69, in _finish result = read_json_from_file(self.output_file).get('result') File "yardstick/yardstick/common/utils.py", line 136, in read_json_from_file return jsonutils.load(f) File "yardstick_venv3/lib/python3.5/site-packages/oslo_serialization/jsonutils.py", line 241, in load return json.load(codecs.getreader(encoding)(fp), **kwargs) File "/usr/lib/python3.5/json/__init__.py", line 265, in load return loads(fp.read(), File "yardstick_venv3/lib/python3.5/codecs.py", line 497, in read data = self.bytebuffer + newdata TypeError: can't concat bytes to str So switch back to jsonutils.loads() which does nothing if the bytes are already decoded. JIRA: YARDSTICK-584 Change-Id: I36acfda3df2b46d16a87f2741a04fe7ee8e8d89b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-24Add common method of openstack in openstack_utilschenjiankun2-22/+343
JIRA: YARDSTICK-569 We have much redundancy code when using nova/neutron/glance... python client. So I write this code in openstack_utils for calling. Change-Id: I322b7577de4933246a15e6742ae5a28bea16eb02 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-23Record test case names when run a task using APIchenjiankun1-0/+2
JIRA: YARDSTICK-509 Currently we use influxdb as database and will not record test case name when run a test suite. So if we must offer test case name if we want to get result from API. I will record test case name when run a test suite. So we needn't offer test case any more when call for get result API. Change-Id: I87ea4770422cbc46aa6671bfcc53bd4498825eef Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-16Merge "Scenario is reporting result for last test only"Rex Lee1-0/+5
2017-02-16Merge "pylint fixes: remove redundant parens, fix comparison order"Rex Lee1-2/+2
2017-02-15Scenario is reporting result for last test onlychenjiankun1-0/+5
JIRA: YARDSTICK-548 If executing Yardstick scenario with multiple tests, result to file is reported only for the last test. In attachment screen output and yardstick.out from following command yardstick task start --suite tests/opnfv/test_suites/opnfv_smoke.yaml --output-file /mnt/log/yardstick.out Change-Id: I8aa446b284dca1bbd1667de2f476ddadfad1337f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-14Merge "more logging fixes"Rex Lee1-1/+1
2017-02-14Merge "Bugfix: write_json_to_file: use json.dump with files"Jing Lu1-1/+2
2017-02-07more logging fixesRoss Brattain1-1/+1
don't use .format() with logging, use regular %s logginer formatter Change-Id: I1ce0d81cc3f81c35003ef453e82c57faeb46c49f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08pylint fixes: remove redundant parens, fix comparison orderRoss Brattain1-2/+2
removed redundant parens in if and while clauses use var != constant, not constant != var. Python doesn't allow for assignment in if statements, so we don't have to use the old C workarounds remove unwanted commas use raw strings for regexps with backslashes, e.g. r'\s' instead of '\s' Change-Id: I7aad645dd3d7f4b4b62f4e4510a425611c9d28f2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08Bugfix: write_json_to_file: use json.dump with filesRoss Brattain1-1/+2
In python3 file objects automatically handle encoding to utf-8 and expect string, not bytes, so use regular json.dump() with the file object We should only use dump_as_bytes to replace json.dumps(), not to replace json.dump() This fixes Python3 issue: Traceback (most recent call last): File "yardstick/main.py", line 52, in <module> main() File "yardstick/main.py", line 49, in main YardstickCLI().main(sys.argv[1:]) File "yardstick/yardstick/cmd/cli.py", line 167, in main self._dispath_func_notask() File "yardstick/yardstick/cmd/cli.py", line 145, in _dispath_func_notask func(CONF.category) File "yardstick/yardstick/cmd/commands/task.py", line 48, in do_start self._init_result_file() File "yardstick/yardstick/cmd/commands/task.py", line 57, in _init_result_file write_json_to_file(consts.DEFAULT_OUTPUT_FILE, data) File "yardstick/yardstick/common/utils.py", line 152, in write_json_to_file write_file(path, jsonutils.dump_as_bytes(data), mode) File "yardstick/yardstick/common/utils.py", line 157, in write_file f.write(data) TypeError: write() argument must be str, not bytes Change-Id: I573419be25d8fa1f015e1507730ba66c05f86686 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-06Push yardstick debug log into the artifactsrexlee87761-0/+2
the log directory should be specific yardstick so it can be easily pushed into artifacts with the directory. JIRA: YARDSTICK-507 Change-Id: If28b0c3938e123c8a55272cb98291fb76aaf6f9e Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-01-23Merge "Record task status if running via CLI"Rex Lee2-0/+12
2017-01-20Record task status if running via CLIchenjiankun2-0/+12
JIRA: YARDSTICK-542 Currently we do not record task status when using CLI to run task. So I add this function. If status=0, task is not done. if status=1, task is done. if status=2, there is an error. Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Create API to get a list of all test caseschenjiankun1-0/+2
JIRA: YARDSTICK-456 Currently we do not have a API to get a list of all test cases; Currently the test case info is from the comment; So I create a API to get a list of all test cases; And create a 'description' attribute to record info of a test case; And use the CLI call this API; Change-Id: Ife800600446683664097835c7b9f11899c85771d Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-19Merge "Add unittest framework for Yardstick API"Jing Lu1-0/+2
2017-01-17Add unittest framework for Yardstick APIchenjiankun1-0/+2
JIRA: YARDSTICK-538 Currently it is hard to test API, So I add a base class as flask document do. In this framework I will mock a temp sqlite database and a server. Change-Id: If881233cb22655617c07ad018201b8ee08492d06 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-16Use """ to replace ''' in docstringchenjiankun1-2/+2
JIRA: YARDSTICK-525 For consistency, we always use """triple double quotes""" around docstrings. Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain6-11/+23
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-10Add API to get the status of async taskchenjiankun2-1/+7
JIRA: YARDSTICK-526 Currently there are many API run a task using sub thread. But we don't know the status of this task. So we need to offer a API to query the status of this task. Change-Id: I8d2cc558750bf9270aed4a7abb8bf35d17894d83 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-30Merge "BugFix: correct Copyright info in openstack_utils.py"Jing Lu1-1/+0
2016-12-27BugFix: correct Copyright info in openstack_utils.pyJingLu51-1/+0
JIRA：YARDSTICK-513 the copyright info in openstack_utils.py is not accurate. (yardstick: this file is copied from rally and slightly modified) this line should be deleted. Change-Id: I6cf8205ee6b142ae87b268829e5ac7454239dd0d Signed-off-by: JingLu5 <lvjing5@huawei.com>