aboutsummaryrefslogtreecommitdiffstats
path: root/tests/unit/benchmark/core
AgeCommit message (Collapse)AuthorFilesLines
2017-10-06fix random test_task unittest fail due to os.environ mockRoss Brattain1-10/+13
sometime Jenkins fails due to what I guess are concurrency problems in os.environ mock ====================================================================== FAIL: tests.unit.benchmark.core.test_task.TaskTestCase.test_parse_suite_with_constraint_no_args tags: worker-10 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python3.5/unittest/mock.py", line 1157, in patched return func(*args, **keywargs) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-euphrates/tests/unit/benchmark/core/test_task.py", line 208, in test_parse_suite_with_constraint_no_args task_files, task_args, task_args_fnames = t.parse_suite() File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-euphrates/yardstick/benchmark/core/task.py", line 455, in parse_suite cur_pod = os.environ.get('NODE_NAME', None) File "/usr/lib/python3.5/unittest/mock.py", line 917, in __call__ return _mock_self._mock_call(*args, **kwargs) File "/usr/lib/python3.5/unittest/mock.py", line 976, in _mock_call result = next(effect) StopIteration Ran 1262 tests in 2.375s FAILED (id=0, failures=1) error: testr failed (1) + testr failing Replace the mock decorator with a context manager to try to reduce the scope and duration of the mock. Change-Id: I342fe6c403e66c53ac4c39fd88fa9047cdfae5d9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 2fadf7aec9e2761c39d29d8af1ee7d69d154652d)
2017-09-19NSB: count from 0, vnf_0, tg_9, private->uplink, public->downlinkRoss Brattain1-12/+12
Also rename private to uplink, public to downlink for scale-out template we need to count from 0 so we can use range() without +1/-1 errors vnf_0, vnf_1 tg_0, tg_1 also fix Ixia defaults Change-Id: I6aecfbb95f99af20f012a9df19c19be77d1b5b77 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-09-13Heat: support non-mesh network toplogyRoss Brattain1-28/+24
Previsouly we added all servers to every network in Heat in a full mesh. To more closely replicate test topology and to limit then number of ports we need to all each server to specify which ports should be connected to each network. This should also allow for some kind of multiport setup. Add optional network_ports dict to each server with network to port_list mapping match inteface based on port name or vld_id replace vld_id matching with network name matching, since network_name == vld_id Change-Id: I5de46b8f673949e3c17d8df6fa96f055c43886ce Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-13utils: create TASK_LOG_DIR if it doesn't existRoss Brattain1-1/+2
Change-Id: I10630599065e6c159b137bd4d6ee89353a1c411b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-11Add real time log view in GUIchenjiankun1-1/+9
JIRA: YARDSTICK-775 We have GUI now, but we can't see real time log in GUI view. So I add real time log view in GUI. Change-Id: Ie83f327ef0a94302afa6b3def764fec6ef5818d1 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-27Adding support for multi-dispatcherDeepak S1-0/+14
JIRA: YARDSTICK-684 This patch adds the support to enable multi-dispatcher of result. e.g. [DEFAULT] debug = False dispatcher = file, http Change-Id: Id0e1a1bba14edd899fcf5b275be1f0a091b6db77 Signed-off-by: Deepak S <deepak.s@linux.intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-13add network info to topologyRoss Brattain1-0/+67
we need to know which network each port is connected to so we can find VLAN or VXLAN ID. To do this we implement a new method for Contexts, Context.get_network(). This method is similar to Context.get_server(), it searches for a given network name in all the contexts. From this we generate a context_cfg["networks"] dict that stores all the network info for the nodes in the scenario. Then when we generate the topology for VNFD, we can lookup a given network by the vld_id and get the network_type, segmentation_id, etc. Then if we need to for example generated traffic on a given VLAN or VXLAN, we have this info available. Define default nd_route_tbl for ACL VNF we need default empty nd_route_tbl for IPv6 route. Change-Id: I9f9cfbd6acabeb4ae4675ca7354390efa57b29e7 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-06-22Yardstick output format unifiedchenjiankun1-0/+1
JIRA: YARDSTICK-658 Currently the yardstick have three dispatcher: file, influxdb, mongodb. (influxdb using API to get result and mongodb using testAPI to get result) But their output format is different. It is hard to use. In this patch, make all dispatchers using the same data source. And make the output format of file and influxdb unified. As for mongodb, since it is related to testAPI, so I make it push data every test case. The unified output format is: http://paste.openstack.org/show/610125/ Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-05-26Pass parameters between scenarioschenjiankun1-0/+28
JIRA: YARDSTICK-641 Allowing parameters to pass between scenarios so that the one test case can be combination of several scenarios. Change-Id: I55a00855e77d5b719a27a069a3ea195d6bbd0ef8 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-04-11standardize ssh authRoss Brattain1-4/+4
we need to be following defautl paramiko rules, first use pkey, then key_filenames (autodetecting ~/.ssh/ keys), then password We have too much boilerplate redudant code everywhere, we need to standardize on a factory function that takes a node dict. Using Python3 ChainMap we can layer overrides and defaults. VNF descriptors have to default key_filename, password to Python None. The only way to do this is to omit key values if the variable is not defined, this way the dict will not have the value and it will default to Python None Add python2 chainmap backport Updated unittest mocking to use ssh.SSH.from_node Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-17Yardstick: User interface for Yardstick.rajesh_4k1-0/+72
Currently Yardstick doesnt have any UI which gives detail analysis of the test-results. This commit generates a HTML page after the execution of a command "yardstick report generate <task-ID> <TC-name>" which intern can be executed after the execution of test-case. Used: Highcharts.js for the graphs. JIRA: YARDSTICK-280 Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
2017-02-17Update missing license headersDeepak S4-0/+32
Change-Id: I063fd37fe25754c94d164ae5a209d15b69322093 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-15Bugfix: AttributeError: 'dict' object has no attribute 'encode'chenjiankun1-0/+24
JIRA: YARDSTICK-557 If we run task with sample/ping.yaml We will encounter below error, here is the log: Traceback (most recent call last): File "/usr/local/bin/yardstick", line 11, in <module> load_entry_point('yardstick==0.1.dev0', 'console_scripts', 'yardstick')() File "/usr/local/lib/python2.7/dist-packages/yardstick/main.py", line 49, in main YardstickCLI().main(sys.argv[1:]) File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/cli.py", line 167, in main self._dispath_func_notask() File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/cli.py", line 145, in _dispath_func_notask func(CONF.category) File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/commands/task.py", line 45, in do_start Task().start(param, **kwargs) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 83, in start self._run(scenarios, run_in_parallel, args.output_file) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 131, in _run runner = run_one_scenario(scenario, output_file) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 410, in run_one_scenario if is_ip_addr(scenario_cfg["target"]): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 358, in is_ip_addr ipaddress.ip_address(addr.encode('utf-8')) AttributeError: 'dict' object has no attribute 'encode' Change-Id: Iba1570416bd8614e38c9e847de730a31d9ddedc2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Create API to get a list of all test caseschenjiankun1-2/+2
JIRA: YARDSTICK-456 Currently we do not have a API to get a list of all test cases; Currently the test case info is from the comment; So I create a API to get a list of all test cases; And create a 'description' attribute to record info of a test case; And use the CLI call this API; Change-Id: Ife800600446683664097835c7b9f11899c85771d Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-14Bugfix:can not run a test suite if not under yardstick root pathchenjiankun1-16/+20
JIRA: YARDSTICK-530 Currently we can not run a test suite if we are not in yardstick root path. The reason is that the file in test suite config file use relative path. So I change it to absolute path when run. Change-Id: I62758bc67f466ac794d339b597562b3be05574fb Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain3-16/+37
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-12-30Split Yardstick CLI with Yardstick core logicchenjiankun8-0/+356
JIRA: YARDSTICK-511 We need to unify yardstick entry. Now the solution is using CLI call API as nova do. This is the first step: coupling the yardstick core logic from CLI. Moving the core logic to yardstick/benchmark/core and the CLI using a object to call yardstick core logic. Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>