Age | Commit message (Collapse) | Author | Files | Lines |
|
Update naming in related files, i.e. pod, docs, docstrings.
Rename TGs and VNFs to follow a single naming convention, for all VNFs:
name: trafficgen_0
role: tg__0
name: vnf_0
role: vnf__0
JIRA: YARDSTICK-1578
Change-Id: I7596b915939f242349a99721b6088ae738dd277f
Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
|
|
JIRA: YARDSTICK-1593
Change-Id: I3ec352dd577c6030fa86a2817e264ba7c80773f1
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
|
|
* changes:
Additional rework of NSB report
Only show metrics for selected nodes in data table
Fix conversion to JS for HTML reports
|
|
|
|
- Make format_for_jstree expect a list of metric names
- Avoid displaying timestamps twice in initial data table
- Sort metrics in initial data table
- Display testcase name
- Fix styling
- Make better use of JS and jQuery features
- Move event handler to JS file
- Avoid adding multiple tbody elements into data table
- Adjust unit tests and functional tests accordingly
JIRA: YARDSTICK-1367
Topic: report/html_table (12 of 12)
Change-Id: I85d853f8e392953cace67e94fa0af2e2492a2b86
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
|
|
Some new fields in InfluxDB have plain text contents that AST cannot
parse as valid expressions, resulting in exceptions being raised by
yardstick CLI commands "report generate" and "report generate-nsb".
Reworked _generate_common() to properly handle and convert:
- unicode, str and missing keys
- None, unicode, str, float, long and int values
- float, long and int stored as unicode or str values
Added test__generate_common() unit test, to verify all possible
conversion flavors.
Also renamed incorrect "tasks" to "metrics".
Also fixed warning in Python3 for some unit tests, caused by
deprecation of unittest.TestCase.assertRaisesRegexp().
JIRA: YARDSTICK-1367
Topic: report/html_table (10 of 12)
Change-Id: Iff75bfd2c1dcaf3806f67f52f0ea594f10aceb5b
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
|
|
JIRA: YARDSTICK-1367
Topic: report/html_table (8 of 12)
Change-Id: I8a2f96224a745334b67dd71875bebb3b69b9adc3
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
|
|
This JavaScript library is available under MIT license.
Also adjusted version for jQuery and jsTree,
and added fallback font-family names.
JIRA: YARDSTICK-1367
Topic: report/html_table (6 of 12)
Change-Id: Ibe8b7e3d1f1365d2cbc019bfc22762aaa365a4e1
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
|
|
JIRA:YARDSTICK-1557
Change-Id: I54ee4d6ef835e8e6ecd55cfa305151e842f88b03
Signed-off-by: cope.li <cope.li@huawei.com>
|
|
Allow the user to select what data to show in the report using
jsTree to navigate a hierarchical metrics list.
JIRA: YARDSTICK-1367
Topic: report/html_table (4 of 11)
Change-Id: I86d782a0a70b80a1cdfaab2f41afb7668066cbf7
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
|
|
``$ yardstick report generate ...`` now renders a template stored
in a file instead of from an imported string
Instead of using Django, templating is done by jinja2, as this is
used in other parts of Yardstick.
JIRA: YARDSTICK-1367
Topic: report/html_table (2 of 11)
Change-Id: Iaff53e7e28903e46164ce0977f6b8adbe04d23d7
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
|
|
* Remove redundant imports
* Rename some constants
* Add additional unit tests
* Reword KeyError message
JIRA: YARDSTICK-1367
Topic: report/html_table (1 of 11)
Change-Id: I3d89c1ed9acaeb7421a0734e33075ebae252c035
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
|
|
Make context types constants. These are the current context types:
- Dummy
- Heat
- Kubernetes
- Node
- StandaloneOvsDpdk
- StandaloneSriov
Any reference to a context type should use the referred constant.
JIRA: YARDSTICK-1292
Change-Id: I1ff0085161f88304d71e3ddad7975f9020e0fc20
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
|
|
This feature will provide Kubernetes context the ability to handle the
context name inside the class itself, providing to the developer an
abstraction of the possible naming limitations in Kubernetes. E.g.:
"dot" character in Pod names is no allowed [1].
[1] https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/58
JIRA: YARDSTICK-1188
Change-Id: I82121f970b550170357a443b6340be7900602a57
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
|
|
Now, yardstick HA test cases (which have 'sla action: monitor') do
not output results if the test case SLA validation fails.
This patch modifies the task runner and the serviceHA scenario so
that during an SLA validation failure the test result data is
captured correctly.
Do similar changes in the general scenario.
JIRA: YARDSTICK-1115
Change-Id: I648a8b229600c0ad089320ac3f803698f73aa800
Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
|
|
|
|
JIRA: YARDSTICK-1080
This bug will only occur in kubernetes test case.
In openstack, we use 'stack_name.context', but some kubernetes
resources(pod, etc) do not support this format. So we use
'name-context'.
Change-Id: I87d61c027d55449914bc283672eb98be3d3fe22f
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
In [1] a new method to rename the "scenario" host names was implemented.
This method parses all sections with host names and applies the
"qualified" name.
Some malformed test cases define the "options" section without content.
This causes that [2] retrieves "None" value instead of an empty
dictionary. This possibility must be handled in the new method.
[1]I44da30dac562c1a4166e084645ae91c17798651d
[2]https://github.com/opnfv/yardstick/blob/4b8b674b65830a24230faed71e8d9a1048139c89/yardstick/benchmark/core/task.py#L630
JIRA: YARDSTICK-1073
Change-Id: I8864b428734ead8c5aa39de5091d3a2a691be060
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
|
|
JIRA: YARDSTICK-1056
Change-Id: I32ab9ae7482f6b81e00d6d3e7e858d980a2dd95f
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
|
|
The command "yardstick report generate task_id yaml_name" was returning
some errors and failing to generate a html report.
With this fix an html report will be generated with all data from influx.
JIRA: YARDSTICK-1023
Change-Id: If9aab4b85cfb2b5809f608568e1adb2ddeec8b78
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
|
|
|
|
|
|
This new option provides to the user the ability to output the rendered
input files. This option could be useful in case the input files are
Jinja2 templates, depending on input arguments. The user can preview
the rendered input tasks files without executing them.
JIRA: YARDSTICK-1020
Change-Id: Ib15ade7e1adcb29beae5e635fb5d02045c4432bb
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
|
|
This change adds two optional flags to the context configuration for heat:
* no_setup -- to re-use an existing OpenStack deployment
* no_teardown -- to skip undeploying the context, so it is available in other testcases
In order to achieve this, resource naming had to be made more predictable.
When no_setup or no_teardown are set:
* Do not add a suffix to context name
* When searching, try with and without a suffix to find a server by name
All behaviors remain the same unless these values are set.
NOTE:
This feature doesn't work with the NSPerf scenario, because the
interfaces in the VNFs are bound to DPDK after the first test, so the
look-up for missing topology information fails in subsequent test
runs, as the interfaces are not visable to the kernel.
This can be addressed either by either unbinding the interfaces from
DPDK at the end of each test, or modifying the look-up for the missing
topology information to query DPDK as well.
The change will be made in a future patch.
JIRA: YARDSTICK-886
Change-Id: I6df5e86e419c283a2bee41917e4f62179aa9c31a
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
|
|
* changes:
Add _create_new_stack method
Update TaskParser to deal with qualified name in Context
Add qualified name to Context
|
|
The context name depends on the defined name in the testcase input
file, the task ID and the flags of the context.
If the context is going to be undeployed at the end of the test, the
task ID is suffixed to the name to avoid interferences with previous
deployments. If the context needs to be deployed at the end of the
test, the name assigned is kept.
This patch makes this process transparent to the developer.
This patch modifies how TaskParser determines the correct context name,
taking into account that the name might change based on context flags.
JIRA: YARDSTICK-886
Change-Id: I44da30dac562c1a4166e084645ae91c17798651d
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
|
|
JIRA: YARDSTICK-989
We used to have real-time influxdb reporting of test KPIs. The user
could monitor using grafana and see the real-time output.
The record format was changed to now only report KPIs at the end of the
test.
This is a problem for test cases which run for a long duration, we need
to wait until the end of the test execution to get any results from
influxdb. If the test fails in between or doesn't exit cleanly for some
reason, we are left with no results stored in influxdb which gives the
user no information.
This is also a regression from the previous behaviour.
Change-Id: I0f476dff9162a359f0286fb421f2e9c4befaa5cc
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
This change makes files pep8 compliant before adding the
reuse context feature
JIRA: YARDSTICK-886
Change-Id: Iae7daaa159a9ddbb5809a9a7ac74f8a53683089a
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
|
|
JIRA: YARDSTICK-855
1. using prettytable to reformat cli below:
yardstick runner list
yardstick scenario list
yardstick testcase list
2. remove redundant function print_hbar
3. fix pep8 problems, the framework using do_list dynamicly,
so disable unused-argument check for now.
Change-Id: I79e1226e21fca4624bf4436414917bd07ecabca0
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Created two new class functions for Scenario class:
* "get_description" will return the first line of the docstring,
if exists.
* "get_scenario_type" will return the value of "__scenario_type__"
variable.
Change-Id: I8e3b47e33e0bae101b874c3975bb9c383397d188
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
|
|
I'm not sure why we hardcoded config_file when
the CONF_FILE constant will work.
Change-Id: I45a1dc17ac2b185c659725f55462214acb74838d
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
We were not draining the queues in the background
runners.
Modify the main runner_join loop to loop over
all the runners and drain them.
The runner join method does extra work for periodic actions
so we can't change its behavior.
Instead create a new poll() method and use that
to check runner status
Change-Id: I9466ba40a6a4c45c82cedff279cbb4817c6b66ad
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Sometimes the runners can hang. Initially
debugging lead to the queue join thread, so I thought
we could cancel all the join threads and everything would be okay.
But it turns out canceling the queue join threads can lead
to corruption of the queues, so when we go to drain the queues
the task hangs.
But it also turns out that we were not properly draining
the queues in the task process. We were waiting for all
the runners to exit, then draining the queues.
This is bad and will cause the queues to fill up and hang
and/or drop data or corrupt the queues.
The proper fix seems to be to draining the queues in a
loop before calling join with a timeout.
Also modified the queue drain loops to no block on queue.get()
Revert "cancel all queue join threads"
This reverts commit 75c0e3a54b8f6e8fd77c7d9d95decab830159929.
Revert "duration runner: add teardown and cancel all queue join threads"
This reverts commit 7eb6abb6931b24e085b139cc3500f4497cdde57d.
Change-Id: Ic4f8e814cf23615621c1250535967716b425ac18
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-785
Change-Id: Ib37498e8df6a520f1d03256b73346fcedab3a177
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
JIRA: YARDSTICK-803
Currently kubernetes test case can only run in master node.
We need to support it run in jump server.
So I add service and use nodePort type.
Then we can login the pod using nodePort.
Change-Id: Ia7900d263f1c5323f132435addec27ad10547ef9
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
We want to generate pod.yaml from Heat contexts so we can
re-use the context without destroying it.
But we don't have node role information and it doesn't
make sense in this case, so make the role optional.
Since we changed Heat to use pkey instead of key_filename,
we can embed the pkey into the pod.yaml, but we have
to make sure to convert the pkey to string, in case
it is a RSAKey object
Change-Id: Ibefcfbd8236e68013a704c39964cb870da825da8
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
JIRA: YARDSTICK-810
Currently host, target is in scenario, but as a input, we prefer it in
scenario['options'].
So I add support for under scenario['options']['server_name']
If we write host in scenario['options']['server_name'], the host ip info
will be written in context.
Change-Id: I90df20467ef5da772d22e9f272a2cac250f822e0
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Previsouly we added all servers to every network
in Heat in a full mesh.
To more closely replicate test topology and to limit
then number of ports we need to all each server
to specify which ports should be connected to each network.
This should also allow for some kind of multiport setup.
Add optional network_ports dict to each server with network to port_list
mapping
match inteface based on port name or vld_id
replace vld_id matching with network name matching, since network_name == vld_id
Change-Id: I5de46b8f673949e3c17d8df6fa96f055c43886ce
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I744353f631cf1771d75f750543e8612f81be71ee
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
JIRA: YARDSTICK-785
Currently if one test case failed, we will log error.
But if one case success, we do not have any tips.
We need log success when one case success.
Change-Id: I0f41ac55f2569f44b787133e3f2594a5c5547f4a
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-801
We need kubernetes test case run in CI.
We have one scenario k8-nosdb-lb-noha for kubernetes.
But for now, we need to do some adoption in CI script.
Also I move check OS_AUTH_URL in heat.py.
Change-Id: Iff86caf77fd74e70469ad2dc43f2238097ce0001
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
There are multiple issues wiht YAML loading.
1. Jinja2 renders None values as a string 'None'. This is not valid YAML
we need to render None values to '~' or 'null' which is the native YAML
None value.
2. Jinja2 renders dict and lists that contain unicode with
u'foo' values. This is not value YAML syntax.
Because we are serializing dict and lists into YAML, we
need to encode them as valid YAML. We can override Jinja2 finalize to
use yaml.dump to dump inline YAML.
We use yaml.safe_dump(elem, default_flow_style=True).replace('\n', '')
to generate valid single-line YAML dict and list values.
But this problem highlights the general difficulties with templating and
loading files.
We could avoid this Python->Jinja2->YAML->Python issue by directly
injecting the list or dict after the YAML is loaded.
I'm not sure of the real utility of these templates.
3. On Python 2 YAML loader is rendering all strings
as unicode. This does not work for Trex because Trex is broken
and badly coded. Trex does type checking against str() which
is different for Python 2 and Python 3.
The default YAML loader will return native string types, str() or unicode()
for Python 2 and Python 3 respectively.
The bad Trex codes is in convert_val:
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_packet_builder_scapy.py#L674
def convert_val (val):
if is_integer(val):
return val
if type(val) == str:
return ipv4_str_to_num (is_valid_ipv4(val))
raise CTRexPacketBuildException(-11,("init val invalid %s ") % val );
This code is doing type(val) == str. This is bad and broken.
We can't fix Trex, so we have to render all strings as native str() types
The bug here was that the Heat template loader template_format.py
was overriding the global YAML loader to always return unicode.
We don't want this global override.
To fix this we have to use local subclasses of the yaml.SafeLoader
class.
But in order to dynamically subclass from CSafeLoader or SafeLoader
we have to use the type() builtin to define a new class at runtime.
Once we have new classes defined, we can safely isolate different
YAML constructors and return unicode or not depending on the case.
To be consistent we implement a new yaml_loader.py module to centralize
all non-Heat template yaml loading to ensure correct uncode/str
conversion
Change-Id: Iebf9cf78fbda390977c390436b0869e7bbf503eb
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-784
Currently we have yardstick report in GUI.
But if users do not use GUI, they can't see this report.
So we need generate a report each task.
After run each test case, we can see report:
http://ip:port/report/report.html
Change-Id: Ic76cf57f55aa6680b91272e210135136f0225373
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Change-Id: I10630599065e6c159b137bd4d6ee89353a1c411b
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-775
We have GUI now, but we can't see real time log in GUI view.
So I add real time log view in GUI.
Change-Id: Ie83f327ef0a94302afa6b3def764fec6ef5818d1
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|