| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
There are multiple issues wiht YAML loading.
1. Jinja2 renders None values as a string 'None'. This is not valid YAML
we need to render None values to '~' or 'null' which is the native YAML
None value.
2. Jinja2 renders dict and lists that contain unicode with
u'foo' values. This is not value YAML syntax.
Because we are serializing dict and lists into YAML, we
need to encode them as valid YAML. We can override Jinja2 finalize to
use yaml.dump to dump inline YAML.
We use yaml.safe_dump(elem, default_flow_style=True).replace('\n', '')
to generate valid single-line YAML dict and list values.
But this problem highlights the general difficulties with templating and
loading files.
We could avoid this Python->Jinja2->YAML->Python issue by directly
injecting the list or dict after the YAML is loaded.
I'm not sure of the real utility of these templates.
3. On Python 2 YAML loader is rendering all strings
as unicode. This does not work for Trex because Trex is broken
and badly coded. Trex does type checking against str() which
is different for Python 2 and Python 3.
The default YAML loader will return native string types, str() or unicode()
for Python 2 and Python 3 respectively.
The bad Trex codes is in convert_val:
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_packet_builder_scapy.py#L674
def convert_val (val):
if is_integer(val):
return val
if type(val) == str:
return ipv4_str_to_num (is_valid_ipv4(val))
raise CTRexPacketBuildException(-11,("init val invalid %s ") % val );
This code is doing type(val) == str. This is bad and broken.
We can't fix Trex, so we have to render all strings as native str() types
The bug here was that the Heat template loader template_format.py
was overriding the global YAML loader to always return unicode.
We don't want this global override.
To fix this we have to use local subclasses of the yaml.SafeLoader
class.
But in order to dynamically subclass from CSafeLoader or SafeLoader
we have to use the type() builtin to define a new class at runtime.
Once we have new classes defined, we can safely isolate different
YAML constructors and return unicode or not depending on the case.
To be consistent we implement a new yaml_loader.py module to centralize
all non-Heat template yaml loading to ensure correct uncode/str
conversion
Change-Id: Iebf9cf78fbda390977c390436b0869e7bbf503eb
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
yaml.safe_load is safer, obviously.
anteater will check for this
template_format use specialized constructor based on yaml.SafeLoader
JIRA: YARDSTICK-760
Change-Id: Ia3b0b3aa0765385a0ee472a4d83f49d424b5a77f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-525
For consistency, we always use """triple double quotes""" around
docstrings.
Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3
This passes unittests on Python 3.5 and passes opnfv_smoke suite
Updates:
use six for urlparse and urlopen
fix exception.message attribute removal
run unittests on python3
use unitest.mock on python 3
fix open mock for vsperf
fix float division by using delta/eplison comparison
use unicode in StringIO
use plugin/sample_config.yaml relative path from test case
fixed apexlake unittests
upgraded to mock 2.0.0 to match python3 unittest.mock features
fixed flake8 issues
implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
implement safe unicode encode/decode with oslo_utils.encodeutils
heat: convert pub key file from bytes to unicode
pkg_resources returns raw bytes, in python3
we have to decode this to utf-8 unicode
so JSON can encode it for heat template
JIRA: YARDSTICK-452
Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I14f21092f5e97abf0629a92a27062846a6d3130b
JIRA:-
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
An external HOT template is a separate yaml file in native
Heat format HOT. The external template is referenced in the task
file and used as template for a "context". Parameters required at
template instantiation are also configured in the task file.
See new sample file ping-hot.yaml
Change-Id: Ie2b7ea96ea90b75ca4e08a29e2223ceeb1474724
JIRA: YARDSTICK-24
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
Ross Brattain
chenjiankun
Jo¶rgen Karlsson
Hans Feldt