add support for Jinja2 in task file

Add support in task file for the template syntax based on Jinja2.

JIRA:YARDSTICK-101

Change-Id: I24be133ba590510612d97a1fce6c024e6edb57e4
Signed-off-by: kubi <jean.gaoliang@huawei.com>

6 files changed, 340 insertions, 6 deletions

diff --git a/docs/Yardstick_task_templates.rst b/docs/Yardstick_task_templates.rst
new file mode 100755
index 000000000..538937fd7
--- /dev/null
+++ b/docs/Yardstick_task_templates.rst
@@ -0,0 +1,141 @@
+Task Template Syntax
+====================
+
+Basic template syntax
+---------------------
+A nice feature of the input task format used in Yardstick is that it supports the template syntax based on Jinja2.
+This turns out to be extremely useful when, say, you have a fixed structure of your task but you want to
+parameterize this task in some way.
+For example, imagine your input task file (task.yaml) runs a set of Ping scenarios:
+
+::
+
+  # Sample benchmark task config file
+  # measure network latency using ping
+  schema: "yardstick:task:0.1"
+
+  scenarios:
+  -
+    type: Ping
+    options:
+      packetsize: 200
+    host: athena.demo
+    target: ares.demo
+
+    runner:
+      type: Duration
+      duration: 60
+      interval: 1
+
+    sla:
+      max_rtt: 10
+      action: monitor
+
+  context:
+      ...
+
+Let's say you want to run the same set of scenarios with the same runner/context/sla,
+but you want to try another packetsize to compare the performance.
+The most elegant solution is then to turn the packetsize name into a template variable:
+
+::
+
+  # Sample benchmark task config file
+  # measure network latency using ping
+
+  schema: "yardstick:task:0.1"
+  scenarios:
+  -
+    type: Ping
+    options:
+      packetsize: {{packetsize}}
+    host: athena.demo
+    target: ares.demo
+
+    runner:
+      type: Duration
+      duration: 60
+      interval: 1
+
+    sla:
+      max_rtt: 10
+      action: monitor
+
+  context:
+      ...
+
+and then pass the argument value for {{packetsize}} when starting a task with this configuration file.
+Yardstick provides you with different ways to do that:
+
+1.Pass the argument values directly in the command-line interface (with either a JSON or YAML dictionary):
+
+::
+
+ yardstick task start samples/ping-template.yaml --task-args '{"packetsize": "200"}'
+
+2.Refer to a file that specifies the argument values (JSON/YAML):
+
+::
+
+ yardstick task start samples/ping-template.yaml --task-args-file args.yaml
+
+Using the default values
+------------------------
+Note that the Jinja2 template syntax allows you to set the default values for your parameters.
+With default values set, your task file will work even if you don't parameterize it explicitly while starting a task.
+The default values should be set using the {% set ... %} clause (task.yaml).For example:
+
+::
+
+  # Sample benchmark task config file
+  # measure network latency using ping
+  schema: "yardstick:task:0.1"
+  {% set packetsize = packetsize or "100" %}
+  scenarios:
+  -
+    type: Ping
+    options:
+    packetsize: {{packetsize}}
+    host: athena.demo
+    target: ares.demo
+
+    runner:
+      type: Duration
+      duration: 60
+      interval: 1
+    ...
+
+If you don't pass the value for {{packetsize}} while starting a task, the default one will be used.
+
+Advanced templates
+------------------
+Yardstick makes it possible to use all the power of Jinja2 template syntax, including the mechanism of built-in functions.
+As an example, let us make up a task file that will do a block storage performance test.
+The input task file (fio-template.yaml) below uses the Jinja2 for-endfor construct to accomplish that:
+
+::
+
+  #Test block sizes of 4KB, 8KB, 64KB, 1MB
+  #Test 5 workloads: read, write, randwrite, randread, rw
+  schema: "yardstick:task:0.1"
+
+   scenarios:
+  {% for bs in ['4k', '8k', '64k', '1024k' ] %}
+    {% for rw in ['read', 'write', 'randwrite', 'randread', 'rw' ] %}
+  -
+    type: Fio
+    options:
+      filename: /home/ec2-user/data.raw
+      bs: {{bs}}
+      rw: {{rw}}
+      ramp_time: 10
+    host: fio.demo
+    runner:
+      type: Duration
+      duration: 60
+      interval: 60
+
+    {% endfor %}
+  {% endfor %}
+  context
+      ...
diff --git a/samples/fio-template.yaml b/samples/fio-template.yaml
new file mode 100644
index 000000000..940446b56
--- /dev/null
+++ b/samples/fio-template.yaml
@@ -0,0 +1,40 @@
+# Sample benchmark task config file
+# measure storage performance using fio
+# Jinja2 Syntax is supported
+# using built-in functions ( Jinja2 for-endfor construct ) to test complex tasks
+# Test block sizes of 4KB, 8KB, 64KB, 1MB
+# Test 5 workloads: 4 corners and 1 mixed :read, write, randwrite, randread, rw
+schema: "yardstick:task:0.1"
+
+scenarios:
+{% for rw in ['read', 'write', 'randwrite', 'randread', 'rw'] %}
+  {% for bs in ['4k', '8k', '64k', '1024k'] %}
+-
+  type: Fio
+  options:
+    filename: /home/ec2-user/data.raw
+    bs: {{bs}}
+    rw: {{rw}}
+    ramp_time: 10
+    duration: 20
+  host: fio.demo
+  runner:
+    type: Iteration
+    iterations: 2
+    interval: 1
+  {% endfor %}
+{% endfor %}
+
+context:
+  name: demo
+  image: yardstick-trusty-server
+  flavor: yardstick-flavor
+  user: ec2-user
+  servers:
+    fio:
+      floating_ip: true
+  networks:
+    test:
+      cidr: "10.0.1.0/24"
+      external_network: "net04_ext"
+
diff --git a/samples/ping-template.yaml b/samples/ping-template.yaml
new file mode 100644
index 000000000..3f10218ab
--- /dev/null
+++ b/samples/ping-template.yaml
@@ -0,0 +1,49 @@
+# Sample benchmark task config file
+# measure network latency using ping
+# Jinja2 Syntax is supported
+# parameterize this task, {{packetsize}} is passed to the scenario as an argument
+# If you don't pass the value for {{packetsize}} while starting a task,
+# the default one will be used.
+
+
+schema: "yardstick:task:0.1"
+{% set packetsize = packetsize or "100" %}
+scenarios:
+-
+  type: Ping
+  options:
+    packetsize: {{packetsize}}
+  host: athena.demo
+  target: ares.demo
+
+  runner:
+    type: Duration
+    duration: 60
+    interval: 1
+
+  sla:
+    max_rtt: 10
+    action: monitor
+
+context:
+  name: demo
+  image: cirros-0.3.3
+  flavor: m1.tiny
+  user: cirros
+
+  placement_groups:
+    pgrp1:
+      policy: "availability"
+
+  servers:
+    athena:
+      floating_ip: true
+      placement: "pgrp1"
+    ares:
+      placement: "pgrp1"
+
+  networks:
+    test:
+      cidr: '10.0.1.0/24'
+      external_network: "net04_ext"
+
diff --git a/setup.py b/setup.py
index f73094ac1..f171aafee 100644..100755
--- a/setup.py
+++ b/setup.py
@@ -19,6 +19,7 @@ setup(
     url="https://www.opnfv.org",
     install_requires=["backport_ipaddress",  # remove with python3
                       "flake8",
+                      "Jinja2>=2.6",
                       "PyYAML>=3.10",
                       "pbr<2.0,>=1.3",
                       "python-glanceclient>=0.12.0",
diff --git a/yardstick/cmd/commands/task.py b/yardstick/cmd/commands/task.py
index 8b9f269c5..f49a258a1 100644..100755
--- a/yardstick/cmd/commands/task.py
+++ b/yardstick/cmd/commands/task.py
@@ -18,7 +18,7 @@ import ipaddress
 
 from yardstick.benchmark.context.model import Context
 from yardstick.benchmark.runners import base as base_runner
-
+from yardstick.common.task_template import TaskTemplate
 from yardstick.common.utils import cliargs
 
 output_file_default = "/tmp/yardstick.out"
@@ -31,6 +31,13 @@ class TaskCommands(object):
     '''
 
     @cliargs("taskfile", type=str, help="path to taskfile", nargs=1)
+    @cliargs("--task-args", dest="task_args",
+             help="Input task args (dict in json). These args are used"
+             "to render input task that is jinja2 template.")
+    @cliargs("--task-args-file", dest="task_args_file",
+             help="Path to the file with input task args (dict in "
+             "json/yaml). These args are used to render input"
+             "task that is jinja2 template.")
     @cliargs("--keep-deploy", help="keep context deployed in cloud",
              action="store_true")
     @cliargs("--parse-only", help="parse the benchmark config file and exit",
@@ -43,7 +50,8 @@ class TaskCommands(object):
         atexit.register(atexit_handler)
 
         parser = TaskParser(args.taskfile[0])
-        scenarios, run_in_parallel = parser.parse()
+        scenarios, run_in_parallel = parser.parse(args.task_args,
+                                                  args.task_args_file)
 
         if args.parse_only:
             sys.exit(0)
@@ -80,20 +88,39 @@ class TaskCommands(object):
 
         print "Done, exiting"
 
-
 # TODO: Move stuff below into TaskCommands class !?
 
+
 class TaskParser(object):
     '''Parser for task config files in yaml format'''
     def __init__(self, path):
         self.path = path
 
-    def parse(self):
+    def parse(self, task_args=None, task_args_file=None):
         '''parses the task file and return an context and scenario instances'''
         print "Parsing task config:", self.path
+
+        try:
+            kw = {}
+            if task_args_file:
+                with open(task_args_file) as f:
+                    kw.update(parse_task_args("task_args_file", f.read()))
+            kw.update(parse_task_args("task_args", task_args))
+        except TypeError:
+            raise TypeError()
+
         try:
-            with open(self.path) as stream:
-                cfg = yaml.load(stream)
+            with open(self.path) as f:
+                try:
+                    input_task = f.read()
+                    rendered_task = TaskTemplate.render(input_task, **kw)
+                except Exception as e:
+                    print(("Failed to render template:\n%(task)s\n%(err)s\n")
+                          % {"task": input_task, "err": e})
+                    raise e
+                print(("Input task is:\n%s\n") % rendered_task)
+
+                cfg = yaml.load(rendered_task)
         except IOError as ioerror:
             sys.exit(ioerror)
 
@@ -181,3 +208,26 @@ def runner_join(runner):
     base_runner.Runner.release(runner)
     if status != 0:
         sys.exit("Runner failed")
+
+
+def print_invalid_header(source_name, args):
+    print(("Invalid %(source)s passed:\n\n %(args)s\n")
+          % {"source": source_name, "args": args})
+
+
+def parse_task_args(src_name, args):
+    try:
+        kw = args and yaml.safe_load(args)
+        kw = {} if kw is None else kw
+    except yaml.parser.ParserError as e:
+        print_invalid_header(src_name, args)
+        print(("%(source)s has to be YAML. Details:\n\n%(err)s\n")
+              % {"source": src_name, "err": e})
+        raise TypeError()
+
+    if not isinstance(kw, dict):
+        print_invalid_header(src_name, args)
+        print(("%(src)s had to be dict, actually %(src_type)s\n")
+              % {"src": src_name, "src_type": type(kw)})
+        raise TypeError()
+    return kw
diff --git a/yardstick/common/task_template.py b/yardstick/common/task_template.py
new file mode 100755
index 000000000..2739323bd
--- /dev/null
+++ b/yardstick/common/task_template.py
@@ -0,0 +1,53 @@
+##############################################################################
+# Copyright (c) 2015 Huawei Technologies Co.,Ltd and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+# yardstick: this file is copied from rally and slightly modified
+##############################################################################
+import re
+import jinja2
+import jinja2.meta
+
+
+class TaskTemplate(object):
+    @classmethod
+    def render(cls, task_template, **kwargs):
+        """Render jinja2 task template to Yardstick input task.
+
+        :param task_template: string that contains template
+        :param kwargs: Dict with template arguments
+        :returns:rendered template str
+        """
+
+        from six.moves import builtins
+
+        ast = jinja2.Environment().parse(task_template)
+        required_kwargs = jinja2.meta.find_undeclared_variables(ast)
+
+        missing = set(required_kwargs) - set(kwargs) - set(dir(builtins))
+        real_missing = [mis for mis in missing
+                        if is_really_missing(mis, task_template)]
+
+        if real_missing:
+            multi_msg = ("Please specify next template task arguments:%s")
+            single_msg = ("Please specify template task argument:%s")
+            raise TypeError((len(real_missing) > 1 and multi_msg or single_msg)
+                            % ", ".join(real_missing))
+        return jinja2.Template(task_template).render(**kwargs)
+
+
+def is_really_missing(mis, task_template):
+    # Removing variables that have default values from
+    # missing. Construction that won't be properly
+    # check is {% set x = x or 1}
+    if re.search(mis.join(["{%\s*set\s+", "\s*=\s*", "[^\w]+"]),
+                 task_template):
+        return False
+    # Also check for a default filter which can show up as
+    # a missing variable
+    if re.search(mis + "\s*\|\s*default\(", task_template):
+        return False
+    return True