summaryrefslogtreecommitdiffstats
path: root/docs/templates
diff options
context:
space:
mode:
Diffstat (limited to 'docs/templates')
-rwxr-xr-xdocs/templates/Yardstick_task_templates.rst141
-rw-r--r--docs/templates/testcase_description_template.rst94
-rw-r--r--docs/templates/testcase_description_v2_template.rst51
3 files changed, 286 insertions, 0 deletions
diff --git a/docs/templates/Yardstick_task_templates.rst b/docs/templates/Yardstick_task_templates.rst
new file mode 100755
index 000000000..538937fd7
--- /dev/null
+++ b/docs/templates/Yardstick_task_templates.rst
@@ -0,0 +1,141 @@
+Task Template Syntax
+====================
+
+Basic template syntax
+---------------------
+A nice feature of the input task format used in Yardstick is that it supports the template syntax based on Jinja2.
+This turns out to be extremely useful when, say, you have a fixed structure of your task but you want to
+parameterize this task in some way.
+For example, imagine your input task file (task.yaml) runs a set of Ping scenarios:
+
+::
+
+ # Sample benchmark task config file
+ # measure network latency using ping
+ schema: "yardstick:task:0.1"
+
+ scenarios:
+ -
+ type: Ping
+ options:
+ packetsize: 200
+ host: athena.demo
+ target: ares.demo
+
+ runner:
+ type: Duration
+ duration: 60
+ interval: 1
+
+ sla:
+ max_rtt: 10
+ action: monitor
+
+ context:
+ ...
+
+Let's say you want to run the same set of scenarios with the same runner/context/sla,
+but you want to try another packetsize to compare the performance.
+The most elegant solution is then to turn the packetsize name into a template variable:
+
+::
+
+ # Sample benchmark task config file
+ # measure network latency using ping
+
+ schema: "yardstick:task:0.1"
+ scenarios:
+ -
+ type: Ping
+ options:
+ packetsize: {{packetsize}}
+ host: athena.demo
+ target: ares.demo
+
+ runner:
+ type: Duration
+ duration: 60
+ interval: 1
+
+ sla:
+ max_rtt: 10
+ action: monitor
+
+ context:
+ ...
+
+and then pass the argument value for {{packetsize}} when starting a task with this configuration file.
+Yardstick provides you with different ways to do that:
+
+1.Pass the argument values directly in the command-line interface (with either a JSON or YAML dictionary):
+
+::
+
+ yardstick task start samples/ping-template.yaml --task-args '{"packetsize": "200"}'
+
+2.Refer to a file that specifies the argument values (JSON/YAML):
+
+::
+
+ yardstick task start samples/ping-template.yaml --task-args-file args.yaml
+
+Using the default values
+------------------------
+Note that the Jinja2 template syntax allows you to set the default values for your parameters.
+With default values set, your task file will work even if you don't parameterize it explicitly while starting a task.
+The default values should be set using the {% set ... %} clause (task.yaml).For example:
+
+::
+
+ # Sample benchmark task config file
+ # measure network latency using ping
+ schema: "yardstick:task:0.1"
+ {% set packetsize = packetsize or "100" %}
+ scenarios:
+ -
+ type: Ping
+ options:
+ packetsize: {{packetsize}}
+ host: athena.demo
+ target: ares.demo
+
+ runner:
+ type: Duration
+ duration: 60
+ interval: 1
+ ...
+
+If you don't pass the value for {{packetsize}} while starting a task, the default one will be used.
+
+Advanced templates
+------------------
+Yardstick makes it possible to use all the power of Jinja2 template syntax, including the mechanism of built-in functions.
+As an example, let us make up a task file that will do a block storage performance test.
+The input task file (fio-template.yaml) below uses the Jinja2 for-endfor construct to accomplish that:
+
+::
+
+ #Test block sizes of 4KB, 8KB, 64KB, 1MB
+ #Test 5 workloads: read, write, randwrite, randread, rw
+ schema: "yardstick:task:0.1"
+
+ scenarios:
+ {% for bs in ['4k', '8k', '64k', '1024k' ] %}
+ {% for rw in ['read', 'write', 'randwrite', 'randread', 'rw' ] %}
+ -
+ type: Fio
+ options:
+ filename: /home/ec2-user/data.raw
+ bs: {{bs}}
+ rw: {{rw}}
+ ramp_time: 10
+ host: fio.demo
+ runner:
+ type: Duration
+ duration: 60
+ interval: 60
+
+ {% endfor %}
+ {% endfor %}
+ context
+ ...
diff --git a/docs/templates/testcase_description_template.rst b/docs/templates/testcase_description_template.rst
new file mode 100644
index 000000000..1651d360c
--- /dev/null
+++ b/docs/templates/testcase_description_template.rst
@@ -0,0 +1,94 @@
+.. Template to be used for test case descriptions in Yardstick Project.
+ Write one .rst per test case.
+ Upload the .rst for the test case in /docs/source/yardstick directory.
+ Review in Gerrit.
+
+.. image:: ../etc/opnfv-logo.png
+ :height: 40
+ :width: 200
+ :alt: OPNFV
+ :align: left
+
+******************
+Test Case <slogan>
+******************
+
+.. contents:: Table of Contents
+ :depth: 3
+
+---------------------
+Test Case Description
+---------------------
+
+Yardstick Test Case ID
+----------------------
+
+OPNFV_YARDSTICK_TC<abc>_<slogan>
+
+where:
+ - <abc>: check Jira issue for the test case
+ - <slogan>: check Jira issue for the test case
+
+
+Purpose
+-------
+
+Describe what is the purpose of the test case
+
+Area
+----
+
+State the area and sub-area covered by the test case.
+
+Areas: Compute, Networking, Storage
+
+Sub-areas: Performance, System limit, QoS
+
+Metrics
+-------
+
+What will be measured, attribute name or collection of attributes, behavior
+
+References
+----------
+
+Reference documentation
+
+--------------
+Pre-requisites
+--------------
+
+Tools
+-----
+
+What tools are used to perform the measurements (e.g. fio, pktgen)
+
+
+Configuration
+-------------
+
+State the .yaml file to use.
+
+State default configuration in the tool(s) used to perform the measurements
+(e.g. fio, pktgen).
+
+State what POD-specific configuration is required to enable running the test
+case in different PODs.
+
+
+State SLA, if applicable.
+
+State test duration.
+
+-------
+Results
+-------
+
+Expected outcome
+----------------
+
+State applicable graphical presentation
+
+State applicable output details
+
+State expected Value, behavior, pass/fail criteria
diff --git a/docs/templates/testcase_description_v2_template.rst b/docs/templates/testcase_description_v2_template.rst
new file mode 100644
index 000000000..0fa2359e9
--- /dev/null
+++ b/docs/templates/testcase_description_v2_template.rst
@@ -0,0 +1,51 @@
+.. Template to be used for test case descriptions in Yardstick Project.
+ Write one .rst per test case.
+ Upload the .rst for the test case in /docs/source/yardstick directory.
+ Review in Gerrit.
+
+.. image:: ../etc/opnfv-logo.png
+ :height: 40
+ :width: 200
+ :alt: OPNFV
+ :align: left
+
+*************************************
+Yardstick Test Case Description TCXXX
+*************************************
+
++-----------------------------------------------------------------------------+
+|test case slogan e.g. Network Latency |
++==============+==============================================================+
+|test case id | e.g. OPNFV_YARDSTICK_TC001_NW Latency |
++--------------+--------------------------------------------------------------+
+|metric | what will be measured, e.g. latency |
++--------------+--------------------------------------------------------------+
+|test purpose | describe what is the purpose of the test case |
++--------------+--------------------------------------------------------------+
+|configuration | what .yaml file to use, state SLA if applicable, state |
+| | test duration, list and describe the scenario options used in|
+| | this TC and also list the options using default values. |
++--------------+--------------------------------------------------------------+
+|test tool | e.g. ping |
++--------------+--------------------------------------------------------------+
+|references | e.g. RFCxxx, ETSI-NFVyyy |
++--------------+--------------------------------------------------------------+
+|applicability | describe variations of the test case which can be |
+| | performend, e.g. run the test for different packet sizes |
++--------------+--------------------------------------------------------------+
+|pre-test | describe configuration in the tool(s) used to perform |
+|conditions | the measurements (e.g. fio, pktgen), POD-specific |
+| | configuration required to enable running the test |
++--------------+------+----------------------------------+--------------------+
+|test sequence | step | description | result |
+| +------+----------------------------------+--------------------+
+| | 1 | use this to describe tests that | what happens in |
+| | | require several steps e.g. | this step |
+| | | step 1 collect logs | e.g. logs collected|
+| +------+----------------------------------+--------------------+
+| | 2 | remove interface | interface down |
+| +------+----------------------------------+--------------------+
+| | N | what is done in step N | what happens |
++--------------+------+----------------------------------+--------------------+
+|test verdict | expected behavior, or SLA, pass/fail criteria |
++--------------+--------------------------------------------------------------+