diff options
author | Fatih Degirmenci <fatih.degirmenci@ericsson.com> | 2015-12-03 14:13:16 +0100 |
---|---|---|
committer | Fatih Degirmenci <fatih.degirmenci@ericsson.com> | 2015-12-03 14:18:00 +0100 |
commit | bd3d1c0ebdc2ed8e9918b7e462a6994468a46d0a (patch) | |
tree | e56edd70b285383bf107316663f669cc2a570a12 /docs/templates | |
parent | 2f7f80ef210977f4ca30056fc9852b8ea4242cca (diff) |
Remove etc and source folder from yardstick documentation
conf.py and logo will come from releng which is common for all projects.
The directory structure in docs folder has not been decided yet so this
change does not require you to change your current way of working.
Once this directory structure is decided by opnfvdocs and releng, the changes
need to be reflected back to all projects, including yardstick.
Change-Id: If0558474a819f4a44e34f790d26c5486751b8266
Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
Diffstat (limited to 'docs/templates')
-rwxr-xr-x | docs/templates/Yardstick_task_templates.rst | 141 | ||||
-rw-r--r-- | docs/templates/testcase_description_template.rst | 94 | ||||
-rw-r--r-- | docs/templates/testcase_description_v2_template.rst | 51 |
3 files changed, 286 insertions, 0 deletions
diff --git a/docs/templates/Yardstick_task_templates.rst b/docs/templates/Yardstick_task_templates.rst new file mode 100755 index 000000000..538937fd7 --- /dev/null +++ b/docs/templates/Yardstick_task_templates.rst @@ -0,0 +1,141 @@ +Task Template Syntax +==================== + +Basic template syntax +--------------------- +A nice feature of the input task format used in Yardstick is that it supports the template syntax based on Jinja2. +This turns out to be extremely useful when, say, you have a fixed structure of your task but you want to +parameterize this task in some way. +For example, imagine your input task file (task.yaml) runs a set of Ping scenarios: + +:: + + # Sample benchmark task config file + # measure network latency using ping + schema: "yardstick:task:0.1" + + scenarios: + - + type: Ping + options: + packetsize: 200 + host: athena.demo + target: ares.demo + + runner: + type: Duration + duration: 60 + interval: 1 + + sla: + max_rtt: 10 + action: monitor + + context: + ... + +Let's say you want to run the same set of scenarios with the same runner/context/sla, +but you want to try another packetsize to compare the performance. +The most elegant solution is then to turn the packetsize name into a template variable: + +:: + + # Sample benchmark task config file + # measure network latency using ping + + schema: "yardstick:task:0.1" + scenarios: + - + type: Ping + options: + packetsize: {{packetsize}} + host: athena.demo + target: ares.demo + + runner: + type: Duration + duration: 60 + interval: 1 + + sla: + max_rtt: 10 + action: monitor + + context: + ... + +and then pass the argument value for {{packetsize}} when starting a task with this configuration file. +Yardstick provides you with different ways to do that: + +1.Pass the argument values directly in the command-line interface (with either a JSON or YAML dictionary): + +:: + + yardstick task start samples/ping-template.yaml --task-args '{"packetsize": "200"}' + +2.Refer to a file that specifies the argument values (JSON/YAML): + +:: + + yardstick task start samples/ping-template.yaml --task-args-file args.yaml + +Using the default values +------------------------ +Note that the Jinja2 template syntax allows you to set the default values for your parameters. +With default values set, your task file will work even if you don't parameterize it explicitly while starting a task. +The default values should be set using the {% set ... %} clause (task.yaml).For example: + +:: + + # Sample benchmark task config file + # measure network latency using ping + schema: "yardstick:task:0.1" + {% set packetsize = packetsize or "100" %} + scenarios: + - + type: Ping + options: + packetsize: {{packetsize}} + host: athena.demo + target: ares.demo + + runner: + type: Duration + duration: 60 + interval: 1 + ... + +If you don't pass the value for {{packetsize}} while starting a task, the default one will be used. + +Advanced templates +------------------ +Yardstick makes it possible to use all the power of Jinja2 template syntax, including the mechanism of built-in functions. +As an example, let us make up a task file that will do a block storage performance test. +The input task file (fio-template.yaml) below uses the Jinja2 for-endfor construct to accomplish that: + +:: + + #Test block sizes of 4KB, 8KB, 64KB, 1MB + #Test 5 workloads: read, write, randwrite, randread, rw + schema: "yardstick:task:0.1" + + scenarios: + {% for bs in ['4k', '8k', '64k', '1024k' ] %} + {% for rw in ['read', 'write', 'randwrite', 'randread', 'rw' ] %} + - + type: Fio + options: + filename: /home/ec2-user/data.raw + bs: {{bs}} + rw: {{rw}} + ramp_time: 10 + host: fio.demo + runner: + type: Duration + duration: 60 + interval: 60 + + {% endfor %} + {% endfor %} + context + ... diff --git a/docs/templates/testcase_description_template.rst b/docs/templates/testcase_description_template.rst new file mode 100644 index 000000000..1651d360c --- /dev/null +++ b/docs/templates/testcase_description_template.rst @@ -0,0 +1,94 @@ +.. Template to be used for test case descriptions in Yardstick Project. + Write one .rst per test case. + Upload the .rst for the test case in /docs/source/yardstick directory. + Review in Gerrit. + +.. image:: ../etc/opnfv-logo.png + :height: 40 + :width: 200 + :alt: OPNFV + :align: left + +****************** +Test Case <slogan> +****************** + +.. contents:: Table of Contents + :depth: 3 + +--------------------- +Test Case Description +--------------------- + +Yardstick Test Case ID +---------------------- + +OPNFV_YARDSTICK_TC<abc>_<slogan> + +where: + - <abc>: check Jira issue for the test case + - <slogan>: check Jira issue for the test case + + +Purpose +------- + +Describe what is the purpose of the test case + +Area +---- + +State the area and sub-area covered by the test case. + +Areas: Compute, Networking, Storage + +Sub-areas: Performance, System limit, QoS + +Metrics +------- + +What will be measured, attribute name or collection of attributes, behavior + +References +---------- + +Reference documentation + +-------------- +Pre-requisites +-------------- + +Tools +----- + +What tools are used to perform the measurements (e.g. fio, pktgen) + + +Configuration +------------- + +State the .yaml file to use. + +State default configuration in the tool(s) used to perform the measurements +(e.g. fio, pktgen). + +State what POD-specific configuration is required to enable running the test +case in different PODs. + + +State SLA, if applicable. + +State test duration. + +------- +Results +------- + +Expected outcome +---------------- + +State applicable graphical presentation + +State applicable output details + +State expected Value, behavior, pass/fail criteria diff --git a/docs/templates/testcase_description_v2_template.rst b/docs/templates/testcase_description_v2_template.rst new file mode 100644 index 000000000..0fa2359e9 --- /dev/null +++ b/docs/templates/testcase_description_v2_template.rst @@ -0,0 +1,51 @@ +.. Template to be used for test case descriptions in Yardstick Project. + Write one .rst per test case. + Upload the .rst for the test case in /docs/source/yardstick directory. + Review in Gerrit. + +.. image:: ../etc/opnfv-logo.png + :height: 40 + :width: 200 + :alt: OPNFV + :align: left + +************************************* +Yardstick Test Case Description TCXXX +************************************* + ++-----------------------------------------------------------------------------+ +|test case slogan e.g. Network Latency | ++==============+==============================================================+ +|test case id | e.g. OPNFV_YARDSTICK_TC001_NW Latency | ++--------------+--------------------------------------------------------------+ +|metric | what will be measured, e.g. latency | ++--------------+--------------------------------------------------------------+ +|test purpose | describe what is the purpose of the test case | ++--------------+--------------------------------------------------------------+ +|configuration | what .yaml file to use, state SLA if applicable, state | +| | test duration, list and describe the scenario options used in| +| | this TC and also list the options using default values. | ++--------------+--------------------------------------------------------------+ +|test tool | e.g. ping | ++--------------+--------------------------------------------------------------+ +|references | e.g. RFCxxx, ETSI-NFVyyy | ++--------------+--------------------------------------------------------------+ +|applicability | describe variations of the test case which can be | +| | performend, e.g. run the test for different packet sizes | ++--------------+--------------------------------------------------------------+ +|pre-test | describe configuration in the tool(s) used to perform | +|conditions | the measurements (e.g. fio, pktgen), POD-specific | +| | configuration required to enable running the test | ++--------------+------+----------------------------------+--------------------+ +|test sequence | step | description | result | +| +------+----------------------------------+--------------------+ +| | 1 | use this to describe tests that | what happens in | +| | | require several steps e.g. | this step | +| | | step 1 collect logs | e.g. logs collected| +| +------+----------------------------------+--------------------+ +| | 2 | remove interface | interface down | +| +------+----------------------------------+--------------------+ +| | N | what is done in step N | what happens | ++--------------+------+----------------------------------+--------------------+ +|test verdict | expected behavior, or SLA, pass/fail criteria | ++--------------+--------------------------------------------------------------+ |