Merge "Refine docs" into stable/colorado

10 files changed, 115 insertions, 88 deletions

diff --git a/docs/userguide/01-introduction.rst b/docs/userguide/01-introduction.rst
index 3db6ce001..9d9cf0fb5 100755
--- a/docs/userguide/01-introduction.rst
+++ b/docs/userguide/01-introduction.rst
@@ -40,18 +40,25 @@ This document consists of the following chapters:
 * Chapter :doc:`02-methodology` describes the methodology implemented by the
   Yardstick Project for :term:`NFVI` verification.
 
-* Chapter :doc:`architecture` provides information on the software architecture
+* Chapter :doc:`03-architecture` provides information on the software architecture
   of yardstick.
+
 * Chapter :doc:`04-vtc-overview` provides information on the :term:`VTC`.
 
-* Chapter :doc:`apexlake_installation` provides instructions to install the
-  experimental framework *ApexLake* and chapter :doc:`apexlake_api` explains
+* Chapter :doc:`05-apexlake_installation` provides instructions to install the
+  experimental framework *ApexLake* and chapter :doc:`06-apexlake_api` explains
   how this framework is integrated in *Yardstick*.
 
-* Chapter :doc:`03-installation` provides instructions to install *Yardstick*.
+* Chapter :doc:`07-installation` provides instructions to install *Yardstick*.
+
+* Chapter :doc:`08-yardstick_plugin` provides information on how to integrate
+  other OPNFV testing projects into *Yardstick*.
 
-* Chapter :doc:`03-list-of-tcs` includes a list of available Yardstick
-  test cases.
+* Chapter :doc:`09-result-store-InfluxDB` provides inforamtion on how to run
+  plug-in test cases and store test results into community's InfluxDB.
+
+* Chapter :doc:`10-list-of-tcs` includes a list of available Yardstick test
+  cases.
 
 
 Contact Yardstick
@@ -60,3 +67,4 @@ Contact Yardstick
 Feedback? `Contact us`_
 
 .. _Contact us: opnfv-users@lists.opnfv.org
+
diff --git a/docs/userguide/02-methodology.rst b/docs/userguide/02-methodology.rst
index 3fa432a98..1849ffd0c 100644
--- a/docs/userguide/02-methodology.rst
+++ b/docs/userguide/02-methodology.rst
@@ -59,7 +59,7 @@ The metrics, as defined by ETSI GS NFV-TST001, are shown in
 :ref:`Table1 <table2_1>`, :ref:`Table2 <table2_2>` and
 :ref:`Table3 <table2_3>`.
 
-In OPNFV Brahmaputra release, generic test cases covering aspects of the listed
+In OPNFV Colorado release, generic test cases covering aspects of the listed
 metrics are available; further OPNFV releases will provide extended testing of
 these metrics.
 The view of available Yardstick test cases cross ETSI definitions in
@@ -169,24 +169,25 @@ options).
 |         |                   |                |                              |
 +---------+-------------------+----------------+------------------------------+
 | Compute | TC003 [1]_        | TC003 [1]_     |  TC013 [1]_                  |
-|         | TC004 [1]_        | TC004 [1]_     |  TC015 [1]_                  |
+|         | TC004             | TC004          |  TC015 [1]_                  |
 |         | TC014             | TC010          |                              |
 |         | TC024             | TC012          |                              |
-|         |                   |                |                              |
+|         | TC069             | TC055          |                              |
 +---------+-------------------+----------------+------------------------------+
-| Network | TC002             | TC001          |  TC016 [1]_                  |
-|         | TC011             | TC008          |  TC018 [1]_                  |
-|         |                   | TC009          |                              |
-|         |                   |                |                              |
+| Network | TC001             | TC001          |  TC016 [1]_                  |
+|         | TC002             | TC008          |  TC018 [1]_                  |
+|         | TC011             | TC009          |                              |
+|         | TC073             | TC075          |                              |
 +---------+-------------------+----------------+------------------------------+
 | Storage | TC005             | TC005          |  TC017 [1]_                  |
-|         |                   |                |                              |
+|         |                   | TC063          |                              |
 +---------+-------------------+----------------+------------------------------+
 
 .. note:: The description in this OPNFV document is intended as a reference for
   users to understand the scope of the Yardstick Project and the
   deliverables of the Yardstick framework. For complete description of
-  the methodology, refer to the ETSI document.
+  the methodology, please refer to the ETSI document.
 
 .. rubric:: Footnotes
 .. [1] To be included in future deliveries.
+
diff --git a/docs/userguide/03-architecture.rst b/docs/userguide/03-architecture.rst
index 3abb67b7d..ace3117c2 100755
--- a/docs/userguide/03-architecture.rst
+++ b/docs/userguide/03-architecture.rst
@@ -222,7 +222,7 @@ Deployment View
 ===============
 Yardstick deployment view shows how the yardstick tool can be deployed into the
 underlying platform. Generally, yardstick tool is installed on JumpServer(see
-`03-installation` for detail installation steps), and JumpServer is
+`07-installation` for detail installation steps), and JumpServer is
 connected with other control/compute servers by networking. Based on this
 deployment, yardstick can run the test cases on these hosts, and get the test
 result for better showing.
@@ -256,8 +256,11 @@ Yardstick Directory structure
            by Heat. Currently contains how to build the yardstick-trusty-server
            image with the different tools that are needed from within the image.
 
+*plugin/* - Plug-in configuration files are stored here.
+
 *vTC/* - Contains the files for running the virtual Traffic Classifier tests.
 
 *yardstick/* - Contains the internals of Yardstick: Runners, Scenario, Contexts,
-               CLI parsing, keys, plotting tools, dispatcher and so on.
+               CLI parsing, keys, plotting tools, dispatcher, plugin
+               install/remove scripts and so on.
 
diff --git a/docs/userguide/07-installation.rst b/docs/userguide/07-installation.rst
index 475719c72..3a0a7975f 100644
--- a/docs/userguide/07-installation.rst
+++ b/docs/userguide/07-installation.rst
@@ -29,9 +29,21 @@ Installing Yardstick on Ubuntu 14.04
 
 .. _install-framework:
 
+You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu
+14.04 Docker image.
+No matter which way you choose to install Yardstick framework, the following
+installation steps are identical.
+If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu
+14.04 Docker image from Docker hub:
+
+::
+
+  docker pull ubuntu:14.04
+
 Installing Yardstick framework
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Install dependencies:
+
 ::
 
   sudo apt-get update && sudo apt-get install -y \
@@ -50,6 +62,7 @@ Install dependencies:
       python-setuptools
 
 Create a python virtual environment, source it and update setuptools:
+
 ::
 
   virtualenv ~/yardstick_venv
@@ -57,6 +70,7 @@ Create a python virtual environment, source it and update setuptools:
   easy_install -U setuptools
 
 Download source code and install python dependencies:
+
 ::
 
   git clone https://gerrit.opnfv.org/gerrit/yardstick
@@ -69,17 +83,6 @@ There is also a YouTube video, showing the above steps:
    :alt: http://www.youtube.com/watch?v=4S4izNolmR0
    :target: http://www.youtube.com/watch?v=4S4izNolmR0
 
-Installing extra tools
-^^^^^^^^^^^^^^^^^^^^^^
-yardstick-plot
-""""""""""""""
-Yardstick has an internal plotting tool ``yardstick-plot``, which can be installed
-using the following command:
-::
-
-  sudo apt-get install -y g++ libfreetype6-dev libpng-dev pkg-config
-  python setup.py develop easy_install yardstick[plot]
-
 .. _guest-image:
 
 Building a guest image
@@ -90,6 +93,7 @@ have sudo rights to use this tool.
 
 Also you may need install several additional packages to use this tool, by
 follwing the commands below:
+
 ::
 
   apt-get update && apt-get install -y \
@@ -99,6 +103,7 @@ follwing the commands below:
 This image can be built using the following command while in the directory where
 Yardstick is installed (``~/yardstick`` if the framework is installed
 by following the commands above):
+
 ::
 
   sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh
@@ -110,6 +115,7 @@ The created image can be added to OpenStack using the ``glance image-create`` or
 via the OpenStack Dashboard.
 
 Example command:
+
 ::
 
   glance --os-image-api-version 1 image-create \
@@ -121,44 +127,18 @@ Example command:
 Installing Yardstick using Docker
 ---------------------------------
 
-Yardstick has two Docker images, first one (**Yardstick-framework**) serves as a
-replacement for installing the Yardstick framework in a virtual environment (for
-example as done in :ref:`install-framework`), while the other image is mostly for
-CI purposes (**Yardstick-CI**).
-
-Yardstick-framework image
-^^^^^^^^^^^^^^^^^^^^^^^^^
-Download the source code:
-
-::
-
-  git clone https://gerrit.opnfv.org/gerrit/yardstick
-
-Build the Docker image and tag it as *yardstick-framework*:
-
-::
-
-  cd yardstick
-  docker build -t yardstick-framework .
-
-Run the Docker instance:
-
-::
-
-  docker run --name yardstick_instance -i -t yardstick-framework
+Yardstick iteself has a Docker image, this Docker image (**Yardstick-stable**)
+serves as a replacement for installing the Yardstick framework in a virtual
+environment (for example as done in :ref:`install-framework`).
+It is recommended to use this Docker image to run Yardstick test.
 
-To build a guest image for Yardstick, see :ref:`guest-image`.
-
-Yardstick-CI image
-^^^^^^^^^^^^^^^^^^
-Pull the Yardstick-CI Docker image from Docker hub:
+Yardstick-stable image
+^^^^^^^^^^^^^^^^^^^^^^
+Pull the Yardstick-stable Docker image from Docker hub:
 
 ::
 
-  docker pull opnfv/yardstick:$DOCKER_TAG
-
-Where ``$DOCKER_TAG`` is latest for master branch, as for the release branches,
-this coincides with its release name, such as brahmaputra.1.0.
+  docker pull opnfv/yardstick:stable
 
 Run the Docker image:
 
@@ -180,21 +160,32 @@ For more details, please refer to the Jenkins job defined in Releng project, lab
 and sshkey are required. See the link
 https://git.opnfv.org/cgit/releng/tree/jjb/yardstick/yardstick-ci-jobs.yml.
 
-Note: exec_tests.sh is used for executing test suite here, furthermore, if someone wants to execute the
-test suite manually, it can be used as long as the parameters are configured correct. Another script
-called run_tests.sh is used for unittest in Jenkins verify job, in local manaul environment,
-it is recommended to run before test suite execuation.
+Note: exec_tests.sh is used for executing test suite here, furthermore, if someone
+wants to execute the test suite manually, it can be used as long as the parameters
+are configured correct. Another script called run_tests.sh is used for unittest in
+Jenkins verify job, in local manaul environment, it is recommended to run before
+test suite execuation.
 
-Basic steps performed by the **Yardstick-CI** container:
+Basic steps performed by the **Yardstick-stable** container:
 
 1. clone yardstick and releng repos
 2. setup OS credentials (releng scripts)
 3. install yardstick and dependencies
 4. build yardstick cloud image and upload it to glance
-5. upload cirros-0.3.3 cloud image to glance
+5. upload cirros-0.3.3 cloud image and ubuntu-14.04 cloud image to glance
 6. run yardstick test scenarios
 7. cleanup
 
+If someone only wants to execute a single test case, one can log into the yardstick-stable
+container first using command:
+
+::
+
+  docker run -it openfv/yardstick /bin/bash
+
+Then in the container run yardstick task command to execute single test case.
+Detailed steps about executing Yardstick test case can be found below.
+
 
 OpenStack parameters and credentials
 ------------------------------------
@@ -240,18 +231,8 @@ of yardstick help command and ping.py test sample:
 Each testing tool supported by Yardstick has a sample configuration file.
 These configuration files can be found in the **samples** directory.
 
-Example invocation of ``yardstick-plot`` tool:
-::
-
-  yardstick-plot -i /tmp/yardstick.out -o /tmp/plots/
-
 Default location for the output is ``/tmp/yardstick.out``.
 
-More info about the tool can be found by executing:
-::
-
-  yardstick-plot -h
-
 
 Deploy InfluxDB and Grafana locally
 ------------------------------------
@@ -259,6 +240,7 @@ Deploy InfluxDB and Grafana locally
 .. pull docker images
 
 Pull docker images
+
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 ::
@@ -299,6 +281,8 @@ Config grafana
   log on using admin/admin and config database resource to be {YOUR_IP_HERE}:8086
 
 .. image:: images/Grafana_config.png
+   :width: 800px
+   :alt: Grafana data source configration
 
 Config yardstick conf
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -388,3 +372,4 @@ the ci environment. "task_args" is where you can specify the task arguments for
 
 All in all, to create a test suite in yardstick, you just need to create a suite yaml file
 and add test cases and constraint or task arguments if necessary.
+
diff --git a/docs/userguide/08-yardstick_plugin.rst b/docs/userguide/08-yardstick_plugin.rst
index e68db650d..f16dedd02 100644
--- a/docs/userguide/08-yardstick_plugin.rst
+++ b/docs/userguide/08-yardstick_plugin.rst
@@ -48,11 +48,11 @@ environment and other dependencies:
 3. Make sure Jump Host have access to the OpenStack Controller API.
 4. Make sure Jump Host must have internet connectivity for downloading docker image.
 5. You need to know where to get basic openstack Keystone authorization info, such as
-OS_PASSWORD, OS_TENANT_NAME, OS_AUTH_URL, OS_USERNAME.
+   OS_PASSWORD, OS_TENANT_NAME, OS_AUTH_URL, OS_USERNAME.
 6. To run a Storperf container, you need to have OpenStack Controller environment
-variables defined and passed to Storperf container. The best way to do this is to
-put environment variables in a "storperf_admin-rc" file. The storperf_admin-rc
-should include credential environment variables at least:
+   variables defined and passed to Storperf container. The best way to do this is to
+   put environment variables in a "storperf_admin-rc" file. The storperf_admin-rc
+   should include credential environment variables at least:
 
 * OS_AUTH_URL
 * OS_TENANT_ID
diff --git a/docs/userguide/09-result-store-InfluxDB.rst b/docs/userguide/09-result-store-InfluxDB.rst
index 5c49e9f7c..a0bb48a80 100644
--- a/docs/userguide/09-result-store-InfluxDB.rst
+++ b/docs/userguide/09-result-store-InfluxDB.rst
@@ -17,7 +17,7 @@ into community's InfluxDB. The framework is shown in Framework_.
 
 
 .. image:: images/InfluxDB_store.png
-   :width: 1200px
+   :width: 800px
    :alt: Store Other Project's Test Results in InfluxDB
 
 Store Storperf Test Results into Community's InfluxDB
@@ -81,6 +81,6 @@ can be accessed by Login_.
 
 
 .. image:: images/results_visualization.png
-   :width: 1200px
+   :width: 800px
    :alt: results visualization
 
diff --git a/docs/userguide/10-list-of-tcs.rst b/docs/userguide/10-list-of-tcs.rst
index 7e8c85433..8798a8f51 100644
--- a/docs/userguide/10-list-of-tcs.rst
+++ b/docs/userguide/10-list-of-tcs.rst
@@ -49,6 +49,7 @@ Generic NFVI Test Case Descriptions
    opnfv_yardstick_tc070.rst
    opnfv_yardstick_tc071.rst
    opnfv_yardstick_tc072.rst
+   opnfv_yardstick_tc073.rst
    opnfv_yardstick_tc075.rst
 
 OPNFV Feature Test Cases
@@ -62,6 +63,16 @@ H A
 
    opnfv_yardstick_tc019.rst
    opnfv_yardstick_tc025.rst
+   opnfv_yardstick_tc045.rst
+   opnfv_yardstick_tc046.rst
+   opnfv_yardstick_tc047.rst
+   opnfv_yardstick_tc048.rst
+   opnfv_yardstick_tc049.rst
+   opnfv_yardstick_tc050.rst
+   opnfv_yardstick_tc051.rst
+   opnfv_yardstick_tc052.rst
+   opnfv_yardstick_tc053.rst
+   opnfv_yardstick_tc054.rst
 
 IPv6
 ----
@@ -87,6 +98,14 @@ Parser
 
    opnfv_yardstick_tc040.rst
 
+   StorPerf
+-----------
+
+.. toctree::
+   :maxdepth: 1
+
+   opnfv_yardstick_tc074.rst
+
 virtual Traffic Classifier
 --------------------------
 
@@ -106,3 +125,4 @@ Templates
 
    testcase_description_v2_template
    Yardstick_task_templates
+
diff --git a/docs/userguide/opnfv_yardstick_tc053.rst b/docs/userguide/opnfv_yardstick_tc053.rst
index 8808d12d9..3c6bbc628 100644
--- a/docs/userguide/opnfv_yardstick_tc053.rst
+++ b/docs/userguide/opnfv_yardstick_tc053.rst
@@ -13,7 +13,7 @@ Yardstick Test Case Description TC053
 |                                                                             |
 +--------------+--------------------------------------------------------------+
 |test case id  | OPNFV_YARDSTICK_TC053: OpenStack Controller Load Balance     |
-|              | Service High Availability-                                   |
+|              | Service High Availability                                    |
 +--------------+--------------------------------------------------------------+
 |test purpose  | This test case will verify the high availability of the      |
 |              | load balance service(current is HAProxy) that supports       |
diff --git a/docs/userguide/opnfv_yardstick_tc074.rst b/docs/userguide/opnfv_yardstick_tc074.rst
index c938f5dfd..92cd51439 100644
--- a/docs/userguide/opnfv_yardstick_tc074.rst
+++ b/docs/userguide/opnfv_yardstick_tc074.rst
@@ -7,7 +7,7 @@
 Yardstick Test Case Description TC074
 *************************************
 
-.. Storperf: https://wiki.opnfv.org/display/storperf/Storperf
+.. _Storperf: https://wiki.opnfv.org/display/storperf/Storperf
 
 +-----------------------------------------------------------------------------+
 |Storperf                                                                     |
@@ -44,7 +44,7 @@ Yardstick Test Case Description TC074
 |              | * timeout: 600 - maximum allowed job time                    |
 |              |                                                              |
 +--------------+--------------------------------------------------------------+
-|test tool     | Storperf                                                     |
+|test tool     | Storperf_                                                    |
 |              |                                                              |
 |              | StorPerf is a tool to measure block and object storage       |
 |              | performance in an NFVI.                                      |
diff --git a/docs/userguide/references.rst b/docs/userguide/references.rst
index 551926135..7f6a0f0f0 100644
--- a/docs/userguide/references.rst
+++ b/docs/userguide/references.rst
@@ -22,18 +22,27 @@ OPNFV
 References used in Test Cases
 =============================
 
+* cachestat: https://github.com/brendangregg/perf-tools/tree/master/fs
 * cirros-image: https://download.cirros-cloud.net
 * cyclictest: https://rt.wiki.kernel.org/index.php/Cyclictest
 * DPDKpktgen: https://github.com/Pktgen/Pktgen-DPDK/
 * DPDK supported NICs: http://dpdk.org/doc/nics
+* fdisk: http://www.tldp.org/HOWTO/Partition/fdisk_partitioning.html
 * fio: http://www.bluestop.org/fio/HOWTO.txt
+* free: http://manpages.ubuntu.com/manpages/trusty/en/man1/free.1.html
 * iperf3: https://iperf.fr/
+* iostat: http://linux.die.net/man/1/iostat
 * Lmbench man-pages: http://manpages.ubuntu.com/manpages/trusty/lat_mem_rd.8.html
 * Memory bandwidth man-pages: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html
-* unixbench: https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench
 * mpstat man-pages: http://manpages.ubuntu.com/manpages/trusty/man1/mpstat.1.html
+* netperf: http://www.netperf.org/netperf/training/Netperf.html
 * pktgen: https://www.kernel.org/doc/Documentation/networking/pktgen.txt
+* RAMspeed: http://alasir.com/software/ramspeed/
+* sar: http://linux.die.net/man/1/sar
 * SR-IOV: https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking
+* Storperf: https://wiki.opnfv.org/display/storperf/Storperf
+* unixbench: https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench
+
 
 Research
 ========
@@ -48,3 +57,4 @@ Standards
 * ETSI NFV: http://www.etsi.org/technologies-clusters/technologies/nfv
 * ETSI GS-NFV TST 001: https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST001_-_Pre-deployment_Validation/
 * RFC2544: https://www.ietf.org/rfc/rfc2544.txt
+