.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0


****************
Run with Ansible
****************

QTIP benchmarking tasks are built upon `Ansible`_ playbooks and roles. If you are familiar with Ansible, it is possible
to run it with ``ansible-playbook`` command.

.. _Ansible: https://www.ansible.com/

Create workspace
================

There is a playbook in ``tests/integration`` used for creating a new workspace for QTIP benchmarking::

    cd qtip/tests/integration
    ansible-playbook workspace-create.yml

You will be prompted for required information for the Pod under test::

    (qtip) ➜  integration git:(master) ✗ ansible-playbook workspace-create.yml
    name of the pod under test (used in reporting) [qtip-pod]:
    scenario deployed in the pod: [default]:
    installer type of the pod (apex|fuel|other) [fuel]:
    master host/vm of the installer (accessible by `ssh <hostname>`): f5
    workspace name (new directory will be created) [workspace]:

    PLAY [localhost] ***************************************************************

    TASK [qtip-workspace : generating workspace] ***********************************

    PLAY RECAP *********************************************************************
    localhost                  : ok=0    changed=0    unreachable=0    failed=0


You may hit ``Enter`` to accept default value

NOTE: if this playbook is moved to other directory, configuration in ``ansible.cfg`` needs to be updated accordingly.
The ansible roles from QTIP, i.e. ``<path_of_qtip>/resources/ansible_roles`` must be added to ``roles_path`` in
`Ansible configuration file`_. For example::

    roles_path = ../../resources/ansible_roles

.. _Ansible configuration file:

Executing benchmark
===================

Before executing the setup playbook, make sure `~/.ssh/config` has been configured properly so that you can login the
**master node** "directly". Skip next section, if you can login with ``ssh <master-host>`` from localhost,

SSH access to master node
-------------------------

It is common that the master node is behind some jump host. In this case, ssh option ``ProxyCommand`` and ``ssh-agent``
shall be required.

Assume that you need to login to deploy server, then login to the master node from there. An example configuration is
as following::

    Host fuel-deploy
      HostName 172.50.0.250
      User root

    Host fuel-master
      HostName 192.168.122.63
      User root
      ProxyCommand ssh -o 'ForwardAgent yes' apex-deploy 'ssh-add && nc %h %p'

If several jumps are required to reach the master node, we may chain the jump hosts like below::

    Host jumphost
      HostName 10.62.105.31
      User zte
      Port 22

    Host fuel-deploy
      HostName 172.50.0.250
      User root
      ProxyJump jumphost

    Host fuel-master
      HostName 192.168.122.63
      User root
      ProxyCommand ssh -o 'ForwardAgent yes' apex-deploy 'ssh-add && nc %h %p'

NOTE: ``ProxyJump`` is equivalent to the long ``ProxyCommand`` option, but it is only available since OpenSSH 7.3

Setup testing environment
-------------------------

Run the setup playbook to generate ansible inventory of system under test by querying the slave nodes from the installer
master::

    cd workspace
    ansible-playbook setup.yml

Currently, QTIP supports automatic discovery from `apex`_ and `fuel`_

.. _apex: https://wiki.opnfv.org/display/apex
.. _fuel: https://wiki.opnfv.org/display/fuel

It will update the ``hosts`` and ``ssh.cfg``

Run the tests
-------------

Run the benchmarks with the following command::

    ansible-playbook run.yml

CAVEAT: QTIP will install required packages in system under test.

Inspect the results
-------------------

The test results and calculated output are stored in ``results``::

    current/
        node-2/
            arithmetic/
                metric.json
                report
                unixbench.log
            dpi/
            ...
        node-4/
        ...
        qtip-pod-qpi.json
    qtip-pod-20170425-1710/
    qtip-pod-20170425-1914/
    ...

The folders are named as ``<pod_name>-<start_time>/`` and the results are organized by *hosts* under test. Inside each
host, the test data are organized by metrics as defined in QPI specification.

For each metrics, it usually includes the following content

* log file generated by the performance testing tool
* metrics collected from the log files
* reported rendered with the metrics collected

Teardown the test environment
-----------------------------

QTIP will create temporary files for testing in system under test. Execute the teardown playbook to clean it up::

    ansible-playbook teardown.yml