summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--.gitignore17
-rw-r--r--docs/dovetailtool/dovetail.tool.configtemplate.rst (renamed from docs/userguide/04-dovetail_config_template.rst)61
-rw-r--r--docs/dovetailtool/dovetail.tool.configuration.rst35
-rw-r--r--docs/dovetailtool/dovetail.tool.installation.rst154
-rw-r--r--docs/dovetailtool/dovetail.tool.overview.rst27
-rw-r--r--docs/dovetailtool/index.rst (renamed from docs/installationprocedure/index.rst)6
-rw-r--r--docs/installationprocedure/feature.configuation.rst45
-rw-r--r--docs/installationprocedure/installation.instruction.rst142
-rw-r--r--docs/userguide/index.rst1
-rw-r--r--dovetail/conf/cmd_config.yml45
-rw-r--r--dovetail/conf/dovetail_config.py36
-rw-r--r--dovetail/conf/dovetail_config.yml1
-rw-r--r--dovetail/conf/functest_config.yml4
-rw-r--r--dovetail/container.py10
-rw-r--r--dovetail/prepare_env.py40
-rw-r--r--dovetail/report.py8
-rwxr-xr-xdovetail/run.py44
-rw-r--r--dovetail/tests/__init__.py0
-rw-r--r--dovetail/tests/unit/__init__.py0
-rw-r--r--dovetail/tests/unit/test_parser.py57
-rw-r--r--dovetail/tests/unit/test_testcase.yaml10
21 files changed, 487 insertions, 256 deletions
diff --git a/.gitignore b/.gitignore
index ded60678..913e3713 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,8 +1,5 @@
*.py[cod]
-# C extensions
-*.so
-
# Packages
*.egg
*.egg-info
@@ -19,18 +16,20 @@ lib
lib64
__pycache__
-# Installer logs
+# Installation logs
pip-log.txt
-# Unit test / coverage reports
+# unittest/coverage reports
.coverage
.tox
nosetests.xml
+unittest_results.log
# Translations
*.mo
-# Mr Developer
-.mr.developer.cfg
-.project
-.pydevproject
+# Sphinx documentation
+*.sw?
+docs_build/
+docs_output/
+dovetail/results
diff --git a/docs/userguide/04-dovetail_config_template.rst b/docs/dovetailtool/dovetail.tool.configtemplate.rst
index fce99d21..9c0748a9 100644
--- a/docs/userguide/04-dovetail_config_template.rst
+++ b/docs/dovetailtool/dovetail.tool.configtemplate.rst
@@ -3,18 +3,19 @@
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+======================
Config Template Syntax
======================
-Dovetail uses Functest/Yardstick Docker container to run its testcases. So you need to give
-configurations for building the container and the commands it needs to do. In dovetail,
-all of these are defined in config yaml files ``dovetail/dovetail/conf/functest_config.yml``
-and ``dovetail/dovetail/conf/yardstick_config.yml``.
+For Dovetail tool, the config files are located in ``dovetail/dovetail/conf``, which are written
+in yaml format. As both functest and yardstick are utilized by Dovetail, their configuration files
+should be configured as follows, within the ``functest_config.yml`` and ``yardstick_config.yml`` files,
+respectively.
-Functest template syntax
-------------------------
+Functest config template syntax
+-------------------------------
-For example, you can define your ``functest_config.yml`` as:
+An example functest configuration is shown as follows:
::
@@ -41,29 +42,19 @@ For example, you can define your ``functest_config.yml`` as:
file_path: 'tempest/tempest.log'
db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
-First, you need to give the image that you want to use for building functest/yardstick container.
-Besides, there also need some envirnment parameters such as ``INSTALLER_TYPE`` and ``INSTALLER_IP``
-and the options for you container. Then the functest/yardstick container can be build with your
-settings.
-
-Second, there need three kinds of commands, ``pre_condition``, ``testcase`` and ``post_condition``.
-If you want to do some cleanups or preparations, the commands can be put into ``pre_condition``
-section orderly. All commands in this section will just be executed once in the begining.
-The ``testcase`` section does the main jobs of the testing. All functest testcases will use the
-container to execute these commands one by one. After finishing that, the test is accomplished
-and the results are stored in files or uploaded to database. The ``post_condition`` section
-does some work such as clean Docker images or something else after all testcases finished.
-All commands in this section will just execute once.
-
-Besides, there need a ``result`` section and it gives the directory of the functest/yardstick
-results. The ``store_type`` should be the same with the cmds in ``testcase``. That means if the
-test results are stored in files, then store_type need to be file and the file_path is also
-needed. If the test results are uploaded to database, then a db_url is needed for acquiring the results.
+- ``image_name`` and ``docker_tag`` sections define the docker image pulled from the public dockerhub.
+- ``envs`` should be correctly configed according to the SUT(System Under Test).
+- ``pre_condition`` represents some cleanups or preparations.
+ ``testcase`` represents the testcases running cmds.
+ ``post_condition`` represents some cleanups needed after all testcases finished.
+- ``result`` section gives the directory of the dovetail tool test result.
+ ``db_url`` gives the database URL of the dovetail results to be stored.
-Yardstick template syntax
--------------------------
+Yardstick config template syntax
+---------------------------------
-The framework of ``yardstick_config.yml`` is almost the same as ``functest_config.yml``.
+The configuration in ``yardstick_config.yml`` is similiar to ``functest_config.yml``,
+and an example is shown as follows:
::
@@ -97,12 +88,13 @@ The framework of ``yardstick_config.yml`` is almost the same as ``functest_confi
file_path: 'yardstick.log'
db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
-The main differences between ``yardstick_config.yml`` and ``functest_config.yml`` are the commands.
+The main differences between ``yardstick_config.yml`` and ``functest_config.yml``
+are the ``cmds`` subsection.
Jinja2 template syntax
----------------------
-Note that you can use jinja2 template for your parameters such as ``{{script_testcase}}``. The
+Jinja2 module can be used to config the ``{{script_testcase}}``. The
parameters are defined in ``dovetail/dovetail/conf/dovetail_config.yml``:
::
@@ -113,8 +105,8 @@ parameters are defined in ``dovetail/dovetail/conf/dovetail_config.yml``:
- name: script_testcase
path: '("scripts", "testcase")'
-Here ``path`` is the path in testcase config files that you can find the value of parameters. Take
-``script_testcase`` as the example. For testcase dovetail.ipv6.tc001:
+Here ``path`` is the path defined in the testcase configuration files.
+Take ``script_testcase`` as an example. For testcase ``dovetail.ipv6.tc001``:
::
@@ -129,6 +121,5 @@ Here ``path`` is the path in testcase config files that you can find the value o
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
-The path ("scripts", "testcase") means 'testcase' subsection of 'scripts' section. So follow
-the path ("scripts", "testcase") we can get the value of ``{{script_testcase}}`` that is
-'tempest_smoke_serial'.
+The path ("scripts", "testcase") means 'testcase' is the subsection of 'scripts' section. From above,
+by following the path ("scripts", "testcase") we can get the value of ``{{script_testcase}}`` is 'tempest_smoke_serial'.
diff --git a/docs/dovetailtool/dovetail.tool.configuration.rst b/docs/dovetailtool/dovetail.tool.configuration.rst
new file mode 100644
index 00000000..8e97e73c
--- /dev/null
+++ b/docs/dovetailtool/dovetail.tool.configuration.rst
@@ -0,0 +1,35 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+=========================
+Testcase Template Syntax
+=========================
+
+The testcases used for compliance and certification are defined in the ``dovetail/testcase`` directory,
+which are defined in yaml format. Take the testcase ``ipv6.tc001.yml`` as an example, it is shown as:
+
+::
+
+ dovetail.ipv6.tc001:
+ name: dovetail.ipv6.tc001
+ objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
+ - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
+ - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
+
+- At least three sections named 'name', 'objective', 'scripts' must be included
+- Section 'name' distinguishes different test cases used for compliance and certification,
+ and should start with ``dovetail.``
+- Section 'objective' describes what this testcase does
+- Section 'scripts' has subsections such as 'type', 'testcase' and 'sub_testcase_list'
+- Two kinds of 'type' is supported by now, functest and yardstick
+- For functest, the 'testcase' represents the testcases in slicing/tier,
+ 'sub_testcase_list' represents the testcases in this slicing compliance and certification will use.
+ For yardstick, since it is not sliced by now, 'sub_testcase_list' is not needed, only to edit the 'testcase' part
+ such as ``yardstick_tc027``
diff --git a/docs/dovetailtool/dovetail.tool.installation.rst b/docs/dovetailtool/dovetail.tool.installation.rst
new file mode 100644
index 00000000..dcd856bc
--- /dev/null
+++ b/docs/dovetailtool/dovetail.tool.installation.rst
@@ -0,0 +1,154 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+===========================
+Dovetail Tool Installation
+===========================
+
+Abstract
+########
+
+The Dovetail tool supports installation either directly on the Linux host, or within a Docker container.
+The detailed installation procedure on the Linux host or via the Docker container are shown
+in the following sections.
+
+There is a need to config the following parameters before Dovetail tool
+running on the SUT (System Under Test):
+
+::
+
+ SUT_TYPE, SUT type, e.g., apex, compass, fuel, joid, etc
+ SUT_IP, SUT external network IP, e.g., 192.168.200.2
+ NODE_NAME, this can be shown in the test result for users to see which pod the dovetail tool runs
+ DEPLOY_SCENARIO, deployment scenario, e.g., os-nosdn-nofeature-ha
+ BUILD_TAG, this can be shown in the test result for users to identify logs
+ CI_DEBUG, true for debug information printed and false for not printed
+ DEPLOY_TYPE, baremetal or virtual
+
+Dovetail tool installation on local Linux host environment
+##########################################################
+
+In order to install Dovetail into a local Linux host environment, the following steps should
+be performed:
+
+Downloading Dovetail tool
+--------------------------
+
+Source code downloading:
+
+::
+
+ git clone https://gerrit.opnfv.org/gerrit/dovetail
+
+Environment preparation
+-----------------------
+
+Dovetail configuration:
+
+::
+
+ cd {dovetail_path}/dovetail/conf
+ vim functest_config.yml
+ vim yardstick_config.yml
+
+To modify the ``envs`` parameters, e.g., SUT_TYPE, SUT_IP, NODE_NAME,
+DEPLOY_SCENARIO, BUILD_TAG, CI_DEBUG, DEPLOY_TYPE, in the yml files.
+
+System dependencies are installed via the ``prepare_env.py`` file, and you will need ``sudo``
+access to complete the installation.
+
+::
+
+ cd {dovetail_path}/dovetail/dovetail
+ python prepare_env.py
+
+At this point the environment is now ready for Dovetail execution.
+
+Compliance and certification test cases
+----------------------------------------
+
+The compliance and certification test cases can be defined under the ``/dovetail/cert``
+directory, which is defined in yaml format.
+A sample file named ``basic.yml`` is provided as follows:
+
+::
+
+ certification_basic:
+ name: certification_basic
+ testcase_list:
+ - dovetail.ipv6.tc001
+
+The testcase listed here is dovetail.ipv6.tc001, defined within ``dovetail/testcase``.
+
+Note: if a new test case yaml file is created, its name should start with ``certification_``,
+in similiar fashion as the sample file ``certification_basic``.
+
+Running Dovetail tool
+---------------------
+
+After environment preparation is complete and test cases added, the Dovetail tool can be run with
+
+::
+
+ python run.py --scenario basic
+
+The value ``basic`` passed to the ``scenario`` flag can be replaced with the test cases yaml file.
+If not argument is given, the basic scenario will be run as the default.
+
+Running Dovetail in a Docker container
+########################################
+
+The Dovetail tool can be run in a Docker container by utilizing the following steps:
+
+Pull Dovetail Docker image from public Dockerhub
+------------------------------------------------
+
+::
+
+ sudo docker pull opnfv/dovetail:<Tag>
+
+<Tag> here is the version, 'latest' is used for the master branch.
+
+Dovetail Docker container creation
+----------------------------------
+
+Next, create the ``dovetail-docker-env`` file to define the environment parameters ::
+
+ INSTALLER_TYPE=compass
+ INSTALLER_IP=192.168.200.2
+ DEPLOY_TYPE=baremetal
+ DEPLOY_SCENARIO=ha-nosdn
+ CI_DEBUG=true
+
+Then to instantiate the Dovetail Docker container, execute::
+
+ sudo docker run --privileged=true --rm -t \
+ --env-file dovetail-docker-env \
+ -v /home/opnfv/dovetail/results:/home/opnfv/dovetail/results \
+ -v /var/run/docker.sock:/var/run/docker.sock \
+ --name <Dovetail_Container_Name> \
+ opnfv/dovetail:<Tag> /bin/bash
+
+To attach dovetail container and Running test cases
+----------------------------------------------------
+
+Before connecting to the container, you can check the container status by running ::
+
+ docker ps -a
+
+Attach to the container by starting it and obtaining a bash prompt with ::
+
+ docker exec -it <Dovetail_Container_Name> bash
+
+Inside the container the following commands can be executed to trigger the testcases ::
+
+ cd /home/opnfv/dovetail/dovetail
+ python run.py --scenario basic
+
+Results Output
+###############
+
+The running log is stored in ``/home/opnfv/dovetail/results/dovetail.log``.
+The certification report is stored in ``/home/opnfv/dovetail/results/dovetail_report.txt``.
diff --git a/docs/dovetailtool/dovetail.tool.overview.rst b/docs/dovetailtool/dovetail.tool.overview.rst
new file mode 100644
index 00000000..a1133f34
--- /dev/null
+++ b/docs/dovetailtool/dovetail.tool.overview.rst
@@ -0,0 +1,27 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+==============
+Dovetail Tool
+==============
+
+What is Dovetail tool
+#####################
+
+A toolset for providing an evaluation of a deployed cloud environment for OPNFV certification.
+
+Overview
+########
+
+Dovetail tool intends on being THE source of tools to provide evaluation of the use of OPNFV trademarks.
+
+Dovetail tool provides users with a wrapper which leverage other test projects, such as Functest,
+Yardstick, etc, that helps to provide evaluation of the use of OPNFV trademarks.
+It does so by validating any NFV(Network Function Virtualization) implementation against the test cases in Dovetail.
+
+Get involved!
+#############
+- Wiki: https://wiki.opnfv.org/display/dovetail/Dovetail+Home
+- Git Repo: https://git.opnfv.org/dovetail
diff --git a/docs/installationprocedure/index.rst b/docs/dovetailtool/index.rst
index 9a755800..a025cd50 100644
--- a/docs/installationprocedure/index.rst
+++ b/docs/dovetailtool/index.rst
@@ -10,5 +10,7 @@ Dovetail Overview
.. toctree::
:maxdepth: 2
- installation.instruction
- feature.configuation
+ dovetail.tool.overview.rst
+ dovetail.tool.installation.rst
+ dovetail.tool.configuation.rst
+ dovetail.tool.configtemplate.rst
diff --git a/docs/installationprocedure/feature.configuation.rst b/docs/installationprocedure/feature.configuation.rst
deleted file mode 100644
index 0bc4ae33..00000000
--- a/docs/installationprocedure/feature.configuation.rst
+++ /dev/null
@@ -1,45 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
-
-Testcase Template Syntax
-=========================
-
-The testcases certification are defined in ``dovetail/dovetail/testcase``.
-Take testcase ``ipv6.tc001.yml`` as an example.
-
-Basic template syntax
----------------------
-
-::
-
- dovetail.ipv6.tc001:
- name: dovetail.ipv6.tc001
- objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
- scripts:
- type: functest
- testcase: tempest_smoke_serial
- sub_testcase_list:
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
- - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
-
-The testcase needs at least three keys named as 'name', 'objective' and 'scripts'. The whole
-dovetail project can just recognize testcases by the 'name' section, which here is
-'dovetail.ipv6.tc001'. The 'objective' describes what exactly this testcase does. In the
-'scripts' part, there are some subsections such as 'type', 'testcase' and 'sub_testcase_list'.
-Dovetail now supports two kinds of types, functest and yardstick. If you define the type as
-functest, then you need to give the functest testcase that you want to use. If the type is
-yardstick, then a yardstick testcase is needed. The 'sub_testcase_list' lists the sub_testcases
-that you put a high value on. Even though the whole testcase faild, we still think it passed
-when all the sub_testcases pass. The sub_testcase_list is just available for functest. When
-for yardstick testcase, retain 'sub_testcase_list' section and keep the content empty.
-
-::
-
- sub_testcase_list:
- -
-
-This is the definition of the dovetail testcases. They can just be tested by adding into
-scenarios such as ``cert/basic.yml``.
diff --git a/docs/installationprocedure/installation.instruction.rst b/docs/installationprocedure/installation.instruction.rst
deleted file mode 100644
index aa1b436a..00000000
--- a/docs/installationprocedure/installation.instruction.rst
+++ /dev/null
@@ -1,142 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
-
-=====================
-Dovetail Installation
-=====================
-
-Abstract
-========
-
-Dovetail currently supports installation on Ubuntu 14.04 or by using a docker image.
-Detailed steps about installing dovetail can be found below.
-
-To use dovetail you should have access to an OpenStack environment,
-with at least Nova, Neutron, Glance, Keystone and Heat installed.
-
-
-Run dovetail on local machine
-=============================
-
-The steps needed to run dovetail on Ubuntu are:
-
-1. Download source code and prepare environment.
-2. Create certification configuration file.
-3. Run certification.
-
-
-Download dovetail and prepare environment
------------------------------------------
-
-The source code of dovetail can be got from Gerrit:
-
-::
-
- git clone https://gerrit.opnfv.org/gerrit/dovetail
-
-After that you need to prepare the environment which means install some tools,
-modules and anything else dovetail needs. A file named ``prepare_env.py`` can
-do all of these for you.
-
-::
-
- cd dovetail/dovetail/
- python prepare_env.py
-
-Now the environment should be prepared for running the certification.
-
-Besides, you may want to install it in a python virtualenv, that is beyond the scope
-of this installation instruction.
-
-Create certification configuration file
----------------------------------------
-
-You can wirte a certification configuration file with the format of yaml,
-which can be defined to meet your own requirements. The yaml file should
-be located in the folder ``dovetail/dovetail/cert``. There provide a sample
-file named ``basic.yml``.
-
-::
-
- certification_basic:
- name: certification_basic
- testcase_list:
- - dovetail.ipv6.tc001
-
-As you can see, the configuration file combines the testcases you want to run
-together, and all of them should already been defined in ``dovetail/dovetail/testcase``.
-The name of the configuration file should be certification_%s, where %s (here is basic)
-can be used to run this certification.
-
-
-Run the certification
----------------------
-
-When you get the dovetail source code, prepare the environment and define the
-certification configuration file, you can then run the certification. Take
-certification_basic as the example. You can run it using the file ``run.py`` with
-basic (scenario name) as its argument.
-
-::
-
- python run.py --scenario basic
-
-If you did not give an argument, it will be default set as basic.
-
-
-Run dovetail on Docker container
-================================
-
-steps:
-1. pull image
-2. run container
-3. config
-4. run testcase
-
-pull image
-----------
-
-Get the latest docker image from docker hub.
-
-::
- sudo docker pull opnfv/dovetail:latest
-
-run container
--------------
-
-Start a container from the image that you just pulled.
-
-::
- sudo docker run -it --privileged=true -v /home/opnfv/dovetail/results:/home/opnfv/dovetail/results
- -v /var/run/docker.sock:/var/run/docker.sock opnfv/dovetail:latest "/bin/bash"
-
-config
-------
-
-For now, we need only change value of ``INSTALLER_TYPE`` and ``INSTALLER_IP``
-of the variable ``envs`` to your own environment.
-
-::
- sudo docker exec -it $(CONTAINER_ID) "/bin/bash"
- vi /home/opnfv/dovetail/dovetail/conf/functest_config.yml
- vi /home/opnfv/dovetail/dovetail/conf/yardstick_config.yml
-
-run testcase
-------------
-
-In docker, you just run following cmds to execute the testcase.
-
-::
- cd /home/opnfv/dovetail/dovetail
- python run.py
-
-
-
-
-Output
-======
-
-The running log is stored in ``/home/opnfv/dovetail/results/dovetail.log``.
-The report of certification is stored in ``/home/opnfv/dovetail/results/dovetail_report.txt``.
diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst
index 0dcf55ce..03246e7b 100644
--- a/docs/userguide/index.rst
+++ b/docs/userguide/index.rst
@@ -14,4 +14,3 @@ Dovetail - user guide
./01-introduction.rst
./02-certification_criteria.rst
./03-certification_progress.rst
- ./04-dovetail_config_template.rst
diff --git a/dovetail/conf/cmd_config.yml b/dovetail/conf/cmd_config.yml
new file mode 100644
index 00000000..63d51ed0
--- /dev/null
+++ b/dovetail/conf/cmd_config.yml
@@ -0,0 +1,45 @@
+cli:
+ arguments:
+ envs:
+ # This is a simple example of arguments.
+ # Dovetail has no need of this kind of parameters currently.
+ # The arguments must be given orderly at the run-time.
+ #
+ # docker_tag:
+ # flags: 'docker_tag'
+ non-envs:
+
+ options:
+ envs:
+ SUT_TYPE:
+ flags:
+ - '--SUT_TYPE'
+ - '-t'
+ help: 'Installer type of the system under test (SUT).'
+ SUT_IP:
+ flags:
+ - '--SUT_IP'
+ - '-i'
+ help: 'IP of the system under test (SUT).'
+ DEPLOY_SCENARIO:
+ flags:
+ - '--DEPLOY_SCENARIO'
+ - '-S'
+ help: 'DEPLOY_SCENARIO of the system under test (SUT).'
+ DEPLOY_TYPE:
+ flags:
+ - '--DEPLOY_TYPE'
+ - '-T'
+ help: 'DEPLOY_TYPE of the system under test (SUT).'
+ CI_DEBUG:
+ flags:
+ - '--CI_DEBUG'
+ - '-d'
+ help: 'CI_DEBUG for showing debug log.'
+ non-envs:
+ scenario:
+ flags:
+ - '--scenario'
+ - '-s'
+ default: 'basic'
+ help: 'certification scenario.'
diff --git a/dovetail/conf/dovetail_config.py b/dovetail/conf/dovetail_config.py
index 6f3eebf2..03d0bb78 100644
--- a/dovetail/conf/dovetail_config.py
+++ b/dovetail/conf/dovetail_config.py
@@ -9,6 +9,7 @@
import yaml
import os
+import re
CERT_PATH = './cert/'
TESTCASE_PATH = './testcase/'
@@ -27,3 +28,38 @@ container_config = {}
container_config['functest'] = dovetail_config['functest']
container_config['yardstick'] = dovetail_config['yardstick']
+
+
+with open(os.path.join(curr_path, dovetail_config['cli_file_name'])) as f:
+ cmd_yml = yaml.safe_load(f)
+ dovetail_config['cli'] = cmd_yml[cmd_yml.keys()[0]]
+
+
+def cmd_name_trans(cmd_name):
+ key = cmd_name.upper()
+ if key == 'SUT_TYPE':
+ key = 'INSTALLER_TYPE'
+ if key == 'SUT_IP':
+ key = 'INSTALLER_IP'
+ return key
+
+
+def update_envs(options):
+ for item in options:
+ if options[item] is not None:
+ key = cmd_name_trans(item)
+ os.environ[key] = options[item]
+ update_config_envs('functest', key)
+ update_config_envs('yardstick', key)
+
+
+def update_config_envs(script_type, key):
+ old_value = re.findall(r'\s+%s=(.*?)(\s+|$)' % key,
+ dovetail_config[script_type]['envs'])
+ if old_value == []:
+ dovetail_config[script_type]['envs'] += \
+ ' -e ' + key + '=' + os.environ[key]
+ else:
+ dovetail_config[script_type]['envs'] = \
+ dovetail_config[script_type]['envs'].replace(old_value[0][0],
+ os.environ[key])
diff --git a/dovetail/conf/dovetail_config.yml b/dovetail/conf/dovetail_config.yml
index 901988f8..1f5de672 100644
--- a/dovetail/conf/dovetail_config.yml
+++ b/dovetail/conf/dovetail_config.yml
@@ -2,6 +2,7 @@
work_dir: /home/opnfv/dovetail
result_dir: /home/opnfv/dovetail/results
report_file: 'dovetail_report.txt'
+cli_file_name: 'cmd_config.yml'
# used for testcase cmd template in jinja2 format
# we have two variables available now
diff --git a/dovetail/conf/functest_config.yml b/dovetail/conf/functest_config.yml
index cd33dc59..ef2b7fb8 100644
--- a/dovetail/conf/functest_config.yml
+++ b/dovetail/conf/functest_config.yml
@@ -5,14 +5,10 @@ functest:
-e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
-e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal'
opts: '-id --privileged=true'
- result_dir: '/home/opnfv/functest/results'
pre_condition:
cmds:
- 'echo test for precondition'
testcase:
- pre_cmd: 'python /home/opnfv/repos/functest/ci/prepare_env.py start'
- exec_cmd: 'python /home/opnfv/repos/functest/ci/run_tests.py -t {{script_testcase}} -r'
- post_cmd: ''
cmds:
- 'python /home/opnfv/repos/functest/ci/prepare_env.py start'
- 'python /home/opnfv/repos/functest/ci/run_tests.py -t {{script_testcase}} -r'
diff --git a/dovetail/container.py b/dovetail/container.py
index 6ff3980d..6d7ac94d 100644
--- a/dovetail/container.py
+++ b/dovetail/container.py
@@ -7,7 +7,7 @@
# http://www.apache.org/licenses/LICENSE-2.0
#
-import os
+
import utils.dovetail_logger as dt_logger
import utils.dovetail_utils as dt_utils
from conf.dovetail_config import dovetail_config
@@ -39,18 +39,12 @@ class Container:
# sshkey="-v /root/.ssh/id_rsa:/root/.ssh/id_rsa "
docker_image = cls.get_docker_image(type)
envs = dovetail_config[type]['envs']
- import re
- for i in ['INSTALLER_TYPE', 'DEPLOY_SCENARIO',
- 'DEPLOY_TYPE', 'CI_DEBUG']:
- envs = re.sub("%s=(\w+)" % i, '%s=%s' % (i, os.getenv(i)), envs)
- envs = re.sub("INSTALLER_IP=\d+\.?\d+\.?\d+\.?\d+",
- 'INSTALLER_IP=' + os.getenv('INSTALLER_IP'), envs)
opts = dovetail_config[type]['opts']
sshkey = ''
result_volume = ' -v %s:%s ' % (dovetail_config['result_dir'],
dovetail_config[type]['result']['dir'])
cmd = 'sudo docker run %s %s %s %s %s /bin/bash' % \
- (opts, envs, sshkey, result_volume, docker_image)
+ (opts, envs, sshkey, result_volume, docker_image)
dt_utils.exec_cmd(cmd, logger)
ret, container_id = \
dt_utils.exec_cmd("sudo docker ps | grep " + docker_image +
diff --git a/dovetail/prepare_env.py b/dovetail/prepare_env.py
index 785d5c3d..3e4d6964 100644
--- a/dovetail/prepare_env.py
+++ b/dovetail/prepare_env.py
@@ -7,13 +7,51 @@
# http://www.apache.org/licenses/LICENSE-2.0
#
+import platform
import utils.dovetail_logger as dt_logger
import utils.dovetail_utils as dt_utils
+def get_os():
+ """Get distro name.
+
+ :returns: return distro name as a string
+ """
+ return platform.dist()[0]
+
+
+def get_install_bin(os):
+ """Get install command binary.
+
+ :returns: return install command according to distro
+ """
+ if os in ['centos', 'redhat']:
+ return 'yum'
+ elif os == 'fedora':
+ return 'dnf'
+ elif os == 'ubuntu':
+ return 'apt-get'
+ else:
+ return None
+
+
+def get_docker_pkgname(os):
+ """Get docker package name.
+
+ :returns: return docker package name according to distro
+ """
+ if os in ['centos', 'fedora', 'redhat']:
+ return 'docker'
+ elif os == 'ubuntu':
+ return 'docker.io'
+ else:
+ return None
+
logger = dt_logger.Logger('prepare_env.py').getLogger()
-cmd = "sudo apt-get -y install docker.io python-pip"
+os_name = get_os()
+cmd = "sudo %s -y install %s python-pip" \
+ % (get_install_bin(os_name), get_docker_pkgname(os_name))
dt_utils.exec_cmd(cmd, logger)
cmd = "sudo pip install click pyyaml jinja2"
diff --git a/dovetail/report.py b/dovetail/report.py
index c937011f..9ea90231 100644
--- a/dovetail/report.py
+++ b/dovetail/report.py
@@ -64,14 +64,14 @@ class Report:
# save to disk as default
@classmethod
def save(cls, report):
- report_file_path = dovetail_config['report_file']
+ report_file_name = dovetail_config['report_file']
try:
with open(os.path.join(dovetail_config['result_dir'],
- report_file_path), 'w') as report_file:
+ report_file_name), 'w') as report_file:
report_file.write(report)
- logger.info('save report to %s' % report_file_path)
+ logger.info('save report to %s' % report_file_name)
except Exception:
- logger.error('Failed to save: %s' % report_file_path)
+ logger.error('Failed to save: %s' % report_file_name)
@classmethod
def get_result(cls, testcase):
diff --git a/dovetail/run.py b/dovetail/run.py
index ddeb3117..39dec07c 100755
--- a/dovetail/run.py
+++ b/dovetail/run.py
@@ -18,6 +18,8 @@ from testcase import Testcase
from testcase import Scenario
from report import Report
from conf.dovetail_config import SCENARIO_NAMING_FMT
+from conf.dovetail_config import dovetail_config
+from conf.dovetail_config import update_envs
logger = dt_logger.Logger('run.py').getLogger()
@@ -69,17 +71,49 @@ def run_test(scenario):
Report.check_result(testcase, db_result)
-@click.command()
-@click.option('--scenario', default='basic', help='certification scenario')
-def main(scenario):
+def filter_env_options(input_dict):
+ envs_options = {}
+ for key, value in input_dict.items():
+ key = key.upper()
+ if key in dovetail_config['cli']['options']['envs']:
+ envs_options[key] = value
+ return envs_options
+
+
+def main(*args, **kwargs):
"""Dovetail certification test entry!"""
logger.info('=======================================')
- logger.info('Dovetail certification: %s!' % scenario)
+ logger.info('Dovetail certification: %s!' % (kwargs['scenario']))
logger.info('=======================================')
+ envs_options = filter_env_options(kwargs)
+ update_envs(envs_options)
+ logger.info('Your new envs for functest: %s' %
+ dovetail_config['functest']['envs'])
+ logger.info('Your new envs for yardstick: %s' %
+ dovetail_config['yardstick']['envs'])
load_testcase()
- scenario_yaml = load_scenario(scenario)
+ scenario_yaml = load_scenario(kwargs['scenario'])
run_test(scenario_yaml)
Report.generate(scenario_yaml)
+
+CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])
+if dovetail_config['cli']['options'] is not None:
+ for key, value in dovetail_config['cli']['options'].items():
+ if value is not None:
+ for k, v in value.items():
+ flags = v['flags']
+ del v['flags']
+ main = click.option(*flags, **v)(main)
+if dovetail_config['cli']['arguments'] is not None:
+ for key, value in dovetail_config['cli']['arguments'].items():
+ if value is not None:
+ for k, v in value.items():
+ flags = v['flags']
+ del v['flags']
+ main = click.argument(flags, **v)(main)
+main = click.command(context_settings=CONTEXT_SETTINGS)(main)
+
+
if __name__ == '__main__':
main()
diff --git a/dovetail/tests/__init__.py b/dovetail/tests/__init__.py
new file mode 100644
index 00000000..e69de29b
--- /dev/null
+++ b/dovetail/tests/__init__.py
diff --git a/dovetail/tests/unit/__init__.py b/dovetail/tests/unit/__init__.py
new file mode 100644
index 00000000..e69de29b
--- /dev/null
+++ b/dovetail/tests/unit/__init__.py
diff --git a/dovetail/tests/unit/test_parser.py b/dovetail/tests/unit/test_parser.py
new file mode 100644
index 00000000..5b003d1a
--- /dev/null
+++ b/dovetail/tests/unit/test_parser.py
@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+#
+# lingui.zeng@huawei.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+"""
+Test 'parser' module
+
+"""
+
+import logging
+import os
+import unittest
+
+import yaml
+
+import dovetail.parser as dovetail_parser
+
+
+class TestParser(unittest.TestCase):
+
+ test_path = os.path.dirname(os.path.realpath(__file__))
+
+ def setUp(self):
+ """Test case setup"""
+ logging.disable(logging.CRITICAL)
+
+ def test_parser_cmd(self):
+ """Test whether the command is correctly parsed."""
+ mock_cmd = "python /functest/ci/run_tests.py -t {{script_testcase}} -r"
+ with open(os.path.join(self.test_path, 'test_testcase.yaml')) as f:
+ mock_testcase_yaml = yaml.safe_load(f)
+ MockTestcase = type('Testcase', (object,), {})
+ mock_testcase = MockTestcase()
+ mock_testcase.testcase = mock_testcase_yaml.values()[0]
+ output = dovetail_parser.Parser.parse_cmd(mock_cmd, mock_testcase)
+ expected_output = ("python /functest/ci/run_tests.py -t "
+ "tempest_smoke_serial -r")
+ self.assertEqual(expected_output, output)
+
+ def test_parser_cmd_fail(self):
+ """Test whether the command is correctly parsed."""
+ mock_cmd = "python /functest/ci/run_tests.py -t {{script_testcase}} -r"
+ mock_testcase_yaml = {}
+ MockTestcase = type('Testcase', (object,), {})
+ mock_testcase = MockTestcase()
+ mock_testcase.testcase = mock_testcase_yaml.values()
+ output = dovetail_parser.Parser.parse_cmd(mock_cmd, mock_testcase)
+ expected_output = ("python /functest/ci/run_tests.py -t "
+ "None -r")
+ self.assertEqual(expected_output, output)
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/dovetail/tests/unit/test_testcase.yaml b/dovetail/tests/unit/test_testcase.yaml
new file mode 100644
index 00000000..1b03262f
--- /dev/null
+++ b/dovetail/tests/unit/test_testcase.yaml
@@ -0,0 +1,10 @@
+dovetail.ipv6.tc001:
+ name: dovetail.ipv6.tc001
+ objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
+ scripts:
+ type: functest
+ testcase: tempest_smoke_serial
+ sub_testcase_list:
+ - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
+ - tempest.api.network.test_networks.BulkNetworkOpsIpV7Test.test_bulk_create_delete_port
+ - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet