1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
|
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
Executing Functest suites
=========================
As mentioned in the configuration guide `[1]`_, Alpine docker containers have
been introduced in Euphrates.
Tier containers have been created.
Assuming that you pulled the container and your environement is ready, you can
simply run the tiers by typing (e.g. with functest-healthcheck)::
sudo docker run --env-file env \
-v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
-v $(pwd)/images:/home/opnfv/functest/images \
opnfv/functest-healthcheck
You should get::
+----------------------------+------------------+---------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+----------------------------+------------------+---------------------+------------------+----------------+
| connection_check | functest | healthcheck | 00:02 | PASS |
| api_check | functest | healthcheck | 03:19 | PASS |
| snaps_health_check | functest | healthcheck | 00:46 | PASS |
+----------------------------+------------------+---------------------+------------------+----------------+
You can run functest-healcheck, functest-smoke, functest-features,
functest-components and functest-vnf.
The result tables show the results by test case, it can be::
* PASS
* FAIL
* SKIP: if the scenario/installer does not support the test case
Manual run
==========
If you want to run the test step by step, you may add docker option then run the
different commands within the docker.
Considering the healthcheck example, running functest manaully means::
sudo docker run -ti --env-file env \
-v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
-v $(pwd)/images:/home/opnfv/functest/images \
opnfv/functest-healthcheck /bin/bash
The docker prompt shall be returned. Then within the docker run the following
commands::
$ source /home/opnfv/functest/conf/openstack.creds
Prepare environment
-------------------
Prior to commencing the Functest environment preparation, we can check
the initial status of the environment. Issue the **functest env status**
command at the prompt::
# functest env status
# Functest environment is not installed.
To prepare the Functest docker container for test case execution, type::
# functest env prepare
# ...
# Functest environment ready to run tests.
you may also type prepare_env instead of functest env prepare.
To list some basic information about an already prepared Functest
docker container environment, issue the **functest env show** at the
prompt::
bash-4.3# functest env show
+------------------------------+---------------------------------+
| FUNCTEST ENVIRONMENT | VALUE |
+------------------------------+---------------------------------+
| STATUS | ready |
| SCENARIO | os-nosdn-nofeature-noha |
| DEBUG FLAG | false |
| BUILD_TAG | None |
| INSTALLER | compass |
| POD | huawei-pod1 |
+------------------------------+---------------------------------+
See configuration guide for details on Functest environnement variables.
Tier
----
Each Alpine container provided on the docker hub matches with a tier.
The following commands are available::
# functest tier list
- 0. healthcheck:
['connection_check', 'api_check', 'snaps_health_check']
# functest tier show healthcheck
+---------------------+---------------+--------------------------+-------------------------------------------------+------------------------------------+
| TIERS | ORDER | CI LOOP | DESCRIPTION | TESTCASES |
+---------------------+---------------+--------------------------+-------------------------------------------------+------------------------------------+
| healthcheck | 0 | (daily)|(weekly) | First tier to be executed to verify the | connection_check api_check |
| | | | basic operations in the VIM. | snaps_health_check |
+---------------------+---------------+--------------------------+-------------------------------------------------+------------------------------------+
To run all the cases of the tier, type::
# functest tier run healthcheck
Testcase
--------
Testcases can be listed, shown and run though the CLI::
# functest testcase list
connection_check
api_check
snaps_health_check
# functest testcase show api_check
+-------------------+--------------------------------------------------+------------------+---------------------------+
| TEST CASE | DESCRIPTION | CRITERIA | DEPENDENCY |
+-------------------+--------------------------------------------------+------------------+---------------------------+
| api_check | This test case verifies the retrieval of | 100 | ^((?!netvirt).)*$ |
| | OpenStack clients: Keystone, Glance, | | |
| | Neutron and Nova and may perform some | | |
| | simple queries. When the config value of | | |
| | snaps.use_keystone is True, functest | | |
| | must have access to the cloud's private | | |
| | network. | | |
+-------------------+--------------------------------------------------+------------------+---------------------------+
# functest testcase run connection_check
...
# functest run all
You can also type run_tests -t all to run all the tests.
Note the list of test cases depend on the installer and the scenario.
Reporting results to the test Database
======================================
In OPNFV CI we collect all the results from CI. A test APi shall be available
as well as a test database `[17]`_.
Functest internal API
=====================
An internal API has been introduced in Euphrates. The goal is to trigger
Functest operations through an API in addition of the CLI.
This could be considered as a first step towards a pseudo micro services
approach where the different test projects could expose and consume APIs to the
other test projects.
In Euphrates the main method of the APIs are:
* Show credentials
* Update openrc file
* Show environment
* Update hosts info for domain name
* Prepare environment
* List all testcases
* Show a testcase
* Run a testcase
* List all tiers
* Show a tier
* List all testcases within given tier
* Get the result of the specified task
* Get the log of the specified task
See `[16]`_ to get examples on how to use the API.
.. _`[1]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
.. _`[16]`: https://wiki.opnfv.org/display/functest/Running+test+cases+via+new+Functest+REST+API
.. _`[17]`: http://docs.opnfv.org/en/latest/testing/testing-dev.html
|
