diff options
17 files changed, 283 insertions, 820 deletions
diff --git a/docs/main.rst b/docs/main.rst new file mode 100644 index 000000000..228e0af98 --- /dev/null +++ b/docs/main.rst @@ -0,0 +1,145 @@ +===========================
+OPNFV functional test guide
+===========================
+
+The goal of this document consists in describing how to run functional tests on OPNFV solution and how to automate these tests.
+
+For release 1, several test cases have been selected:
+ * Rally Bench test suite
+ * Tempest tes suite
+ * OpenDaylight test suite
+ * vPing
+ * vIMS
+
+
+.. _prereqs:
+
+-------------
+Prerequisites
+-------------
+We assume that an OPNFV solution has been installed (System Under Test).
+For release 1, the tools needed for functional testing are not part of the installer and are not automatically installed.
+
+.. _pharos: https://wiki.opnfv.org/pharos
+
+It is recommended to install the tools on the jump host server as defined in the Pharos project.
+
+.. _installation:
+
+------------
+Installation
+------------
+
+Rally bench test suite
+======================
+
+Create or enter a folder where you want to check out the tool repos. Follow `Rally installation procedure`_.
+
+.. _`Rally installation procedure`: https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html
+
+
+Tempest
+=======
+
+It is possible to use Rally to perform Tempest tests. See `tempest installation guide using Rally`_.
+
+.. _`tempest installation guide using Rally`: https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler/
+
+OpenDaylight
+============
+
+vPing
+=====
+
+vIMS
+====
+
+
+.. _manualtest:
+
+--------------
+Manual testing
+--------------
+
+Rally
+=====
+Check your deployment::
+
+ # rally deployment check
+ keystone endpoints are valid and following service are available:
+ +-------------+-----------+------------+
+ | Services | Type | Status |
+ +-----------+-------------+------------+
+ | cinder | volume | Available |
+ | cinderv2 | volumev2 | Available |
+ | glance | image | Available |
+ | keystone | identity | Available |
+ | neutron | network | Available |
+ | nova | compute | Available |
+ | nova_ec2 | compute_ec2 | Available |
+ | novav3 | computev3 | Available |
+ +-----------+-------------+------------+
+
+Create a new opnfv scenario directory and run test suite::
+
+ # cd ~/rally/samples/tasks/scenario/
+ # mkdir opnfv
+ # wget http://git.opnfv.org/.. <TODO>
+ # rally task start --abort-on-sla-failure ./opnfv.json
+
+Tempest
+=======
+
+If we consider running Tempest suite with Rally::
+
+ # rally verify start
+ # rally verify list
+
+
+
+
+OpenDaylight
+============
+
+vPing
+=====
+
+vIMS
+====
+
+
+.. _automatictest:
+
+------------------
+Testing Automation
+------------------
+
+Connection of your platform
+===========================
+If you want to add your platform to the community automation, you need to declare your machine as a Jenkins slave.
+ * Send a mail to OPNFV LF Helpdesk (opnfv-helpdesk@rt.linuxfoundation.org)
+ * Create a local user jenkins on your machine
+ * wget http://mirrors.jenkins-ci.org/war/1.599/jenkins.war
+ * Extract contents, find the file named slave.jar and copy it to somewhere which jenkins user created in first step can access.
+ * Create a directory /home/jenkins/opnfv_slave_root
+ * check the java version (>1.7.0_75)
+ * Contact Linux Foundation to manage authentication of your server
+ * A key/token will be produced. Establish connection towards OPNFV Jenkins by using below command: java -jar slave.jar -jnlpUrl https://build.opnfv.org/ci/computer/<slave_name>/slave-agent.jnlp -secret <token>
+
+Continuous integration scripts
+==============================
+
+.. _references:
+
+----------
+References
+----------
+
+OPNFV main site: opnfvmain_.
+
+OPNFV functional test page: opnfvfunctest_.
+
+IRC support chan: #opnfv-testperf
+
+.. _opnfvmain: http://www.opnfv.org
+.. _opnfvfunctest: https://wiki.opnfv.org/opnfv_functional_testing
diff --git a/functest.rst b/functest.rst index ecb9768ae..d8606aaaf 100644 --- a/functest.rst +++ b/functest.rst @@ -46,6 +46,18 @@ It is possible to use Rally to perform Tempest tests. See `tempest installation OpenDaylight ============ +ODL wiki page describes system preparation and running tests. See `Integration Group CSIT`_. + +.. _`Integration Group CSIT`: https://wiki.opendaylight.org/view/CrossProject:Integration_Group:CSIT + +Summary: Set up python2.7 virtual environment:: + + mkvirtualenv robot + + pip install requests + pip install robotframework + pip install robotframework-sshlibrary + pip install robotframework-requests vPing ===== @@ -100,6 +112,10 @@ If we consider running Tempest suite with Rally:: OpenDaylight ============ +Tests can be executed with script *start_test.sh* from directory *functest/testcases/Controllers/ODL/CI*. For usage example see:: + + # bash start_test.sh -h + vPing ===== diff --git a/testcases/Controllers/ODL/CI/libraries/Common.py b/testcases/Controllers/ODL/CI/libraries/Common.py deleted file mode 100644 index e748caad8..000000000 --- a/testcases/Controllers/ODL/CI/libraries/Common.py +++ /dev/null @@ -1,81 +0,0 @@ -""" -Library for the robot based system test tool of the OpenDaylight project. -Authors: Baohua Yang@IBM, Denghui Huang@IBM -Updated: 2013-11-14 -""" -import collections -import xml.etree.ElementTree as ET - -''' -Common constants and functions for the robot framework. -''' - -def collection_should_contain(collection, *members): - """ - Fail if not every members is in the collection. - """ - if not isinstance(collection, collections.Iterable): - return False - for m in members: - if m not in collection: - return False - else: - return True - -def combine_strings(*strings): - """ - Combines the given `strings` together and returns the result. - The given strings are not altered by this keyword. - """ - result = '' - for s in strings: - if isinstance(s,str) or isinstance(s,unicode): - result += s - if result == '': - return None - else: - return result - - -def compare_xml(xml1, xml2): - """ - compare the two XML files to see if they contain the same data - but could be if different order. - It just split the xml in to lines and just check the line is in - the other file - """ - for line in xml1.rstrip().split('\n'): - if line not in xml2.rstrip().split('\n'): - return False - - for line in xml2.rstrip().split('\n'): - if line not in xml1.rstrip().split('\n'): - return False - - return True - -def num_of_nodes(depth, fanout): - '''returns num of switches of a mininet with tree topology - with particular depth and fanout parameters - ''' - result = 0 - for i in xrange(depth): - result += fanout**i - return result - -def num_of_links_for_node(nodeid, leaflist, fanout): - ''' - If the given node is a leaf node, there will be an only one link for it - and nodeid will be represented 2 times in topology - If the given node is not a leaf node, then there will be fanout+1 links - for it and nodeid will be represented (fanout+1)*2 times in topology - - p.s. root node is excluded. - ''' - if nodeid in leaflist: - return 1 - return (fanout+1) - -if __name__ == '__main__': - print num_of_nodes(3,4) - pass diff --git a/testcases/Controllers/ODL/CI/libraries/RequestsLibrary.py b/testcases/Controllers/ODL/CI/libraries/RequestsLibrary.py deleted file mode 100644 index 3ef4375d6..000000000 --- a/testcases/Controllers/ODL/CI/libraries/RequestsLibrary.py +++ /dev/null @@ -1,264 +0,0 @@ -import requests -import json - -from urllib import urlencode - - -import robot - -from robot.libraries.BuiltIn import BuiltIn - - -class RequestsLibrary(object): - ROBOT_LIBRARY_SCOPE = 'Global' - - def __init__(self): - self._cache = robot.utils.ConnectionCache('No sessions created') - self.builtin = BuiltIn() - - def _utf8_urlencode(self, data): - if not type(data) is dict: - return data - - utf8_data = {} - for k,v in data.iteritems(): - utf8_data[k] = unicode(v).encode('utf-8') - return urlencode(utf8_data) - - def create_session(self, alias, url, headers={}, cookies=None, - auth=None, timeout=None, proxies=None, - verify=False): - - """ Create Session: create a HTTP session to a server - - `url` Base url of the server - - `alias` Robot Framework alias to identify the session - - `headers` Dictionary of default headers - - `auth` Dictionary of username & password for HTTP Basic Auth - - `timeout` connection timeout - - `proxies` proxy server url - - `verify` set to True if Requests should verify the certificate - """ - - self.builtin.log('Creating session: %s' % alias, 'DEBUG') - auth = requests.auth.HTTPBasicAuth(*auth) if auth else None - s = session = requests.Session() - s.headers.update(headers) - s.auth = auth if auth else s.auth - s.proxies = proxies if proxies else s.proxies - - s.verify = self.builtin.convert_to_boolean(verify) - - # cant pass these into the Session anymore - self.timeout = timeout - self.cookies = cookies - self.verify = verify - - # cant use hooks :( - s.url = url - - self._cache.register(session, alias=alias) - return session - - def delete_all_sessions(self): - """ Removes all the session objects """ - - self._cache.empty_cache() - - def to_json(self, content): - """ Convert a string to a JSON object - - `content` String content to convert into JSON - """ - return json.loads(content) - - - def _get_url(self, session, uri): - ''' Helpere method to get the full url - ''' - url = session.url - if uri: - slash = '' if uri.startswith('/') else '/' - url = "%s%s%s" %(session.url, slash, uri) - return url - - def get(self, alias, uri, headers=None): - """ Send a GET request on the session object found using the - given `alias` - - `alias` that will be used to identify the Session object in the cache - - `uri` to send the GET request to - - `headers` a dictionary of headers to use with the request - """ - - session = self._cache.switch(alias) - resp = session.get(self._get_url(session, uri), - headers=headers, - cookies=self.cookies, timeout=self.timeout) - - # store the last response object - session.last_resp = resp - return resp - - def post(self, alias, uri, data={}, headers=None, files={}): - """ Send a POST request on the session object found using the - given `alias` - - `alias` that will be used to identify the Session object in the cache - - `uri` to send the GET request to - - `data` a dictionary of key-value pairs that will be urlencoded - and sent as POST data - or binary data that is sent as the raw body content - - `headers` a dictionary of headers to use with the request - - `files` a dictionary of file names containing file data to POST to the server - """ - - session = self._cache.switch(alias) - data = self._utf8_urlencode(data) - - resp = session.post(self._get_url(session, uri), - data=data, headers=headers, - files=files, - cookies=self.cookies, timeout=self.timeout) - - # store the last response object - session.last_resp = resp - self.builtin.log("Post response: " + resp.content, 'DEBUG') - return resp - - def postjson(self, alias, uri, data={}, headers=None, files={}): - """ Send a POST request on the session object found using the - given `alias` - - `alias` that will be used to identify the Session object in the cache - - `uri` to send the GET request to - - `data` a dictionary of key-value pairs that will be urlencoded - and sent as POST data - or binary data that is sent as the raw body content - - `headers` a dictionary of headers to use with the request - - `files` a dictionary of file names containing file data to POST to the server - """ - - session = self._cache.switch(alias) - data = json.dumps(data) - - resp = session.post(self._get_url(session, uri), - data=data, headers=headers, - files=files, - cookies=self.cookies, timeout=self.timeout) - - # store the last response object - session.last_resp = resp - self.builtin.log("Post response: " + resp.content, 'DEBUG') - return resp - - def put(self, alias, uri, data=None, headers=None): - """ Send a PUT request on the session object found using the - given `alias` - - `alias` that will be used to identify the Session object in the cache - - `uri` to send the PUT request to - - `headers` a dictionary of headers to use with the request - - """ - - session = self._cache.switch(alias) - #data = self._utf8_urlencode(data) - data = json.dumps(data) - - resp = session.put(self._get_url(session, uri), - data=data, headers=headers, - cookies=self.cookies, timeout=self.timeout) - - self.builtin.log("PUT response: %s DEBUG" % resp.content) - - # store the last response object - session.last_resp = resp - return resp - - def put_xml(self, alias, uri, data=None, headers=None): - """ Send a PUT_xml request on the session object found using the - given `alias` - - `alias` that will be used to identify the Session object in the cache - - `uri` to send the PUT_xml request to - - `headers` a dictionary of headers to use with the request - - """ - - session = self._cache.switch(alias) - data = self._utf8_urlencode(data) - #data = json.dumps(data) - - resp = session.put(self._get_url(session, uri), - data=data, headers=headers, - cookies=self.cookies, timeout=self.timeout) - - self.builtin.log("PUT response: %s DEBUG" % resp.content) - - # store the last response object - session.last_resp = resp - return resp - - def delete(self, alias, uri, data=(), headers=None): - """ Send a DELETE request on the session object found using the - given `alias` - - `alias` that will be used to identify the Session object in the cache - - `uri` to send the DELETE request to - - `headers` a dictionary of headers to use with the request - - """ - - session = self._cache.switch(alias) - args = "?%s" % urlencode(data) if data else '' - resp = session.delete("%s%s" % (self._get_url(session, uri), args), - headers=headers, cookies=self.cookies, - timeout=self.timeout) - - # store the last response object - session.last_resp = resp - return resp - - - def head(self, alias, uri, headers=None): - """ Send a HEAD request on the session object found using the - given `alias` - - `alias` that will be used to identify the Session object in the cache - - `uri` to send the HEAD request to - - `headers` a dictionary of headers to use with the request - - """ - - session = self._cache.switch(alias) - resp = session.head(self._get_url(session, uri), headers=headers, - cookies=self.cookies, timeout=self.timeout) - - # store the last response object - session.last_resp = resp - return resp diff --git a/testcases/Controllers/ODL/CI/libraries/Utils.txt b/testcases/Controllers/ODL/CI/libraries/Utils.txt deleted file mode 100644 index 913ba22c0..000000000 --- a/testcases/Controllers/ODL/CI/libraries/Utils.txt +++ /dev/null @@ -1,106 +0,0 @@ -*** Settings *** -Library SSHLibrary -Library ./UtilLibrary.py - -*** Variables *** -${start} sudo mn --controller=remote,ip=${CONTROLLER} --topo tree,1 --switch ovsk,protocols=OpenFlow13 -${linux_prompt} > - -*** Keywords *** -Start Suite - [Documentation] Basic setup/cleanup work that can be done safely before any system - ... is run. - Log Start the test on the base edition - ${mininet_conn_id}= Open Connection ${MININET} prompt=${linux_prompt} timeout=30s - Set Suite Variable ${mininet_conn_id} - Login With Public Key ${MININET_USER} ${USER_HOME}/.ssh/id_rsa any - Write sudo ovs-vsctl set-manager ptcp:6644 - Read Until ${linux_prompt} - Write sudo mn -c - Read Until ${linux_prompt} - Write ${start} - Read Until mininet> - Sleep 6 - -Stop Suite - [Documentation] Cleanup/Shutdown work that should be done at the completion of all - ... tests - Log Stop the test on the base edition - Switch Connection ${mininet_conn_id} - Read - Write exit - Read Until ${linux_prompt} - Close Connection - -Ensure All Nodes Are In Response - [Arguments] ${URI} ${node_list} - [Documentation] A GET is made to the supplied ${URI} and every item in the ${node_list} - ... is verified to exist in the repsonse. This keyword currently implies that it's node - ... specific but any list of strings can be given in ${node_list}. Refactoring of this - ... to make it more generic should be done. (see keyword "Check For Elements At URI") - : FOR ${node} IN @{node_list} - \ ${resp} RequestsLibrary.Get session ${URI} - \ Should Be Equal As Strings ${resp.status_code} 200 - \ Should Contain ${resp.content} ${node} - -Check Nodes Stats - [Arguments] ${node} - [Documentation] A GET on the /node/${node} API is made and specific flow stat - ... strings are checked for existence. - ${resp} RequestsLibrary.Get session ${REST_CONTEXT}/node/${node} - Should Be Equal As Strings ${resp.status_code} 200 - Should Contain ${resp.content} flow-capable-node-connector-statistics - Should Contain ${resp.content} flow-table-statistics - -Check That Port Count Is Ok - [Arguments] ${node} ${count} - [Documentation] A GET on the /port API is made and the specified port ${count} is - ... verified. A more generic Keyword "Check For Specific Number Of Elements At URI" - ... also does this work and further consolidation should be done. - ${resp} RequestsLibrary.Get session ${REST_CONTEXT}/${CONTAINER}/port - Log ${resp.content} - Should Be Equal As Strings ${resp.status_code} 200 - Should Contain X Times ${resp.content} ${node} ${count} - -Check For Specific Number Of Elements At URI - [Arguments] ${uri} ${element} ${expected_count} - [Documentation] A GET is made to the specified ${URI} and the specific count of a - ... given element is done (as supplied by ${element} and ${expected_count}) - ${resp} RequestsLibrary.Get session ${uri} - Log ${resp.content} - Should Be Equal As Strings ${resp.status_code} 200 - Should Contain X Times ${resp.content} ${element} ${expected_count} - -Check For Elements At URI - [Arguments] ${uri} ${elements} - [Documentation] A GET is made at the supplied ${URI} and every item in the list of - ... ${elements} is verified to exist in the response - ${resp} RequestsLibrary.Get session ${uri} - Log ${resp.content} - Should Be Equal As Strings ${resp.status_code} 200 - : FOR ${i} IN @{elements} - \ Should Contain ${resp.content} ${i} - -Check For Elements Not At URI - [Arguments] ${uri} ${elements} - [Documentation] A GET is made at the supplied ${URI} and every item in the list of - ... ${elements} is verified to NOT exist in the response - ${resp} RequestsLibrary.Get session ${uri} - Log ${resp.content} - Should Be Equal As Strings ${resp.status_code} 200 - : FOR ${i} IN @{elements} - \ Should Not Contain ${resp.content} ${i} - -Extract Value From Content - [Arguments] ${content} ${index} ${strip}=nostrip - [Documentation] Will take the given response content and return the value at the given index as a string - ${value}= Get Json Value ${content} ${index} - ${value}= Convert To String ${value} - ${value}= Run Keyword If '${strip}' == 'strip' Strip Quotes ${value} - [Return] ${value} - -Strip Quotes - [Arguments] ${string_to_strip} - [Documentation] Will strip ALL quotes from given string and return the new string - ${string_to_return}= Replace String ${string_to_strip} " \ count=-1 - [Return] ${string_to_return} diff --git a/testcases/Controllers/ODL/CI/start_tests.sh b/testcases/Controllers/ODL/CI/start_tests.sh new file mode 100644 index 000000000..e2f94a65a --- /dev/null +++ b/testcases/Controllers/ODL/CI/start_tests.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# Script requires that test environment is created already +# it includes python2.7 virtual env with robot packages and git + +# Colors +green='\033[0;32m' +light_green='\033[1;32m' +nc='\033[0m' # No Color + +usage="Script for starting ODL tests. Tests to be executed are specified in test_list.txt file. + +usage: +[var=value] bash $(basename "$0") [-h] + +where: + -h show this help text + var one of the following: OSTACK_IP, ODL_PORT, USER, PASS, PATH_TO_VENV + value new value for var + +example: + OSTACK_IP=oscontro1 ODL_PORT=8080 bash $(basename "$0")" + +while getopts ':h' option; do + case "$option" in + h) echo "$usage" + exit + ;; + \?) printf "illegal option: -%s\n" "$OPTARG" >&2 + echo "$usage" >&2 + exit 1 + ;; + esac +done + +echo -e "${green}Current environment parameters for ODL suite.${nc}" +# Following vars might be also specified as CLI params +set -x +PATH_TO_VENV=${PATH_TO_VENV:-~/.virtualenvs/robot/bin/activate} +OSTACK_IP=${OSTACK_IP:-'oscontrol'} +ODL_PORT=${ODL_PORT:-8081} +USR_NAME=${USR_NAME:-'admin'} +PASS=${PASS:-'octopus'} +set +x + +echo -e "${green}Cloning ODL integration git repo.${nc}" +if [ -d integration ]; then + cd integration + git checkout -- . + git pull + cd - +else + git clone https://github.com/opendaylight/integration.git +fi + +# Change openstack password for admin tenant in neutron suite +sed -i "s/\"password\": \"admin\"/\"password\": \"${PASS}\"/" integration/test/csit/suites/openstack/neutron/__init__.robot + +echo -e "${green}Activate python virtual env.${nc}" +source $PATH_TO_VENV + +# List of tests are specified in test_list.txt +# those are relative paths to test directories from integartion suite +echo -e "${green}Executing chosen tests.${nc}" +while read line +do + # skip comments + [[ ${line:0:1} == "#" ]] && continue + # skip empty lines + [[ -z "${line}" ]] && continue + + echo -e "${light_green}Starting test: $line ${nc}" + pybot -v OPENSTACK:${OSTACK_IP} -v PORT:${ODL_PORT} -v CONTROLLER:${OSTACK_IP} $line +done < test_list.txt + +echo -e "${green}Deactivate venv.${nc}" +deactivate + +# Now we can copy output.xml, log.html and report.xml files generated by robot. diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/010__networks.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/010__networks.txt deleted file mode 100644 index ac5080a2e..000000000 --- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/010__networks.txt +++ /dev/null @@ -1,58 +0,0 @@ -*** Settings *** -Documentation Checking Network created in OpenStack are pushed to OpenDaylight -Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH} -Suite Teardown Delete All Sessions -Library SSHLibrary -Library Collections -Library OperatingSystem -Library ../../../libraries/RequestsLibrary.py -Library ../../../libraries/Common.py -Variables ../../../variables/Variables.py - -*** Variables *** -${ODLREST} /controller/nb/v2/neutron/networks -${OSREST} /v2.0/networks -${postNet} {"network":{"name":"odl_network","admin_state_up":true}} - -*** Test Cases *** -Check OpenStack Networks - [Documentation] Checking OpenStack Neutron for known networks - [Tags] Network Neutron OpenStack - Log ${X-AUTH} - ${resp} get OSSession ${OSREST} - Should be Equal As Strings ${resp.status_code} 200 - ${OSResult} To Json ${resp.content} - Set Suite Variable ${OSResult} - Log ${OSResult} - -Check OpenDaylight Networks - [Documentation] Checking OpenDaylight Neutron API for Known Networks - [Tags] Network Neutron OpenDaylight - Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH} - ${resp} get ODLSession ${ODLREST} - Should be Equal As Strings ${resp.status_code} 200 - ${ODLResult} To Json ${resp.content} - Set Suite Variable ${ODLResult} - Log ${ODLResult} - -Create Network - [Documentation] Create new network in OpenStack - [Tags] Create Network OpenStack Neutron - Log ${postNet} - ${resp} post OSSession ${OSREST} data=${postNet} - Should be Equal As Strings ${resp.status_code} 201 - ${result} To JSON ${resp.content} - ${result} Get From Dictionary ${result} network - ${NETID} Get From Dictionary ${result} id - Log ${result} - Log ${NETID} - Set Global Variable ${NETID} - sleep 2 - -Check Network - [Documentation] Check Network created in OpenDaylight - [Tags] Check Network OpenDaylight - ${resp} get ODLSession ${ODLREST}/${NetID} - Should be Equal As Strings ${resp.status_code} 200 - - diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/020__subnets.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/020__subnets.txt deleted file mode 100644 index 5e0c417d8..000000000 --- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/020__subnets.txt +++ /dev/null @@ -1,58 +0,0 @@ -*** Settings *** -Documentation Checking Subnets created in OpenStack are pushed to OpenDaylight -Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH} -Suite Teardown Delete All Sessions -Library SSHLibrary -Library Collections -Library OperatingSystem -Library ../../../libraries/RequestsLibrary.py -Library ../../../libraries/Common.py -Variables ../../../variables/Variables.py - -*** Variables *** -${ODLREST} /controller/nb/v2/neutron/subnets -${OSREST} /v2.0/subnets -${data} {"subnet":{"name":"odl_subnet","network_id":"${NETID}","ip_version":4,"cidr":"172.16.64.0/24","allocation_pools":[{"start":"172.16.64.20","end":"172.16.64.120"}]}} - -*** Test Cases *** -Check OpenStack Subnets - [Documentation] Checking OpenStack Neutron for known Subnets - [Tags] Subnets Neutron OpenStack - Log ${X-AUTH} - ${resp} get OSSession ${OSREST} - Should be Equal As Strings ${resp.status_code} 200 - ${OSResult} To Json ${resp.content} - Set Suite Variable ${OSResult} - Log ${OSResult} - -Check OpenDaylight subnets - [Documentation] Checking OpenDaylight Neutron API for Known Subnets - [Tags] Subnets Neutron OpenDaylight - Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH} - ${resp} get ODLSession ${ODLREST} - Should be Equal As Strings ${resp.status_code} 200 - ${ODLResult} To Json ${resp.content} - Set Suite Variable ${ODLResult} - Log ${ODLResult} - -Create New subnet - [Documentation] Create new subnet in OpenStack - [Tags] Create Subnet OpenStack Neutron - Log ${data} - ${resp} post OSSession ${OSREST} data=${data} - Should be Equal As Strings ${resp.status_code} 201 - ${result} To JSON ${resp.content} - ${result} Get From Dictionary ${result} subnet - ${SUBNETID} Get From Dictionary ${result} id - Log ${result} - Log ${SUBNETID} - Set Global Variable ${SUBNETID} - sleep 2 - -Check New subnet - [Documentation] Check new subnet created in OpenDaylight - [Tags] Check subnet OpenDaylight - ${resp} get ODLSession ${ODLREST}/${SUBNETID} - Should be Equal As Strings ${resp.status_code} 200 - - diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/030__ports.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/030__ports.txt deleted file mode 100644 index 5aafd51d6..000000000 --- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/030__ports.txt +++ /dev/null @@ -1,58 +0,0 @@ -*** Settings *** -Documentation Checking Port created in OpenStack are pushed to OpenDaylight -Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH} -Suite Teardown Delete All Sessions -Library SSHLibrary -Library Collections -Library OperatingSystem -Library ../../../libraries/RequestsLibrary.py -Library ../../../libraries/Common.py -Variables ../../../variables/Variables.py - -*** Variables *** -${ODLREST} /controller/nb/v2/neutron/ports -${OSREST} /v2.0/ports -${data} {"port":{"name":"odl_port","network_id":"${NETID}","admin_state_up": true}} - -*** Test Cases *** -Check OpenStack ports - [Documentation] Checking OpenStack Neutron for known ports - [Tags] Ports Neutron OpenStack - Log ${X-AUTH} - ${resp} get OSSession ${OSREST} - Should be Equal As Strings ${resp.status_code} 200 - ${OSResult} To Json ${resp.content} - Set Suite Variable ${OSResult} - Log ${OSResult} - -Check OpenDaylight ports - [Documentation] Checking OpenDaylight Neutron API for Known Ports - [Tags] Ports Neutron OpenDaylight - Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH} - ${resp} get ODLSession ${ODLREST} - Should be Equal As Strings ${resp.status_code} 200 - ${ODLResult} To Json ${resp.content} - Set Suite Variable ${ODLResult} - Log ${ODLResult} - -Create New Port - [Documentation] Create new port in OpenStack - [Tags] Create port OpenStack Neutron - Log ${data} - ${resp} post OSSession ${OSREST} data=${data} - Should be Equal As Strings ${resp.status_code} 201 - ${result} To JSON ${resp.content} - ${result} Get From Dictionary ${result} port - ${PORTID} Get From Dictionary ${result} id - Log ${result} - Log ${PORTID} - Set Global Variable ${PORTID} - sleep 2 - -Check New Port - [Documentation] Check new port created in OpenDaylight - [Tags] Check subnet OpenDaylight - ${resp} get ODLSession ${ODLREST}/${PORTID} - Should be Equal As Strings ${resp.status_code} 200 - - diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/040__delete_ports.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/040__delete_ports.txt deleted file mode 100644 index 5aca2c869..000000000 --- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/040__delete_ports.txt +++ /dev/null @@ -1,37 +0,0 @@ -*** Settings *** -Documentation Checking Port deleted in OpenStack are deleted also in OpenDaylight -Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH} -Suite Teardown Delete All Sessions -Library SSHLibrary -Library Collections -Library OperatingSystem -Library ../../../libraries/RequestsLibrary.py -Library ../../../libraries/Common.py -Variables ../../../variables/Variables.py - -*** Variables *** -${ODLREST} /controller/nb/v2/neutron/ports -${OSREST} /v2.0/ports/${PORTID} -${data} {"port":{"network_id":"${NETID}","admin_state_up": true}} - -*** Test Cases *** -Delete New Port - [Documentation] Delete previously created port in OpenStack - [Tags] Delete port OpenStack Neutron - Log ${data} - ${resp} delete OSSession ${OSREST} - Should be Equal As Strings ${resp.status_code} 204 - Log ${resp.content} - sleep 2 - -Check Port Deleted - [Documentation] Check port deleted in OpenDaylight - [Tags] Check port deleted OpenDaylight - Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH} - ${resp} get ODLSession ${ODLREST} - Should be Equal As Strings ${resp.status_code} 200 - ${ODLResult} To Json ${resp.content} - Set Suite Variable ${ODLResult} - Log ${ODLResult} - ${resp} get ODLSession ${ODLREST}/${PORTID} - Should be Equal As Strings ${resp.status_code} 404 diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/050__delete_subnets.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/050__delete_subnets.txt deleted file mode 100644 index 04c73cc2e..000000000 --- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/050__delete_subnets.txt +++ /dev/null @@ -1,37 +0,0 @@ -*** Settings *** -Documentation Checking Subnets deleted in OpenStack are deleted also in OpenDaylight -Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH} -Suite Teardown Delete All Sessions -Library SSHLibrary -Library Collections -Library OperatingSystem -Library ../../../libraries/RequestsLibrary.py -Library ../../../libraries/Common.py -Variables ../../../variables/Variables.py - -*** Variables *** -${ODLREST} /controller/nb/v2/neutron/subnets -${OSREST} /v2.0/subnets/${SUBNETID} -${data} {"subnet":{"network_id":"${NETID}","ip_version":4,"cidr":"172.16.64.0/24","allocation_pools":[{"start":"172.16.64.20","end":"172.16.64.120"}]}} - -*** Test Cases *** -Delete New subnet - [Documentation] Delete previously created subnet in OpenStack - [Tags] Delete Subnet OpenStack Neutron - Log ${data} - ${resp} delete OSSession ${OSREST} - Should be Equal As Strings ${resp.status_code} 204 - Log ${resp.content} - sleep 2 - -Check New subnet deleted - [Documentation] Check subnet deleted in OpenDaylight - [Tags] Check subnet deleted OpenDaylight - Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH} - ${resp} get ODLSession ${ODLREST} - Should be Equal As Strings ${resp.status_code} 200 - ${ODLResult} To Json ${resp.content} - Set Suite Variable ${ODLResult} - Log ${ODLResult} - ${resp} get ODLSession ${ODLREST}/${SUBNETID} - Should be Equal As Strings ${resp.status_code} 404 diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/060__delete_networks.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/060__delete_networks.txt deleted file mode 100644 index 31963ec97..000000000 --- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/060__delete_networks.txt +++ /dev/null @@ -1,37 +0,0 @@ -*** Settings *** -Documentation Checking Network deleted in OpenStack are deleted also in OpenDaylight -Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH} -Suite Teardown Delete All Sessions -Library SSHLibrary -Library Collections -Library OperatingSystem -Library ../../../libraries/RequestsLibrary.py -Library ../../../libraries/Common.py -Variables ../../../variables/Variables.py - -*** Variables *** -${ODLREST} /controller/nb/v2/neutron/networks -${OSREST} /v2.0/networks/${NETID} -${postNet} {"network":{"name":"odl_network","admin_state_up":true}} - -*** Test Cases *** -Delete Network - [Documentation] Delete network in OpenStack - [Tags] Delete Network OpenStack Neutron - Log ${postNet} - ${resp} delete OSSession ${OSREST} - Should be Equal As Strings ${resp.status_code} 204 - Log ${resp.content} - sleep 2 - -Check Network deleted - [Documentation] Check Network deleted in OpenDaylight - [Tags] Check Network OpenDaylight - Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH} - ${resp} get ODLSession ${ODLREST} - Should be Equal As Strings ${resp.status_code} 200 - ${ODLResult} To Json ${resp.content} - Set Suite Variable ${ODLResult} - Log ${ODLResult} - ${resp} get ODLSession ${ODLREST}/${NetID} - Should be Equal As Strings ${resp.status_code} 404 diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/__init__.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/__init__.txt deleted file mode 100644 index 4112b32ce..000000000 --- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/__init__.txt +++ /dev/null @@ -1,27 +0,0 @@ -*** Settings *** -Documentation Test suite for Neutron Plugin -Suite Setup Start Suite -Suite Teardown Stop Suite -Library SSHLibrary -Library Collections -Library ../../../libraries/RequestsLibrary.py -Library ../../../libraries/Common.py -Variables ../../../variables/Variables.py - -*** Variables *** -${UserInfo}= {"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "octopus"}}} - -** Keywords *** -Start Suite - Create Session KeyStoneSession http://${OPENSTACK}:5000 headers=${HEADERS} - ${resp} post KeyStoneSession /v2.0/tokens ${UserInfo} - Should Be Equal As Strings ${resp.status_code} 200 - ${result} To JSON ${resp.content} - ${result} Get From Dictionary ${result} access - ${result} Get From Dictionary ${result} token - ${TOKEN} Get From Dictionary ${result} id - ${X-AUTH} Create Dictionary X-Auth-Token ${TOKEN} Content-Type application/json - Set Global Variable ${X-AUTH} -Stop Suite - Delete All Sessions - diff --git a/testcases/Controllers/ODL/CI/test_list.txt b/testcases/Controllers/ODL/CI/test_list.txt new file mode 100644 index 000000000..e5e52129b --- /dev/null +++ b/testcases/Controllers/ODL/CI/test_list.txt @@ -0,0 +1,5 @@ +# List of tests` which will be executed by script start_test.sh +# You can specify path to specific robot test file or directory (in that case all tests from directory will be executed) + +integration/test/csit/suites/integration/basic/ +integration/test/csit/suites/openstack/neutron/ diff --git a/testcases/Controllers/ODL/CI/variables/Variables.py b/testcases/Controllers/ODL/CI/variables/Variables.py deleted file mode 100644 index f406eec3f..000000000 --- a/testcases/Controllers/ODL/CI/variables/Variables.py +++ /dev/null @@ -1,20 +0,0 @@ -""" -Library for the robot based system test tool of the OpenDaylight project. -""" -import collections - -# Global variables -CONTROLLER = '10.2.91.18' -PORT = '8081' -PREFIX = 'http://' + CONTROLLER + ':' + PORT -USER = 'admin' -PWD = 'admin' -AUTH = [u'admin',u'admin'] -HEADERS={'Content-Type': 'application/json'} -HEADERS_XML={'Content-Type': 'application/xml'} -ACCEPT_XML={'Accept': 'application/xml'} - -#TOKEN -AUTH_TOKEN_API='/oauth2/token' -REVOKE_TOKEN_API='/oauth2/revoke' - diff --git a/testcases/Controllers/ODL/ODL.md b/testcases/Controllers/ODL/ODL.md index ceb04195e..375f20e6c 100644 --- a/testcases/Controllers/ODL/ODL.md +++ b/testcases/Controllers/ODL/ODL.md @@ -23,11 +23,6 @@ vcrpy==1.0.2 wsgiref==0.1.2 ## Running tests - -Parameters are specified in: - -ODL/variables/Variables.py - -but can be also overidden in command line e.g.: - -pybot -v OPENSTACK:10.2.91.18 -v PORT:8081 -v CONTROLLER 10.2.91.18 . +For more info: +cd CI +bash start_test.sh -h diff --git a/testcases/VIM/OpenStack/OpenStack.md b/testcases/VIM/OpenStack/OpenStack.md index a57699312..e1a85c305 100644 --- a/testcases/VIM/OpenStack/OpenStack.md +++ b/testcases/VIM/OpenStack/OpenStack.md @@ -4,27 +4,38 @@ Original Rally testsuites can be found here: https://github.com/stackforge/rally --- ## Intro -In order to perform functional and performance testing, we use a dedicated VM. - - TODO: add test-tool VM in architecture and puppetise test-tool VM - -On this VM, we installed: -* Rally: https://wiki.openstack.org/wiki/Rally - - TODO RobotFramework: http://robotframework.org, SIPP http://sipp.sourceforge.net/ - - +In order to perform functional and performance testing, we use Rally (see https://wiki.openstack.org/wiki/Rally for details). +Rally must be installed as jenkins user on the jumphost machine of the OPNFV solution. ## Installation & Configuration ### Rally -* Log on the test-tool VM -* install Rally (https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html) - -openrc file (info from OpenStack) is needed, password is required during configuration procedure +* Log on jumphost machine as jenkins user +* Create the file existing.json, adapt it to your OpenStack (until agreement on default passwords) +```bash +{ + "type": "ExistingCloud", + "auth_url": "http://example.net:5000/v2.0/", + "region_name": "RegionOne", + "endpoint_type": "public", + "admin": { + "username": "admin", + "password": "myadminpass", + "tenant_name": "demo" + }, + "https_insecure": False, + "https_cacert": "", +} +``` +* Install Rally (ref https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html) -* check +```bash +git clone https://git.openstack.org/stackforge/rally +./rally/install_rally.sh -v +rally deployment create --file=existing.json --name=existing +``` +* you can check the available OpenStack services ```bash # rally deployment check keystone endpoints are valid and following service are available: @@ -40,10 +51,8 @@ keystone endpoints are valid and following service are available: | nova_ec2 | compute_ec2 | Available | | novav3 | computev3 | Available | +-----------+-------------+------------+ - ``` - -* For Rally scenario, follow https://rally.readthedocs.org/en/latest/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html +* You can start Rally scenario manually, follow https://rally.readthedocs.org/en/latest/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html ```bash # rally task start ./samples/tasks/scenarios/nova/my-boot-and-delete.json -------------------------------------------------------------------------------- @@ -118,14 +127,8 @@ HINTS: Using task: f42c8aed-00a6-4715-9951-945b4fb97c32 ``` -* For Tempest, follow the instructions https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler -* In first step Rally scenario were fine but Tempest scenarios failed due to configuration -Apply patch https://review.openstack.org/#/c/163330/ -```bash -pip uninstall rally && cd ./rally && python setup.py install -``` +* For Tempest, you can run the test manually by following the instructions https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler -You shall be able to run Rally/Tempest towards your OpenStack ```bash root@rally:~/rally# rally verify start [...] @@ -188,10 +191,6 @@ Tests: Rally includes a reporting tool https://rally.readthedocs.org/en/latest/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html - - - - ## Test description ### Rally @@ -207,5 +206,13 @@ ceilometer designate glance keystone neutron quotas requests tempest- tempest tests can be retrieved at https://github.com/openstack/tempest +tests have been grouped and are available in https://git.opnfv.org/cgit/functest/tree/testcases/VIM/OpenStack/CI/suites + ## Automation + +For automation, 2 job-templates have been created in https://git.opnfv.org/cgit/releng/tree/jjb/functest/functest.yml + +* functest-vim_bench-test: this template runs automatically a python script that runs the different rally scenario (except Tempest) +* functest-vim_tempest-test: this template runs the rally command rally verify start + |