aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--docs/main.rst145
-rw-r--r--functest.rst16
-rw-r--r--testcases/Controllers/ODL/CI/libraries/Common.py81
-rw-r--r--testcases/Controllers/ODL/CI/libraries/RequestsLibrary.py264
-rw-r--r--testcases/Controllers/ODL/CI/libraries/Utils.txt106
-rw-r--r--testcases/Controllers/ODL/CI/start_tests.sh78
-rw-r--r--testcases/Controllers/ODL/CI/suites/openstack/neutron/010__networks.txt58
-rw-r--r--testcases/Controllers/ODL/CI/suites/openstack/neutron/020__subnets.txt58
-rw-r--r--testcases/Controllers/ODL/CI/suites/openstack/neutron/030__ports.txt58
-rw-r--r--testcases/Controllers/ODL/CI/suites/openstack/neutron/040__delete_ports.txt37
-rw-r--r--testcases/Controllers/ODL/CI/suites/openstack/neutron/050__delete_subnets.txt37
-rw-r--r--testcases/Controllers/ODL/CI/suites/openstack/neutron/060__delete_networks.txt37
-rw-r--r--testcases/Controllers/ODL/CI/suites/openstack/neutron/__init__.txt27
-rw-r--r--testcases/Controllers/ODL/CI/test_list.txt5
-rw-r--r--testcases/Controllers/ODL/CI/variables/Variables.py20
-rw-r--r--testcases/Controllers/ODL/ODL.md11
-rw-r--r--testcases/VIM/OpenStack/OpenStack.md65
17 files changed, 283 insertions, 820 deletions
diff --git a/docs/main.rst b/docs/main.rst
new file mode 100644
index 000000000..228e0af98
--- /dev/null
+++ b/docs/main.rst
@@ -0,0 +1,145 @@
+===========================
+OPNFV functional test guide
+===========================
+
+The goal of this document consists in describing how to run functional tests on OPNFV solution and how to automate these tests.
+
+For release 1, several test cases have been selected:
+ * Rally Bench test suite
+ * Tempest tes suite
+ * OpenDaylight test suite
+ * vPing
+ * vIMS
+
+
+.. _prereqs:
+
+-------------
+Prerequisites
+-------------
+We assume that an OPNFV solution has been installed (System Under Test).
+For release 1, the tools needed for functional testing are not part of the installer and are not automatically installed.
+
+.. _pharos: https://wiki.opnfv.org/pharos
+
+It is recommended to install the tools on the jump host server as defined in the Pharos project.
+
+.. _installation:
+
+------------
+Installation
+------------
+
+Rally bench test suite
+======================
+
+Create or enter a folder where you want to check out the tool repos. Follow `Rally installation procedure`_.
+
+.. _`Rally installation procedure`: https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html
+
+
+Tempest
+=======
+
+It is possible to use Rally to perform Tempest tests. See `tempest installation guide using Rally`_.
+
+.. _`tempest installation guide using Rally`: https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler/
+
+OpenDaylight
+============
+
+vPing
+=====
+
+vIMS
+====
+
+
+.. _manualtest:
+
+--------------
+Manual testing
+--------------
+
+Rally
+=====
+Check your deployment::
+
+ # rally deployment check
+ keystone endpoints are valid and following service are available:
+ +-------------+-----------+------------+
+ | Services | Type | Status |
+ +-----------+-------------+------------+
+ | cinder | volume | Available |
+ | cinderv2 | volumev2 | Available |
+ | glance | image | Available |
+ | keystone | identity | Available |
+ | neutron | network | Available |
+ | nova | compute | Available |
+ | nova_ec2 | compute_ec2 | Available |
+ | novav3 | computev3 | Available |
+ +-----------+-------------+------------+
+
+Create a new opnfv scenario directory and run test suite::
+
+ # cd ~/rally/samples/tasks/scenario/
+ # mkdir opnfv
+ # wget http://git.opnfv.org/.. <TODO>
+ # rally task start --abort-on-sla-failure ./opnfv.json
+
+Tempest
+=======
+
+If we consider running Tempest suite with Rally::
+
+ # rally verify start
+ # rally verify list
+
+
+
+
+OpenDaylight
+============
+
+vPing
+=====
+
+vIMS
+====
+
+
+.. _automatictest:
+
+------------------
+Testing Automation
+------------------
+
+Connection of your platform
+===========================
+If you want to add your platform to the community automation, you need to declare your machine as a Jenkins slave.
+ * Send a mail to OPNFV LF Helpdesk (opnfv-helpdesk@rt.linuxfoundation.org)
+ * Create a local user jenkins on your machine
+ * wget http://mirrors.jenkins-ci.org/war/1.599/jenkins.war
+ * Extract contents, find the file named slave.jar and copy it to somewhere which jenkins user created in first step can access.
+ * Create a directory /home/jenkins/opnfv_slave_root
+ * check the java version (>1.7.0_75)
+ * Contact Linux Foundation to manage authentication of your server
+ * A key/token will be produced. Establish connection towards OPNFV Jenkins by using below command: java -jar slave.jar -jnlpUrl https://build.opnfv.org/ci/computer/<slave_name>/slave-agent.jnlp -secret <token>
+
+Continuous integration scripts
+==============================
+
+.. _references:
+
+----------
+References
+----------
+
+OPNFV main site: opnfvmain_.
+
+OPNFV functional test page: opnfvfunctest_.
+
+IRC support chan: #opnfv-testperf
+
+.. _opnfvmain: http://www.opnfv.org
+.. _opnfvfunctest: https://wiki.opnfv.org/opnfv_functional_testing
diff --git a/functest.rst b/functest.rst
index ecb9768ae..d8606aaaf 100644
--- a/functest.rst
+++ b/functest.rst
@@ -46,6 +46,18 @@ It is possible to use Rally to perform Tempest tests. See `tempest installation
OpenDaylight
============
+ODL wiki page describes system preparation and running tests. See `Integration Group CSIT`_.
+
+.. _`Integration Group CSIT`: https://wiki.opendaylight.org/view/CrossProject:Integration_Group:CSIT
+
+Summary: Set up python2.7 virtual environment::
+
+ mkvirtualenv robot
+
+ pip install requests
+ pip install robotframework
+ pip install robotframework-sshlibrary
+ pip install robotframework-requests
vPing
=====
@@ -100,6 +112,10 @@ If we consider running Tempest suite with Rally::
OpenDaylight
============
+Tests can be executed with script *start_test.sh* from directory *functest/testcases/Controllers/ODL/CI*. For usage example see::
+
+ # bash start_test.sh -h
+
vPing
=====
diff --git a/testcases/Controllers/ODL/CI/libraries/Common.py b/testcases/Controllers/ODL/CI/libraries/Common.py
deleted file mode 100644
index e748caad8..000000000
--- a/testcases/Controllers/ODL/CI/libraries/Common.py
+++ /dev/null
@@ -1,81 +0,0 @@
-"""
-Library for the robot based system test tool of the OpenDaylight project.
-Authors: Baohua Yang@IBM, Denghui Huang@IBM
-Updated: 2013-11-14
-"""
-import collections
-import xml.etree.ElementTree as ET
-
-'''
-Common constants and functions for the robot framework.
-'''
-
-def collection_should_contain(collection, *members):
- """
- Fail if not every members is in the collection.
- """
- if not isinstance(collection, collections.Iterable):
- return False
- for m in members:
- if m not in collection:
- return False
- else:
- return True
-
-def combine_strings(*strings):
- """
- Combines the given `strings` together and returns the result.
- The given strings are not altered by this keyword.
- """
- result = ''
- for s in strings:
- if isinstance(s,str) or isinstance(s,unicode):
- result += s
- if result == '':
- return None
- else:
- return result
-
-
-def compare_xml(xml1, xml2):
- """
- compare the two XML files to see if they contain the same data
- but could be if different order.
- It just split the xml in to lines and just check the line is in
- the other file
- """
- for line in xml1.rstrip().split('\n'):
- if line not in xml2.rstrip().split('\n'):
- return False
-
- for line in xml2.rstrip().split('\n'):
- if line not in xml1.rstrip().split('\n'):
- return False
-
- return True
-
-def num_of_nodes(depth, fanout):
- '''returns num of switches of a mininet with tree topology
- with particular depth and fanout parameters
- '''
- result = 0
- for i in xrange(depth):
- result += fanout**i
- return result
-
-def num_of_links_for_node(nodeid, leaflist, fanout):
- '''
- If the given node is a leaf node, there will be an only one link for it
- and nodeid will be represented 2 times in topology
- If the given node is not a leaf node, then there will be fanout+1 links
- for it and nodeid will be represented (fanout+1)*2 times in topology
-
- p.s. root node is excluded.
- '''
- if nodeid in leaflist:
- return 1
- return (fanout+1)
-
-if __name__ == '__main__':
- print num_of_nodes(3,4)
- pass
diff --git a/testcases/Controllers/ODL/CI/libraries/RequestsLibrary.py b/testcases/Controllers/ODL/CI/libraries/RequestsLibrary.py
deleted file mode 100644
index 3ef4375d6..000000000
--- a/testcases/Controllers/ODL/CI/libraries/RequestsLibrary.py
+++ /dev/null
@@ -1,264 +0,0 @@
-import requests
-import json
-
-from urllib import urlencode
-
-
-import robot
-
-from robot.libraries.BuiltIn import BuiltIn
-
-
-class RequestsLibrary(object):
- ROBOT_LIBRARY_SCOPE = 'Global'
-
- def __init__(self):
- self._cache = robot.utils.ConnectionCache('No sessions created')
- self.builtin = BuiltIn()
-
- def _utf8_urlencode(self, data):
- if not type(data) is dict:
- return data
-
- utf8_data = {}
- for k,v in data.iteritems():
- utf8_data[k] = unicode(v).encode('utf-8')
- return urlencode(utf8_data)
-
- def create_session(self, alias, url, headers={}, cookies=None,
- auth=None, timeout=None, proxies=None,
- verify=False):
-
- """ Create Session: create a HTTP session to a server
-
- `url` Base url of the server
-
- `alias` Robot Framework alias to identify the session
-
- `headers` Dictionary of default headers
-
- `auth` Dictionary of username & password for HTTP Basic Auth
-
- `timeout` connection timeout
-
- `proxies` proxy server url
-
- `verify` set to True if Requests should verify the certificate
- """
-
- self.builtin.log('Creating session: %s' % alias, 'DEBUG')
- auth = requests.auth.HTTPBasicAuth(*auth) if auth else None
- s = session = requests.Session()
- s.headers.update(headers)
- s.auth = auth if auth else s.auth
- s.proxies = proxies if proxies else s.proxies
-
- s.verify = self.builtin.convert_to_boolean(verify)
-
- # cant pass these into the Session anymore
- self.timeout = timeout
- self.cookies = cookies
- self.verify = verify
-
- # cant use hooks :(
- s.url = url
-
- self._cache.register(session, alias=alias)
- return session
-
- def delete_all_sessions(self):
- """ Removes all the session objects """
-
- self._cache.empty_cache()
-
- def to_json(self, content):
- """ Convert a string to a JSON object
-
- `content` String content to convert into JSON
- """
- return json.loads(content)
-
-
- def _get_url(self, session, uri):
- ''' Helpere method to get the full url
- '''
- url = session.url
- if uri:
- slash = '' if uri.startswith('/') else '/'
- url = "%s%s%s" %(session.url, slash, uri)
- return url
-
- def get(self, alias, uri, headers=None):
- """ Send a GET request on the session object found using the
- given `alias`
-
- `alias` that will be used to identify the Session object in the cache
-
- `uri` to send the GET request to
-
- `headers` a dictionary of headers to use with the request
- """
-
- session = self._cache.switch(alias)
- resp = session.get(self._get_url(session, uri),
- headers=headers,
- cookies=self.cookies, timeout=self.timeout)
-
- # store the last response object
- session.last_resp = resp
- return resp
-
- def post(self, alias, uri, data={}, headers=None, files={}):
- """ Send a POST request on the session object found using the
- given `alias`
-
- `alias` that will be used to identify the Session object in the cache
-
- `uri` to send the GET request to
-
- `data` a dictionary of key-value pairs that will be urlencoded
- and sent as POST data
- or binary data that is sent as the raw body content
-
- `headers` a dictionary of headers to use with the request
-
- `files` a dictionary of file names containing file data to POST to the server
- """
-
- session = self._cache.switch(alias)
- data = self._utf8_urlencode(data)
-
- resp = session.post(self._get_url(session, uri),
- data=data, headers=headers,
- files=files,
- cookies=self.cookies, timeout=self.timeout)
-
- # store the last response object
- session.last_resp = resp
- self.builtin.log("Post response: " + resp.content, 'DEBUG')
- return resp
-
- def postjson(self, alias, uri, data={}, headers=None, files={}):
- """ Send a POST request on the session object found using the
- given `alias`
-
- `alias` that will be used to identify the Session object in the cache
-
- `uri` to send the GET request to
-
- `data` a dictionary of key-value pairs that will be urlencoded
- and sent as POST data
- or binary data that is sent as the raw body content
-
- `headers` a dictionary of headers to use with the request
-
- `files` a dictionary of file names containing file data to POST to the server
- """
-
- session = self._cache.switch(alias)
- data = json.dumps(data)
-
- resp = session.post(self._get_url(session, uri),
- data=data, headers=headers,
- files=files,
- cookies=self.cookies, timeout=self.timeout)
-
- # store the last response object
- session.last_resp = resp
- self.builtin.log("Post response: " + resp.content, 'DEBUG')
- return resp
-
- def put(self, alias, uri, data=None, headers=None):
- """ Send a PUT request on the session object found using the
- given `alias`
-
- `alias` that will be used to identify the Session object in the cache
-
- `uri` to send the PUT request to
-
- `headers` a dictionary of headers to use with the request
-
- """
-
- session = self._cache.switch(alias)
- #data = self._utf8_urlencode(data)
- data = json.dumps(data)
-
- resp = session.put(self._get_url(session, uri),
- data=data, headers=headers,
- cookies=self.cookies, timeout=self.timeout)
-
- self.builtin.log("PUT response: %s DEBUG" % resp.content)
-
- # store the last response object
- session.last_resp = resp
- return resp
-
- def put_xml(self, alias, uri, data=None, headers=None):
- """ Send a PUT_xml request on the session object found using the
- given `alias`
-
- `alias` that will be used to identify the Session object in the cache
-
- `uri` to send the PUT_xml request to
-
- `headers` a dictionary of headers to use with the request
-
- """
-
- session = self._cache.switch(alias)
- data = self._utf8_urlencode(data)
- #data = json.dumps(data)
-
- resp = session.put(self._get_url(session, uri),
- data=data, headers=headers,
- cookies=self.cookies, timeout=self.timeout)
-
- self.builtin.log("PUT response: %s DEBUG" % resp.content)
-
- # store the last response object
- session.last_resp = resp
- return resp
-
- def delete(self, alias, uri, data=(), headers=None):
- """ Send a DELETE request on the session object found using the
- given `alias`
-
- `alias` that will be used to identify the Session object in the cache
-
- `uri` to send the DELETE request to
-
- `headers` a dictionary of headers to use with the request
-
- """
-
- session = self._cache.switch(alias)
- args = "?%s" % urlencode(data) if data else ''
- resp = session.delete("%s%s" % (self._get_url(session, uri), args),
- headers=headers, cookies=self.cookies,
- timeout=self.timeout)
-
- # store the last response object
- session.last_resp = resp
- return resp
-
-
- def head(self, alias, uri, headers=None):
- """ Send a HEAD request on the session object found using the
- given `alias`
-
- `alias` that will be used to identify the Session object in the cache
-
- `uri` to send the HEAD request to
-
- `headers` a dictionary of headers to use with the request
-
- """
-
- session = self._cache.switch(alias)
- resp = session.head(self._get_url(session, uri), headers=headers,
- cookies=self.cookies, timeout=self.timeout)
-
- # store the last response object
- session.last_resp = resp
- return resp
diff --git a/testcases/Controllers/ODL/CI/libraries/Utils.txt b/testcases/Controllers/ODL/CI/libraries/Utils.txt
deleted file mode 100644
index 913ba22c0..000000000
--- a/testcases/Controllers/ODL/CI/libraries/Utils.txt
+++ /dev/null
@@ -1,106 +0,0 @@
-*** Settings ***
-Library SSHLibrary
-Library ./UtilLibrary.py
-
-*** Variables ***
-${start} sudo mn --controller=remote,ip=${CONTROLLER} --topo tree,1 --switch ovsk,protocols=OpenFlow13
-${linux_prompt} >
-
-*** Keywords ***
-Start Suite
- [Documentation] Basic setup/cleanup work that can be done safely before any system
- ... is run.
- Log Start the test on the base edition
- ${mininet_conn_id}= Open Connection ${MININET} prompt=${linux_prompt} timeout=30s
- Set Suite Variable ${mininet_conn_id}
- Login With Public Key ${MININET_USER} ${USER_HOME}/.ssh/id_rsa any
- Write sudo ovs-vsctl set-manager ptcp:6644
- Read Until ${linux_prompt}
- Write sudo mn -c
- Read Until ${linux_prompt}
- Write ${start}
- Read Until mininet>
- Sleep 6
-
-Stop Suite
- [Documentation] Cleanup/Shutdown work that should be done at the completion of all
- ... tests
- Log Stop the test on the base edition
- Switch Connection ${mininet_conn_id}
- Read
- Write exit
- Read Until ${linux_prompt}
- Close Connection
-
-Ensure All Nodes Are In Response
- [Arguments] ${URI} ${node_list}
- [Documentation] A GET is made to the supplied ${URI} and every item in the ${node_list}
- ... is verified to exist in the repsonse. This keyword currently implies that it's node
- ... specific but any list of strings can be given in ${node_list}. Refactoring of this
- ... to make it more generic should be done. (see keyword "Check For Elements At URI")
- : FOR ${node} IN @{node_list}
- \ ${resp} RequestsLibrary.Get session ${URI}
- \ Should Be Equal As Strings ${resp.status_code} 200
- \ Should Contain ${resp.content} ${node}
-
-Check Nodes Stats
- [Arguments] ${node}
- [Documentation] A GET on the /node/${node} API is made and specific flow stat
- ... strings are checked for existence.
- ${resp} RequestsLibrary.Get session ${REST_CONTEXT}/node/${node}
- Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} flow-capable-node-connector-statistics
- Should Contain ${resp.content} flow-table-statistics
-
-Check That Port Count Is Ok
- [Arguments] ${node} ${count}
- [Documentation] A GET on the /port API is made and the specified port ${count} is
- ... verified. A more generic Keyword "Check For Specific Number Of Elements At URI"
- ... also does this work and further consolidation should be done.
- ${resp} RequestsLibrary.Get session ${REST_CONTEXT}/${CONTAINER}/port
- Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- Should Contain X Times ${resp.content} ${node} ${count}
-
-Check For Specific Number Of Elements At URI
- [Arguments] ${uri} ${element} ${expected_count}
- [Documentation] A GET is made to the specified ${URI} and the specific count of a
- ... given element is done (as supplied by ${element} and ${expected_count})
- ${resp} RequestsLibrary.Get session ${uri}
- Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- Should Contain X Times ${resp.content} ${element} ${expected_count}
-
-Check For Elements At URI
- [Arguments] ${uri} ${elements}
- [Documentation] A GET is made at the supplied ${URI} and every item in the list of
- ... ${elements} is verified to exist in the response
- ${resp} RequestsLibrary.Get session ${uri}
- Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- : FOR ${i} IN @{elements}
- \ Should Contain ${resp.content} ${i}
-
-Check For Elements Not At URI
- [Arguments] ${uri} ${elements}
- [Documentation] A GET is made at the supplied ${URI} and every item in the list of
- ... ${elements} is verified to NOT exist in the response
- ${resp} RequestsLibrary.Get session ${uri}
- Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- : FOR ${i} IN @{elements}
- \ Should Not Contain ${resp.content} ${i}
-
-Extract Value From Content
- [Arguments] ${content} ${index} ${strip}=nostrip
- [Documentation] Will take the given response content and return the value at the given index as a string
- ${value}= Get Json Value ${content} ${index}
- ${value}= Convert To String ${value}
- ${value}= Run Keyword If '${strip}' == 'strip' Strip Quotes ${value}
- [Return] ${value}
-
-Strip Quotes
- [Arguments] ${string_to_strip}
- [Documentation] Will strip ALL quotes from given string and return the new string
- ${string_to_return}= Replace String ${string_to_strip} " \ count=-1
- [Return] ${string_to_return}
diff --git a/testcases/Controllers/ODL/CI/start_tests.sh b/testcases/Controllers/ODL/CI/start_tests.sh
new file mode 100644
index 000000000..e2f94a65a
--- /dev/null
+++ b/testcases/Controllers/ODL/CI/start_tests.sh
@@ -0,0 +1,78 @@
+#!/bin/bash
+# Script requires that test environment is created already
+# it includes python2.7 virtual env with robot packages and git
+
+# Colors
+green='\033[0;32m'
+light_green='\033[1;32m'
+nc='\033[0m' # No Color
+
+usage="Script for starting ODL tests. Tests to be executed are specified in test_list.txt file.
+
+usage:
+[var=value] bash $(basename "$0") [-h]
+
+where:
+ -h show this help text
+ var one of the following: OSTACK_IP, ODL_PORT, USER, PASS, PATH_TO_VENV
+ value new value for var
+
+example:
+ OSTACK_IP=oscontro1 ODL_PORT=8080 bash $(basename "$0")"
+
+while getopts ':h' option; do
+ case "$option" in
+ h) echo "$usage"
+ exit
+ ;;
+ \?) printf "illegal option: -%s\n" "$OPTARG" >&2
+ echo "$usage" >&2
+ exit 1
+ ;;
+ esac
+done
+
+echo -e "${green}Current environment parameters for ODL suite.${nc}"
+# Following vars might be also specified as CLI params
+set -x
+PATH_TO_VENV=${PATH_TO_VENV:-~/.virtualenvs/robot/bin/activate}
+OSTACK_IP=${OSTACK_IP:-'oscontrol'}
+ODL_PORT=${ODL_PORT:-8081}
+USR_NAME=${USR_NAME:-'admin'}
+PASS=${PASS:-'octopus'}
+set +x
+
+echo -e "${green}Cloning ODL integration git repo.${nc}"
+if [ -d integration ]; then
+ cd integration
+ git checkout -- .
+ git pull
+ cd -
+else
+ git clone https://github.com/opendaylight/integration.git
+fi
+
+# Change openstack password for admin tenant in neutron suite
+sed -i "s/\"password\": \"admin\"/\"password\": \"${PASS}\"/" integration/test/csit/suites/openstack/neutron/__init__.robot
+
+echo -e "${green}Activate python virtual env.${nc}"
+source $PATH_TO_VENV
+
+# List of tests are specified in test_list.txt
+# those are relative paths to test directories from integartion suite
+echo -e "${green}Executing chosen tests.${nc}"
+while read line
+do
+ # skip comments
+ [[ ${line:0:1} == "#" ]] && continue
+ # skip empty lines
+ [[ -z "${line}" ]] && continue
+
+ echo -e "${light_green}Starting test: $line ${nc}"
+ pybot -v OPENSTACK:${OSTACK_IP} -v PORT:${ODL_PORT} -v CONTROLLER:${OSTACK_IP} $line
+done < test_list.txt
+
+echo -e "${green}Deactivate venv.${nc}"
+deactivate
+
+# Now we can copy output.xml, log.html and report.xml files generated by robot.
diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/010__networks.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/010__networks.txt
deleted file mode 100644
index ac5080a2e..000000000
--- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/010__networks.txt
+++ /dev/null
@@ -1,58 +0,0 @@
-*** Settings ***
-Documentation Checking Network created in OpenStack are pushed to OpenDaylight
-Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH}
-Suite Teardown Delete All Sessions
-Library SSHLibrary
-Library Collections
-Library OperatingSystem
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${ODLREST} /controller/nb/v2/neutron/networks
-${OSREST} /v2.0/networks
-${postNet} {"network":{"name":"odl_network","admin_state_up":true}}
-
-*** Test Cases ***
-Check OpenStack Networks
- [Documentation] Checking OpenStack Neutron for known networks
- [Tags] Network Neutron OpenStack
- Log ${X-AUTH}
- ${resp} get OSSession ${OSREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${OSResult} To Json ${resp.content}
- Set Suite Variable ${OSResult}
- Log ${OSResult}
-
-Check OpenDaylight Networks
- [Documentation] Checking OpenDaylight Neutron API for Known Networks
- [Tags] Network Neutron OpenDaylight
- Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH}
- ${resp} get ODLSession ${ODLREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${ODLResult} To Json ${resp.content}
- Set Suite Variable ${ODLResult}
- Log ${ODLResult}
-
-Create Network
- [Documentation] Create new network in OpenStack
- [Tags] Create Network OpenStack Neutron
- Log ${postNet}
- ${resp} post OSSession ${OSREST} data=${postNet}
- Should be Equal As Strings ${resp.status_code} 201
- ${result} To JSON ${resp.content}
- ${result} Get From Dictionary ${result} network
- ${NETID} Get From Dictionary ${result} id
- Log ${result}
- Log ${NETID}
- Set Global Variable ${NETID}
- sleep 2
-
-Check Network
- [Documentation] Check Network created in OpenDaylight
- [Tags] Check Network OpenDaylight
- ${resp} get ODLSession ${ODLREST}/${NetID}
- Should be Equal As Strings ${resp.status_code} 200
-
-
diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/020__subnets.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/020__subnets.txt
deleted file mode 100644
index 5e0c417d8..000000000
--- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/020__subnets.txt
+++ /dev/null
@@ -1,58 +0,0 @@
-*** Settings ***
-Documentation Checking Subnets created in OpenStack are pushed to OpenDaylight
-Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH}
-Suite Teardown Delete All Sessions
-Library SSHLibrary
-Library Collections
-Library OperatingSystem
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${ODLREST} /controller/nb/v2/neutron/subnets
-${OSREST} /v2.0/subnets
-${data} {"subnet":{"name":"odl_subnet","network_id":"${NETID}","ip_version":4,"cidr":"172.16.64.0/24","allocation_pools":[{"start":"172.16.64.20","end":"172.16.64.120"}]}}
-
-*** Test Cases ***
-Check OpenStack Subnets
- [Documentation] Checking OpenStack Neutron for known Subnets
- [Tags] Subnets Neutron OpenStack
- Log ${X-AUTH}
- ${resp} get OSSession ${OSREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${OSResult} To Json ${resp.content}
- Set Suite Variable ${OSResult}
- Log ${OSResult}
-
-Check OpenDaylight subnets
- [Documentation] Checking OpenDaylight Neutron API for Known Subnets
- [Tags] Subnets Neutron OpenDaylight
- Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH}
- ${resp} get ODLSession ${ODLREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${ODLResult} To Json ${resp.content}
- Set Suite Variable ${ODLResult}
- Log ${ODLResult}
-
-Create New subnet
- [Documentation] Create new subnet in OpenStack
- [Tags] Create Subnet OpenStack Neutron
- Log ${data}
- ${resp} post OSSession ${OSREST} data=${data}
- Should be Equal As Strings ${resp.status_code} 201
- ${result} To JSON ${resp.content}
- ${result} Get From Dictionary ${result} subnet
- ${SUBNETID} Get From Dictionary ${result} id
- Log ${result}
- Log ${SUBNETID}
- Set Global Variable ${SUBNETID}
- sleep 2
-
-Check New subnet
- [Documentation] Check new subnet created in OpenDaylight
- [Tags] Check subnet OpenDaylight
- ${resp} get ODLSession ${ODLREST}/${SUBNETID}
- Should be Equal As Strings ${resp.status_code} 200
-
-
diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/030__ports.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/030__ports.txt
deleted file mode 100644
index 5aafd51d6..000000000
--- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/030__ports.txt
+++ /dev/null
@@ -1,58 +0,0 @@
-*** Settings ***
-Documentation Checking Port created in OpenStack are pushed to OpenDaylight
-Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH}
-Suite Teardown Delete All Sessions
-Library SSHLibrary
-Library Collections
-Library OperatingSystem
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${ODLREST} /controller/nb/v2/neutron/ports
-${OSREST} /v2.0/ports
-${data} {"port":{"name":"odl_port","network_id":"${NETID}","admin_state_up": true}}
-
-*** Test Cases ***
-Check OpenStack ports
- [Documentation] Checking OpenStack Neutron for known ports
- [Tags] Ports Neutron OpenStack
- Log ${X-AUTH}
- ${resp} get OSSession ${OSREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${OSResult} To Json ${resp.content}
- Set Suite Variable ${OSResult}
- Log ${OSResult}
-
-Check OpenDaylight ports
- [Documentation] Checking OpenDaylight Neutron API for Known Ports
- [Tags] Ports Neutron OpenDaylight
- Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH}
- ${resp} get ODLSession ${ODLREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${ODLResult} To Json ${resp.content}
- Set Suite Variable ${ODLResult}
- Log ${ODLResult}
-
-Create New Port
- [Documentation] Create new port in OpenStack
- [Tags] Create port OpenStack Neutron
- Log ${data}
- ${resp} post OSSession ${OSREST} data=${data}
- Should be Equal As Strings ${resp.status_code} 201
- ${result} To JSON ${resp.content}
- ${result} Get From Dictionary ${result} port
- ${PORTID} Get From Dictionary ${result} id
- Log ${result}
- Log ${PORTID}
- Set Global Variable ${PORTID}
- sleep 2
-
-Check New Port
- [Documentation] Check new port created in OpenDaylight
- [Tags] Check subnet OpenDaylight
- ${resp} get ODLSession ${ODLREST}/${PORTID}
- Should be Equal As Strings ${resp.status_code} 200
-
-
diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/040__delete_ports.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/040__delete_ports.txt
deleted file mode 100644
index 5aca2c869..000000000
--- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/040__delete_ports.txt
+++ /dev/null
@@ -1,37 +0,0 @@
-*** Settings ***
-Documentation Checking Port deleted in OpenStack are deleted also in OpenDaylight
-Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH}
-Suite Teardown Delete All Sessions
-Library SSHLibrary
-Library Collections
-Library OperatingSystem
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${ODLREST} /controller/nb/v2/neutron/ports
-${OSREST} /v2.0/ports/${PORTID}
-${data} {"port":{"network_id":"${NETID}","admin_state_up": true}}
-
-*** Test Cases ***
-Delete New Port
- [Documentation] Delete previously created port in OpenStack
- [Tags] Delete port OpenStack Neutron
- Log ${data}
- ${resp} delete OSSession ${OSREST}
- Should be Equal As Strings ${resp.status_code} 204
- Log ${resp.content}
- sleep 2
-
-Check Port Deleted
- [Documentation] Check port deleted in OpenDaylight
- [Tags] Check port deleted OpenDaylight
- Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH}
- ${resp} get ODLSession ${ODLREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${ODLResult} To Json ${resp.content}
- Set Suite Variable ${ODLResult}
- Log ${ODLResult}
- ${resp} get ODLSession ${ODLREST}/${PORTID}
- Should be Equal As Strings ${resp.status_code} 404
diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/050__delete_subnets.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/050__delete_subnets.txt
deleted file mode 100644
index 04c73cc2e..000000000
--- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/050__delete_subnets.txt
+++ /dev/null
@@ -1,37 +0,0 @@
-*** Settings ***
-Documentation Checking Subnets deleted in OpenStack are deleted also in OpenDaylight
-Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH}
-Suite Teardown Delete All Sessions
-Library SSHLibrary
-Library Collections
-Library OperatingSystem
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${ODLREST} /controller/nb/v2/neutron/subnets
-${OSREST} /v2.0/subnets/${SUBNETID}
-${data} {"subnet":{"network_id":"${NETID}","ip_version":4,"cidr":"172.16.64.0/24","allocation_pools":[{"start":"172.16.64.20","end":"172.16.64.120"}]}}
-
-*** Test Cases ***
-Delete New subnet
- [Documentation] Delete previously created subnet in OpenStack
- [Tags] Delete Subnet OpenStack Neutron
- Log ${data}
- ${resp} delete OSSession ${OSREST}
- Should be Equal As Strings ${resp.status_code} 204
- Log ${resp.content}
- sleep 2
-
-Check New subnet deleted
- [Documentation] Check subnet deleted in OpenDaylight
- [Tags] Check subnet deleted OpenDaylight
- Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH}
- ${resp} get ODLSession ${ODLREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${ODLResult} To Json ${resp.content}
- Set Suite Variable ${ODLResult}
- Log ${ODLResult}
- ${resp} get ODLSession ${ODLREST}/${SUBNETID}
- Should be Equal As Strings ${resp.status_code} 404
diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/060__delete_networks.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/060__delete_networks.txt
deleted file mode 100644
index 31963ec97..000000000
--- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/060__delete_networks.txt
+++ /dev/null
@@ -1,37 +0,0 @@
-*** Settings ***
-Documentation Checking Network deleted in OpenStack are deleted also in OpenDaylight
-Suite Setup Create Session OSSession http://${OPENSTACK}:9696 headers=${X-AUTH}
-Suite Teardown Delete All Sessions
-Library SSHLibrary
-Library Collections
-Library OperatingSystem
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${ODLREST} /controller/nb/v2/neutron/networks
-${OSREST} /v2.0/networks/${NETID}
-${postNet} {"network":{"name":"odl_network","admin_state_up":true}}
-
-*** Test Cases ***
-Delete Network
- [Documentation] Delete network in OpenStack
- [Tags] Delete Network OpenStack Neutron
- Log ${postNet}
- ${resp} delete OSSession ${OSREST}
- Should be Equal As Strings ${resp.status_code} 204
- Log ${resp.content}
- sleep 2
-
-Check Network deleted
- [Documentation] Check Network deleted in OpenDaylight
- [Tags] Check Network OpenDaylight
- Create Session ODLSession http://${CONTROLLER}:${PORT} headers=${HEADERS} auth=${AUTH}
- ${resp} get ODLSession ${ODLREST}
- Should be Equal As Strings ${resp.status_code} 200
- ${ODLResult} To Json ${resp.content}
- Set Suite Variable ${ODLResult}
- Log ${ODLResult}
- ${resp} get ODLSession ${ODLREST}/${NetID}
- Should be Equal As Strings ${resp.status_code} 404
diff --git a/testcases/Controllers/ODL/CI/suites/openstack/neutron/__init__.txt b/testcases/Controllers/ODL/CI/suites/openstack/neutron/__init__.txt
deleted file mode 100644
index 4112b32ce..000000000
--- a/testcases/Controllers/ODL/CI/suites/openstack/neutron/__init__.txt
+++ /dev/null
@@ -1,27 +0,0 @@
-*** Settings ***
-Documentation Test suite for Neutron Plugin
-Suite Setup Start Suite
-Suite Teardown Stop Suite
-Library SSHLibrary
-Library Collections
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${UserInfo}= {"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "octopus"}}}
-
-** Keywords ***
-Start Suite
- Create Session KeyStoneSession http://${OPENSTACK}:5000 headers=${HEADERS}
- ${resp} post KeyStoneSession /v2.0/tokens ${UserInfo}
- Should Be Equal As Strings ${resp.status_code} 200
- ${result} To JSON ${resp.content}
- ${result} Get From Dictionary ${result} access
- ${result} Get From Dictionary ${result} token
- ${TOKEN} Get From Dictionary ${result} id
- ${X-AUTH} Create Dictionary X-Auth-Token ${TOKEN} Content-Type application/json
- Set Global Variable ${X-AUTH}
-Stop Suite
- Delete All Sessions
-
diff --git a/testcases/Controllers/ODL/CI/test_list.txt b/testcases/Controllers/ODL/CI/test_list.txt
new file mode 100644
index 000000000..e5e52129b
--- /dev/null
+++ b/testcases/Controllers/ODL/CI/test_list.txt
@@ -0,0 +1,5 @@
+# List of tests` which will be executed by script start_test.sh
+# You can specify path to specific robot test file or directory (in that case all tests from directory will be executed)
+
+integration/test/csit/suites/integration/basic/
+integration/test/csit/suites/openstack/neutron/
diff --git a/testcases/Controllers/ODL/CI/variables/Variables.py b/testcases/Controllers/ODL/CI/variables/Variables.py
deleted file mode 100644
index f406eec3f..000000000
--- a/testcases/Controllers/ODL/CI/variables/Variables.py
+++ /dev/null
@@ -1,20 +0,0 @@
-"""
-Library for the robot based system test tool of the OpenDaylight project.
-"""
-import collections
-
-# Global variables
-CONTROLLER = '10.2.91.18'
-PORT = '8081'
-PREFIX = 'http://' + CONTROLLER + ':' + PORT
-USER = 'admin'
-PWD = 'admin'
-AUTH = [u'admin',u'admin']
-HEADERS={'Content-Type': 'application/json'}
-HEADERS_XML={'Content-Type': 'application/xml'}
-ACCEPT_XML={'Accept': 'application/xml'}
-
-#TOKEN
-AUTH_TOKEN_API='/oauth2/token'
-REVOKE_TOKEN_API='/oauth2/revoke'
-
diff --git a/testcases/Controllers/ODL/ODL.md b/testcases/Controllers/ODL/ODL.md
index ceb04195e..375f20e6c 100644
--- a/testcases/Controllers/ODL/ODL.md
+++ b/testcases/Controllers/ODL/ODL.md
@@ -23,11 +23,6 @@ vcrpy==1.0.2
wsgiref==0.1.2
## Running tests
-
-Parameters are specified in:
-
-ODL/variables/Variables.py
-
-but can be also overidden in command line e.g.:
-
-pybot -v OPENSTACK:10.2.91.18 -v PORT:8081 -v CONTROLLER 10.2.91.18 .
+For more info:
+cd CI
+bash start_test.sh -h
diff --git a/testcases/VIM/OpenStack/OpenStack.md b/testcases/VIM/OpenStack/OpenStack.md
index a57699312..e1a85c305 100644
--- a/testcases/VIM/OpenStack/OpenStack.md
+++ b/testcases/VIM/OpenStack/OpenStack.md
@@ -4,27 +4,38 @@ Original Rally testsuites can be found here: https://github.com/stackforge/rally
---
## Intro
-In order to perform functional and performance testing, we use a dedicated VM.
-
- TODO: add test-tool VM in architecture and puppetise test-tool VM
-
-On this VM, we installed:
-* Rally: https://wiki.openstack.org/wiki/Rally
-
- TODO RobotFramework: http://robotframework.org, SIPP http://sipp.sourceforge.net/
-
-
+In order to perform functional and performance testing, we use Rally (see https://wiki.openstack.org/wiki/Rally for details).
+Rally must be installed as jenkins user on the jumphost machine of the OPNFV solution.
## Installation & Configuration
### Rally
-* Log on the test-tool VM
-* install Rally (https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html)
-
-openrc file (info from OpenStack) is needed, password is required during configuration procedure
+* Log on jumphost machine as jenkins user
+* Create the file existing.json, adapt it to your OpenStack (until agreement on default passwords)
+```bash
+{
+ "type": "ExistingCloud",
+ "auth_url": "http://example.net:5000/v2.0/",
+ "region_name": "RegionOne",
+ "endpoint_type": "public",
+ "admin": {
+ "username": "admin",
+ "password": "myadminpass",
+ "tenant_name": "demo"
+ },
+ "https_insecure": False,
+ "https_cacert": "",
+}
+```
+* Install Rally (ref https://rally.readthedocs.org/en/latest/tutorial/step_0_installation.html)
-* check
+```bash
+git clone https://git.openstack.org/stackforge/rally
+./rally/install_rally.sh -v
+rally deployment create --file=existing.json --name=existing
+```
+* you can check the available OpenStack services
```bash
# rally deployment check
keystone endpoints are valid and following service are available:
@@ -40,10 +51,8 @@ keystone endpoints are valid and following service are available:
| nova_ec2 | compute_ec2 | Available |
| novav3 | computev3 | Available |
+-----------+-------------+------------+
-
```
-
-* For Rally scenario, follow https://rally.readthedocs.org/en/latest/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html
+* You can start Rally scenario manually, follow https://rally.readthedocs.org/en/latest/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html
```bash
# rally task start ./samples/tasks/scenarios/nova/my-boot-and-delete.json
--------------------------------------------------------------------------------
@@ -118,14 +127,8 @@ HINTS:
Using task: f42c8aed-00a6-4715-9951-945b4fb97c32
```
-* For Tempest, follow the instructions https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler
-* In first step Rally scenario were fine but Tempest scenarios failed due to configuration
-Apply patch https://review.openstack.org/#/c/163330/
-```bash
-pip uninstall rally && cd ./rally && python setup.py install
-```
+* For Tempest, you can run the test manually by following the instructions https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler
-You shall be able to run Rally/Tempest towards your OpenStack
```bash
root@rally:~/rally# rally verify start
[...]
@@ -188,10 +191,6 @@ Tests:
Rally includes a reporting tool
https://rally.readthedocs.org/en/latest/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html
-
-
-
-
## Test description
### Rally
@@ -207,5 +206,13 @@ ceilometer designate glance keystone neutron quotas requests tempest-
tempest tests can be retrieved at https://github.com/openstack/tempest
+tests have been grouped and are available in https://git.opnfv.org/cgit/functest/tree/testcases/VIM/OpenStack/CI/suites
+
## Automation
+
+For automation, 2 job-templates have been created in https://git.opnfv.org/cgit/releng/tree/jjb/functest/functest.yml
+
+* functest-vim_bench-test: this template runs automatically a python script that runs the different rally scenario (except Tempest)
+* functest-vim_tempest-test: this template runs the rally command rally verify start
+