summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-05-23Get HA test case results on failureMiikka Koistinen5-76/+56
Now, yardstick HA test cases (which have 'sla action: monitor') do not output results if the test case SLA validation fails. This patch modifies the task runner and the serviceHA scenario so that during an SLA validation failure the test result data is captured correctly. Do similar changes in the general scenario. fix missing variable k s/envrioment/environment JIRA: YARDSTICK-1115 Change-Id: I648a8b229600c0ad089320ac3f803698f73aa800 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com> (cherry picked from commit dd87e15c9c4f0aff3577c1bcd68bd86dd2d64898) Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-05-24Test case spec for SDN controller resilience in HA configGeorg Kunz2-0/+197
This is the test case specification for verifying the HA capabilities of a SDN controller running in a HA configuration. JIRA: HA-33 JIRA: YARDSTICK-955 Change-Id: I016ec1ef7bb51b79f8c21440abf8f04c923ef9dc Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
2018-05-20Test case spec for SDN Virtual Switch resilience.Dimitrios Markou2-0/+185
This is a test case for verifying the proper functionality of SDN Vswitches which are located in the Compute nodes of an OpenStack environment. This test case is unaware of the configuration (non-HA,HA) JIRA: HA-34 JIRA: YARDSTICK-955 Change-Id: I701bf1f9527b1a31b9cb52e102734e10c4162766 Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com> (cherry picked from commit 4cc2882c73a64e243f98c0448ca2d1f9a677953c)
2018-05-17Merge "Update samples/test_suite.yaml" into stable/fraserRodolfo Alonso Hernandez1-1/+2
2018-05-17Merge "Fix log typos in benchmark/scenarios/availability" into stable/fraserRodolfo Alonso Hernandez8-41/+37
2018-05-17Merge "Remove unused methods in SampleVNF" into stable/fraserRex Lee2-90/+0
2018-05-16Fix log typos in benchmark/scenarios/availabilityMiikka Koistinen8-41/+37
This commit fixes multiple log message typos and all the emerged pylint errors. MonitorProcess/MonitorOpenstackCmd.verify_SLA() repeat already logged information, so these prints are removed. JIRA: YARDSTICK-1145 Change-Id: Ifef26e4b4ff7766089caec24785511969c2d663e Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com> (cherry picked from commit 6fa51bdcc7ad7e4d407f2803fc05b1cfaf66c115)
2018-05-16Merge "Update the dummy-scenario-heat-context testcase" into stable/fraserRodolfo Alonso Hernandez1-3/+4
2018-05-16Merge "Document for Euphrates test case results" into stable/fraserJing Lu36-2886/+2086
2018-05-15Document for Euphrates test case resultsJingLu536-2886/+2086
JIRA: YARDSTICK-1132 In B and C releases, Yardstick provides an overview of the results of test cases developed by Yardstick, executed on OPNFV community labs. This work has not been done for the D and E releases, although result analysis for Euphrates is available at (https://wiki.opnfv.org/display/yardstick/Euphrates+release+test+results+analysis). We plan to restart to provide a basic test case results analysis for Fraser and compare performance data with Euphrates. This JIRA ticket will be resolved into 3 commits: 1. Document for E release test case data; 2. Document for F release test case data; 3. Document for results analysis. Change-Id: I372a60807ac3dfaab87e70cdb956c1059b019f7a Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 4c20fb57684bc21e7ae0461341088100e518307d)
2018-05-10Update samples/test_suite.yamlEmma Foley1-1/+2
The sample test suite at sample/test_suite.yaml contained incorrect syntax. It has been updated to add the "default" parameter to "task_args" when running a testcase. JIRA: YARDSTICK-1125 Change-Id: I53409168e47d969317145623df1e2c789396cc32 Signed-off-by: Emma Foley <emma.l.foley@intel.com> (cherry picked from commit 7b00d64494bdd61f2ab37005de76fa5f6496d45d)
2018-05-08Implement Virtual Switch resilience test caseDimitrios Markou3-0/+329
This testcase tests the resilience of the Virtual Switch in an OpenStack environment. It can only run in a non Ubuntu distro because it is limited by this bug [0] [0]: JIRA: YARDSTICK-1140 Change-Id: Ia1480fba9f3549d1144c80699056b501d3b07a0f Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com> (cherry picked from commit 53f5951a47fcaa5a3f8bcd6ea1020e54a1b285d3)
2018-05-08Bug Fix: Start openvswitch service in Centos distroDimitrios Markou1-4/+2
JIRA: YARDSTICK-1142 Change-Id: I82c479c95e10de37416081eacf2d8bee396e50dc Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com> (cherry picked from commit d00660e583ef89fe07149c7f0cffcced7cd72133)
2018-05-08Add support for restarting a serviceDimitrios Markou1-3/+4
JIRA: YARDSTICK-1138 Change-Id: Id57defb574eb7e0f5be81b52803cdb05f46796ab Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com> (cherry picked from commit 131b9f04a360ccc894510a275bd33c57634608fc)
2018-05-08Add ODL HA testcaseDimitrios Markou2-0/+283
This testcase confirms the resiliency of Opendaylight in a HA deployment SPEC: https://gerrit.opnfv.org/gerrit/#/c/50991 JIRA: HA-33 JIRA: YARSTICK-955 Change-Id: I4cb39d1970ed0333a43fbb58f8c7138908379dcb Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com> (cherry picked from commit d16b2e14fe420a646d4399986bd9f4d1ae3a606d)
2018-05-08Add pod.yaml files for ApexDimitrios Markou3-0/+153
JIRA: YARDSTICK-1097 Change-Id: If116174c562bc08c0b7602324599bc85d8c60be0 Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com> (cherry picked from commit 6f374141466640655cc57277b4b49d6deffa9092)
2018-05-04Update the dummy-scenario-heat-context testcaseEmma Foley1-3/+4
The testcase is included in the docs to demonstrate definining a heat context. It previously used cirros, but this is not included by default. Set-up instructions/scripts add yardstick-image and yardstick-flavor, so these have been added to the testcase instead. A default value for context_name was also added. Change-Id: I5f642086971487b334ae48a5cf5c9b71e9b99e2e Signed-off-by: Emma Foley <emma.l.foley@intel.com> (cherry picked from commit f37e4dcf771caec3be52e95047997aa22cbd7a88)
2018-05-03Remove unused methods in SampleVNFEmma Foley2-90/+0
The following private methods are unused in yardstick/network_services/vnf_generic/vnf/sample_vnf.py:SampleVNF * _build_ports(self) * _get_route_data(self, route_index, route_type) * _get_port0localip6(self) * _get_port1localip6(self) * _get_port0prefixlen6(self) * _get_port1prefixlen6(self) * _get_port0gateway6(self) * _get_port1gateway6(self) They are being removed as: * _get_port* functions are already implemented and used in MultiPortConfig * _get_route_data is an auxillary functions used by the previous ones * _build_ports is implemented in the helpers JIRA: YARDSTICK-1122 Change-Id: Ie7a706cd87b610ea56f89214118a02e612a70dc0 Signed-off-by: Emma Foley <emma.l.foley@intel.com> (cherry picked from commit 50bcd3e477bc29abb43ed73643d551ff55a73403)
2018-04-26[release] Yardstick Fraser 6.0.0 Release notesopnfv-6.0.0rexlee87761-305/+209
JIRA: YARDSTICK-1129 Change-Id: I5adb2e9547c58b90d3eafb0e2b90015ce5472cfa Signed-off-by: rexlee8776 <limingjiang@huawei.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit a324c1b922ae2598dbe25e9b2885620ef09f4901)
2018-04-26Bugfix: ansible can't not run without configchenjiankun1-1/+1
JIRA: YARDSTICK-1135 Ansible need ansible.cfg when run in the remote node. However the file name is wrong. Need: setup.cfg -> ansible.cfg Change-Id: Ifa674fb624ca83d90c3ac9ad3bfe148013bf8bfd Signed-off-by: chenjiankun <chenjiankun1@huawei.com> (cherry picked from commit 170842f20426ad244f3e49b3780b0256302b0be3)
2018-04-25nsb_setup: pin docker image version to release tagRoss Brattain1-1/+1
The install scripts pull tags for stable releases. We need to temporarily pin the image tag to the unreleased tag because we have to update the code before we actually tag the image. Or maybe we should make this a parameter to the script JIRA: YARDSTICK-1137 Change-Id: I2e5f13ffe19d1c19fd7b8e071df28fd225be3d79 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-04-24Merge "extra loopback cleanup" into stable/fraserRoss Brattain2-6/+15
2018-04-24Merge "[docs] Updated baremetal instalation instructions" into stable/fraserRoss Brattain1-0/+9
2018-04-24Merge "Bugfix: Can't get image list in API" into stable/fraserJack Chan6-47/+105
2018-04-23remove more old apexlake testcasesRoss Brattain2-8/+0
fix fraser failures Change-Id: I8bb12a762054a4c3274cc6e0297e948b92700e7e Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-04-23[docs] Updated baremetal instalation instructionsChornyi, TarasX1-0/+9
JIRA: YARDSTICK-1111 Change-Id: Iefab24f74c87a0855c2d70090ac34256ffa9200c Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com> (cherry picked from commit 1c4a5e82aec771aba616879a9a351bade6d6d8d0)
2018-04-22extra loopback cleanupRoss Brattain2-6/+15
try to cleanup in the shell script in case ansible errors maybe it is hopeless because of docker JIRA: YARDSTICK-1096 JIRA: YARDSTICK-1054 Change-Id: I759ded4731a7710a194a81530cfb35376eb7800d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-04-20disable NSB (DPDK) image load for ARMRoss Brattain1-1/+3
we skip the building now but we also need to skip the image loading JIRA: YARDSTICK-1124 Change-Id: I27c40d6319e74374282203132cfc2c7ebff9b094 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 15cec319ab06f161c239ead78da9ecdf1673b7ce)
2018-04-20disable DPDK images for ARMRoss Brattain1-1/+3
We are trying to compile DPDK for the os-nosdn-ovs-ha scenario because we added TC042  "Measure network latency using testpmd and pktgen-dpdk". However it looks like TC042 is only enabled for compass in opnfv_os-nosdn-kvm_ovs_dpdk-ha https://gerrit.opnfv.org/gerrit/#/c/54659/1/tests/opnfv/test_suites/opnfv_os-nosdn-kvm_ovs_dpdk-ha_daily.yaml - file_name: opnfv_yardstick_tc042.yaml constraint: installer: compass pod: huawei-pod1 We don't actually need TC042 and DPDK or os-nosdn-ovs-ha. But in order to run TC042 we have to know beforehand that we are going to use DPDK, so we have a conditional in the load_images.sh script that tries to guess when we will need a DPDK image. The conditional only checks for *[_-]ovs[_-]* if [[ $DEPLOY_SCENARIO == *[_-]ovs[_-]* ]]; then ansible-playbook \ -e img_property="nsb" \ -e YARD_IMG_ARCH=${YARD_IMG_ARCH} \ -vvv -i inventory.ini build_yardstick_image.yml if [ ! -f "${QCOW_NSB_IMAGE}" ]; then echo "Failed building QCOW NSB image" exit 1 fi fi For this release we won't be able to get DPDK images to compile on ARM, we should fix this conditional and not try to compile. Also change the shell glob to match *[_-]ovs_dpdk[_-]* to only enable DPDK images for ovs_dpdk scenarios JIRA: YARDSTICK-1124 Change-Id: I03142e6b8aa00dc9cdf9b446868fb723cd10198b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 50154694f8dc37726d456cfb8efddc6d3e126dbd)
2018-04-18remove apex lake tc007 from test suitesRoss Brattain4-52/+0
fixes opnfv_os-nosdn-fdio-noha_daily.yaml failures due to missing testcase Change-Id: I5f050d72334da46b94de5fa3662047bf694f7e3e Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 01ef85fc8234c8ddeaae65eceaf5bbb9545b9e98)
2018-04-18Merge "[userguide] Use literalinclude directive in 05-operation" into ↵Emma Foley1-75/+9
stable/fraser
2018-04-17Bugfix: Can't get image list in APIchenjiankun6-47/+105
JIRA: YARDSTICK-1110 To match OpenStack Pike release, we upgrade novaclient from 7.1.1 to 9.1.1. And the client interface has changed, it no longer has nova_client.images attribute. So I use shade instead. Later will change all same issue. Change-Id: Ie4f54069d4346e44e2ad925439930504b945cbad Signed-off-by: chenjiankun <chenjiankun1@huawei.com> (cherry picked from commit 5ac59e495d74b64ab377f1ba25bb00b11da14e67)
2018-04-17Merge "Bugfix: interface down HA tc050 attack and monitor logistic wrong" ↵Jack Chan2-91/+42
into stable/fraser
2018-04-16[userguide] Use literalinclude directive in 05-operationEmma Foley1-75/+9
Uses literalinclude directive instead of having a copy of the file content in documentation. This means that any update to this file is detected, and the most up-to-date content is used. If a testcase is updated, the updates wil be picked up. The examples used are ones that are tested reqularly. JIRA: YARDSTICK-1066 Change-Id: I15d967eb04d1e6f4348683d8746fa9989dceda94 Signed-off-by: Emma Foley <emma.l.foley@intel.com> (cherry picked from commit ee9995c42368ee04ba06cd101b086c57dbbb3bba)
2018-04-16Merge "[docs] update the user guide for "yardstick-operation"" into ↵Rex Lee4-1/+367
stable/fraser
2018-04-16Bugfix: interface down HA tc050 attack and monitor logistic wrongrexlee87762-91/+42
1. simplify the attack to be one interface 2. move monitor step before attack 3. update rst doc to match JIRA: YARDSTICK-1120 Change-Id: Icfa21d58e7ccdbeaca8a9a1a3098cceb21c981c3 Signed-off-by: rexlee8776 <limingjiang@huawei.com> (cherry picked from commit af7f0bea3a0d24bdf084622acc9932ccd261e42e)
2018-04-13Merge "[docs] Move user guide chapters to make room for new chapter." into ↵Rex Lee15-551/+437
stable/fraser
2018-04-12Bug fix in using Unixbenchliyin1-1/+1
JIRA: YARDSTICK-1117 This bug will cause the error in running unixbench. "no such file named ./run" So we have to fix it. Change-Id: I141fcb5567fbb0e3df97aee677cb946a25a9788e Signed-off-by: liyin <liyin11@huawei.com> (cherry picked from commit 941d3b0c0fa4583c49fa6fd2aa3686a1b55ab187)
2018-04-10[docs] update the user guide for "yardstick-operation"Emma Foley4-1/+367
* Add a new chapter: operation which shows how to use Yardstick and how to create test cases * Include details on how to reuse and keep VNFs between testcases JIRA: YARDSTICK-885 JIRA: YARDSTICK-886 Change-Id: I630e701136b9d573539fe69ea0a42018a67d040f Signed-off-by: Emma Foley <emma.l.foley@intel.com> (cherry picked from commit 5b4cee9606f26f4c208fcd1aec544ddd436b8162)
2018-04-10[docs] Move user guide chapters to make room for new chapter.Emma Foley15-551/+437
* Increment the numbering on chapters * Fix some of the changes flagged by doc8 Change-Id: I9dbf4a3d33c2b38f7567bc3493466e72a59e1a0c Signed-off-by: Emma Foley <emma.l.foley@intel.com> (cherry picked from commit 3248682fcc789f95ff8246a8d1f208a6a3abd46d)
2018-04-10Bugfix: k8 scenario fixrexlee87762-45/+19
JIRA: YARDSTICK-1109 Change-Id: Ic61a63099c3d2d4af5fa5634299e4b2946d53eaf Signed-off-by: rexlee8776 <limingjiang@huawei.com> (cherry picked from commit 3b1316647b50bc3bbb1eb6216aa07fd4dff34edf)
2018-04-07docker/supervisor.sh: Fix condition block endAlexandru Avadanii1-3/+3
Fixes: 18546b4f Change-Id: I51fcb48e790ff7c57273a3f6712ccb2b9bc4aa32 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit ead90064d12be552bd334828d41ffffa75fad01b)
2018-04-06[AArch64] Dockerfile: Fix malformed patchAlexandru Avadanii1-2/+2
Change-Id: Ia2dfc6c4def8e71e36747b8f27cbf22700f676b1 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 161f7d676b4404a5445681a7871e086b751fdafc)
2018-03-31ADD tc042 into CIliyin3-0/+31
JIRA: YARDSTICK-1108 We add this test case in ovs_dpdk-ha and ovs_dpdk-noha testsuites also add image build into load_images script Change-Id: I2b0c6b106dd98c3693df18dba46259ff8ef0a76e Signed-off-by: liyin <liyin11@huawei.com>
2018-03-29Merge "Fix RabbitMQ service installation and initialization"Ross Brattain2-3/+7
2018-03-29Merge "don't print exceptions in atexit handler"Ross Brattain1-1/+1
2018-03-29Merge "task: don't hide exceptions in Task"Ross Brattain1-6/+9
2018-03-29don't print exceptions in atexit handlerRoss Brattain1-1/+1
atexit handler calls terminate_all after the regular python execution path, and it looks like the traceback stack is None somehow. In this context log.debug("", exc_info=True) doesn't work it prints out NoneType on Python3 and causes other problems on Python2.7 remove the exc_info=True from the logging call JIRA: YARDSTICK-1107 Change-Id: Ida0a0ced7ff5e017e2f8608880e3bb531724af95 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-03-28add new Fraser scenariosRoss Brattain10-0/+304
New scenarios from wiki: https://wiki.opnfv.org/display/SWREL/Fraser+Scenario+Statu Change-Id: Ifd6e45e73be2bbb99743aa3f4981d22899aab92a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-03-28Merge "restore Heat stack failure logs for CI"Ross Brattain2-1/+27