aboutsummaryrefslogtreecommitdiffstats
path: root/tests
AgeCommit message (Collapse)AuthorFilesLines
2017-06-05Merge "Pass parameters between scenarios"Rex Lee3-0/+73
2017-05-27Imporve monitor_process pass criteriaJingLu56-17/+17
JIRA: YARDSTICK-660 The monitor func()'s criteria in the monitor_process.py now is whether at least one process of the specific controller node service is recovered. But in reality is more resonable to use whether processes have been recoverd to it's original amount. This patch is aiming at improving the isssue Change-Id: I950ce2a89555801b96092735b0d670e892049927 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-05-26Pass parameters between scenarioschenjiankun3-0/+73
JIRA: YARDSTICK-641 Allowing parameters to pass between scenarios so that the one test case can be combination of several scenarios. Change-Id: I55a00855e77d5b719a27a069a3ea195d6bbd0ef8 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-05-18increase monitor number in cases.HuanLi12-0/+32
change cases to multi-monitor mode and improve monitor accuracy. JIRA: YARDSTICK-397 Change-Id: I6840423dd83899b769df76fbee808f9bf5dc4cfd Signed-off-by: HuanLi <lihuansse@tongji.edu.cn>
2017-05-16fix multi-monitor unitest hangRoss Brattain1-8/+8
need to mock SSH.from_node().execute.return_value because we switch most SSH calls to use from_node() factory function Change-Id: Ibea525c2fcc243d41de8ccbcd40bc98532d69006 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-05-16Bugfix: change monitor command in tc019JingLu51-1/+1
JIRA: YARDSTICK-655 The original openstack command used for monitoring high availability openstack nova-api service in tc019 is "nova image-list", the "openstack image list" command used in tc019 now may not correctly show the nova-api service status. This is patch swtich the monitor command to "openstack server list". Change-Id: I8810f28b4328ceb4d911a3041a24774ae9cae8f0 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-05-16Merge "Add a new monitor type: MultiMonitor that can run any number of other ↵Ross Brattain1-0/+61
monitors at the same time."
2017-05-12Create flavor from heat contextDanielMartinBuckley3-3/+181
JIRA: YARDSTICK-582 Create a customizable flavor via heat context. All heat parameters are configurable including Core Affinity. The default flavor name is XXXX-flavor where XXXX is stackname. Flavor attributes are taken from the heat context file. If a flavor attribute is not used it takes default attribute value. If flavor name is not specified it uses the server name + "-flavor" or stack-name + "-flavor". Compute node specific attributes are configurable via "extra_specs" attribute. See https://docs.openstack.org/admin-guide/compute-flavors.html for details. Change-Id: If4015970b889b0b95bfa8eba9491ebf31e92f2c7 Signed-off-by: DanielMartinBuckley <daniel.m.buckley@intel.com>
2017-05-11Merge "Extend TC008 to run pktgen-dpdk inside VM Need a fast path inside VM ↵Rex Lee3-6/+255
to verify full throughput of SRIOV and OVS-dpdk."
2017-05-10Merge "pass user_data to heat template"Ross Brattain1-0/+21
2017-05-06Bugfix: Support HA test cases in TripleOJingLu51-1/+1
Change-Id: Ib1f6f45677e66ca88fb546ea0662f52588e9d336 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-05-04Merge "Bugfix: Local Openstack Operation in HA test frameworks"Jing Lu2-2/+32
2017-05-04Bugfix: Local Openstack Operation in HA test frameworkstjuyinkanglin2-2/+32
JIRA: YARDSTICK-635 Change-Id: Ic27517714db9325e7a3b1ef623c49af61c36b2b5 Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-05-04Merge "Modify HA test cases and HA test script to support other installers"Jing Lu2-12/+20
2017-05-04Modify HA test cases and HA test script to support other installerstjuyinkanglin2-12/+20
JIRA: YARDSTICK-633 Change-Id: I65fd1ba11504dc61485f83c3bcc93bec4d41883b Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-05-03Merge "run ha test case in compass pod"Jing Lu1-4/+28
2017-05-03fix ha issue when run tc050~tc054 in cirexlee87761-3/+3
JIRA: YARDSTICK-634 Change-Id: I46681c7e8afe391eef9c5309470028167e911950 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-05-03run ha test case in compass podrexlee87761-4/+28
JIRA: YARDSTICK-634 Change-Id: I586d5177598efeda8ac3937d1eb04cd5b97b6ec6 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-05-01Extend TC008 to run pktgen-dpdk inside VMJing Zhang3-6/+255
Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk. Update 1: Change newly added file names to avoid conflict Update 2: Add more unit test cases Update 3: Fix default parameter typo for testpmd Update 4: Adapted to the pktgen-dpdk prompt change from "Pktgen>" to "Pktgen:/>", now just expect "Pktgen" Update 5: Per comment, merge common functions between latency and throughput tests to utils.py Update 6: Per comment, seperate the test case from TC008 to a new test case TC077 Change-Id: I1f7471d4ba77636a3a66c79c2652578321312185 JIRA: YARDSTICK-614 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-04-27bugfix: fix HA testcasesJingLu51-1/+1
Change-Id: Ib38e94610a108ff7195cefdfaf048e0f4fd894e8 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-25Bugfix: Make tc019 abd tc025 accept --task-args options to change pod.yamlxudan2-2/+6
JIRA: YARDSTICK-628 Change-Id: I7d1b84232a7477a2854ef1625cc63521999eff65 Signed-off-by: xudan <xudan16@huawei.com>
2017-04-24Merge "standardize ssh auth"Rex Lee39-335/+391
2017-04-23Merge "Add SRIOV support"Kubi5-2/+92
2017-04-18Add SRIOV supportJing Zhang5-2/+92
A generic provider network solution is introduced. To identify whether a network specified in the test case is a provider network new attributes are introduced in the test case network section: networks: test-net: cidr: '192.168.1.0/24' provider: "sriov" physical_network: 'physnet1' If the "provider" attribute is present, the network is an existing provider network. If the value is "sriov", binding:vnic_type=direct is added to the interface in the heat deployment template. In orchestrator/heat.py, the interface creating functions are given a new parameter that tells if the network in use is a provider network. The benchmark/contexts/model.py is changed to store the value of the provider attribute from the test case and function calls to port creation is updated with the provider parameter. The same change is made in contexts/heat.py as well. Also calls for creating a new tenant network is replaced for creating a new provider network if the provider attribute is present. Update-1: Change test_model.py Update-2: Per comment, change comment style to """" Update-3: Change test_heat.py Update-4: Add unit test cases to pass coverage test Update-5: Add SRIOV provider network example in opnfv_yardstick_tc008.yaml Update-6: Per comment, remove empty line in orchestrator/test_heat.py Update-7: Per comment, change comment lines in orchestrator/test_heat.py Update-8: Add more unit test cases to pass coverage test Update-9: Change to create SRIOV provider network on the fly so as to support co-current test runs Update-10: Per comment, init physical_network to 'physnet1' Change-Id: I76004c4fcc9bffcfd8ed021fd647e0cecb346ef4 JIRA: YARDSTICK-612 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-04-12Add a new monitor type: MultiMonitor that can run any number of other ↵LiHuan1-0/+61
monitors at the same time. JIRA: YARDSTICK-397 Change-Id: Ic5cb79f0820029e306373abead1ea43fac9abee2 Signed-off-by: HuanLi <lihuansse@tongji.edu.cn>
2017-04-11standardize ssh authRoss Brattain39-335/+391
we need to be following defautl paramiko rules, first use pkey, then key_filenames (autodetecting ~/.ssh/ keys), then password We have too much boilerplate redudant code everywhere, we need to standardize on a factory function that takes a node dict. Using Python3 ChainMap we can layer overrides and defaults. VNF descriptors have to default key_filename, password to Python None. The only way to do this is to omit key values if the variable is not defined, this way the dict will not have the value and it will default to Python None Add python2 chainmap backport Updated unittest mocking to use ssh.SSH.from_node Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-04-06pip install yardstick failed in ApexJingLu52-17/+0
JIRA: YARDSTICK-625 In Apex, yardstick uninstall and install via pip failed. This bug blocks yardstick CI jobs. This patch enables pip install yardstick in editable mode. So the install_yardstick() in yardstick_verify is unnecessary. Change-Id: If290dac5fee5584601064ae9753ae2340643d3ca Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-06Bugfix: fix ubuntu image name typoJingLu51-1/+1
Change-Id: I8722f2ddd096bc784f45d4bcf58ea6a84d319fb2 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-05Bugfix: fix query job status in TC074JingLu54-5/+5
JIRA: YARDSTICK-621 The storperf job status query in TC074 didn' work properly. The cause is the "type: status" is not passed in the URL. This patch also update storperf VM image to xenial. Change-Id: Idd9f501416b24612f6045a57ba2a95e2ed3a9572 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-05Merge "Add opnfv_os-ovn-nofeature-noha_daily test suite"Jing Lu1-0/+62
2017-04-05Merge "Yardstick virtualenv support"Jing Lu1-12/+9
2017-04-05Add opnfv_os-ovn-nofeature-noha_daily test suiteJingLu51-0/+62
JIRA: YARDSTICK-624 Change-Id: Ia783bec53e4d889736db7e510d28c32377d4f9b2 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-01Merge "yardstick offline support"Jing Lu2-36/+53
2017-04-01Bugfix: take test case modification into effect in load_images.shJingLu51-0/+2
JIRA: YARDSTICK-622 Bug log: https://build.opnfv.org/ci/view/armband/job/yardstick-fuel-armband-baremetal-daily-danube/24/console This error is because the file changes done in the load_images.sh is not updated. After file modification, pip install -U . is required to tack changes into effect. Change-Id: I8be3ffc963e3d5cec861b68a33efc2fa5bdabbd2 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-01Yardstick virtualenv supportchenjiankun1-12/+9
JIRA: YARDSTICK-620 Currently we recommend using docker to run yardstick. And it is hard to use virtualenv to install yardstick. So I modify install.sh in yardstick root path. It will support using virtualenv to install yardstick(including API) in linux. In this patch, I make yardstick support read yardstick configuration have priority over constants. Change-Id: I9ea1241b228532a6497451e6c8f232173ddb783e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-31Merge "Store packetsize in influxdb"Rex Lee1-3/+3
2017-03-31Store packetsize in influxdbJing Zhang1-3/+3
This allows calculating throughput in Gbps. Add: test_pktgen.py Change-Id: I7e2132ec71985c14570ef18b952b2264ebf7abbc JIRA: YARKSTICK-611 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-03-30Bugfix: Add test suite for opnfv_os-odl_l3-ovs-ha_daily scenarioJingLu51-0/+62
JIRA: YARDSTICK-617 Apex has a new scenario 'opnfv_os-odl_l3-ovs-ha_daily' in CI. This patch add yardstick test suite for this scenario. Change-Id: I50eb84fb506e7cf87cbd6bcb67f3df6d185d8b6b Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-03-28yardstick offline supportchenjiankun2-36/+53
JIRA: YARDSTICK-597 Currently yardstick can not run offline. In some case this will block the process, causing yardstick job failed The reason is yardstick will pull the latest code when run in CI. Actually we need not pull the latest code because the code in yardstick is already the latest code. So I remove them. Also I check the environment variable and yardstick-image when fetch openrc file and load images. Change-Id: I797da272932ab5292aa40585c82c119aa71b8849 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-28Merge "Bugfix: Error validating value 'cirros-0.3.5'"Jing Lu1-21/+25
2017-03-28Merge "Bugfix:add storperf flavor for TC074"Jing Lu3-5/+17
2017-03-28Bugfix: Error validating value 'cirros-0.3.5'JingLu51-21/+25
JIRA: YARDSTICK-607 some scenarios come with a Cirros-0.3.5 image, Yardstick's load_images.sh will load a second cirros-0.3.5 image, causing the following error: 2017-03-27 07:42:58,317 yardstick.orchestrator.heat heat.py:449 INFO Creating stack 'demo-2ada98f3' error: failed to deploy stack: 'ERROR: Property error: : resources.ares.demo-2ada98f3.properties.image: : Error validating value 'cirros-0.3.5': No images unique match found for {'name': u'cirros-0.3.5'} This patch fixs this error by skipping load cirros image if there already exist one in the glance. Change-Id: I4d43d82d331f6575efd4560be9b6be175e03cd1b Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-03-27Merge "model: pylint fixes"Rex Lee1-1/+1
2017-03-27Merge "Bugfix:HA test cases rely on dynamic ip addresses in fuel pods"Jing Lu10-10/+30
2017-03-25Bugfix:HA test cases rely on dynamic ip addresses in fuel podsJingLu510-10/+30
JIRA: YARDSTICK-602 change HA test cases to rely on dynamical generated pod.yaml in fuel pods Change-Id: I0efb01489eb2dd94babf1a45bff2fbf6adef5f6e Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-03-25Bugfix:add storperf flavor for TC074JingLu53-5/+17
JIRA: YARDSTICK-604 the storperf container now uses a flavor called "storperf", if this flavor is absent, stack creating process will fail. also the openstack variables that passed to the storperf container also needs a update Change-Id: Iab4be6491cc9b24526948e5571f0476ec885a93d Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-03-24Update cirros image to latest versionMichael Polenchuk3-5/+5
Change-Id: Iffa1e4529e5b6e9664ac503a5815ea9a034d7515 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-03-23model: pylint fixesRoss Brattain1-1/+1
remove redundnant parens replace type with isinstance fix import order Change-Id: I0407cbbf6993290f392f17c398827603b5bc1ebd Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-23Bugfix: TC002 image error in fuel-arm pod scenariosJingLu51-2/+2
JIRA: YARDSTICK-601 This error is cause because the TC 002 configuration file has been modified in image section, the old sed command in load_images.sh "sed -i 's/image: cirros-0.3.3/image: TestVM/g' tests/opnfv/test_cases/opnfv_yardstick_tc002.yaml samples/ping.yaml" didn't work properly. Change-Id: Ie897213622d05bb267f94b466eb2529e797d484e Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-03-22Merge "Yardstick: User interface for Yardstick."Jing Lu1-0/+72