aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-10-09[docs] Transition to local docs build job.Emma Foley6-0/+39
Docs builds have transitioned to using local build jobs. This allows Yardstick docs to be built and verified locally. The time taken to build and verify docs in the gate for new changes should be reduced, as the docs for each project will be built separately. This change also allow contributers to test their docs builds locally, using tox:: tox -e docs tox -e docs-linkcheck TODO: Update the dev guide to include new tox environment Related: DOCS-191 JIRA: YARDSTICK-1415 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Change-Id: Ib8bab85475dcd5127a2c41a2d55dce8f8d9da574
2018-10-05Update Ixia tg to use new API for traffic model creationSerhiy Pshyk1-1/+4
JIRA: YARDSTICK-1464 Change-Id: Ibbced9e0e04e077ec5c23fc15e97308dd11b7004 Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
2018-10-03Merge "Fix influxdb "field type conflict" error"Emma Foley1-3/+3
2018-10-03Merge "Fix NSB PROX L3FWD dropping packets"Abhijit Sinha2-2/+14
2018-10-03Merge "NSB PROX NFVi Test does not stop after reaching expected precision"Abhijit Sinha12-19/+36
2018-10-02Fix NSB PROX L3FWD dropping packetsPatrice Buriez2-2/+14
JIRA: YARDSTICK-1037 Adjusted number of RX/TX descriptors to 2048, instead of default 256, in order to cope with interrupts during which no packets are received by the core, which results in the NIC dropping packets when it gets out of available descriptors to buffer them. Also adjusted mempool size, because more descriptors need more mbufs. This change aligns L3FWD test case with other test cases, for which number of RX/TX descriptors is already set to 2048. Change-Id: I11378c078398aca799295b619bb3d14dbee04ca3 Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-10-02Fix influxdb "field type conflict" errorPatrice Buriez1-3/+3
JIRA: YARDSTICK-1457 Field types must be consistent across points in a measurement, otherwise influxdb rejects newer points with "field type conflict" error. This error was faced when "theor_max_throughput" field, meant to be assigned as a float from "TxThroughput" sample, was instead assigned as an int from initial value of theor_max_thruput variable. This especially occured for subsequent packet sizes, when theor_max_thruput variable was reset to its default int value. This fix changes the default value of theor_max_thruput variable to 0.0, and also makes sure that the default value for "Actual_throughput" field is consistently a float. Change-Id: If8d4f1297094709fe9657ab3e2b8adaad09815b2 Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-10-02Modify IxNextgen API for traffic model configurationSerhiy Pshyk2-31/+97
Changes to IxNextgen API: - modify API for traffic model configuration so that it can be used to create/configure different models - add methods to start and stop protocols on IxNetwork - add method for getting assigned ports JIRA: YARDSTICK-1455 Change-Id: I43c10fea9ba83994caf373ceff8b0252a45439b3 Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
2018-10-02NSB PROX NFVi Test does not stop after reaching expected precisionDanielMartinBuckley12-19/+36
JIRA: YARDSTICK-1419 When using prox_binsearch algorithm, a binary search is performed, increasing lower bound when step was successful and decreasing upper bound when the step was a failure. This runs until the test_precision (as specified in the traffic profile) is reached. When the test precision is reached, the test is not completed until the runner duration is reached. As runner duration is usually high (e.g. 1800sec), the tests take much too long to execute. This makes it difficult to create test suites. Change-Id: I6cc503a09fb534a556c61c805e6df4786bb8cc4b Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-09-22Update the pinned sampleVNF versionPatrice Buriez1-1/+13
JIRA: YARDSTICK-1414 Updated the sampleVNF version to the most recent validated commit. Provided comments on accepted version types and values, and on where users can find alternative values, in case they want to try something different than the version recommended and validated for Yardstick/NSB. Change-Id: I336f96237fcfa8b580c91c787d722d368cc6c015 Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-09-21Merge "SA: Fix private key file name generation issue"Abhijit Sinha1-1/+1
2018-09-21Merge "[NSB] Adds support for running "black box" test cases for VNFs"Abhijit Sinha7-0/+348
2018-09-21[NSB] Adds support for running "black box" test cases for VNFsChornyi, TarasX7-0/+348
Using AgnosticVNF is now possible to create a "black box" testcase for any VNF. Example rfc2544 test with TRex as trafficgen samples/vnf_samples/nsut/agnostic/tc_baremetal_rfc2544_ipv4_64B_trex.yaml and Ixia samples/vnf_samples/nsut/agnostic/tc_baremetal_rfc2544_latency_ipv4_64B_ixia.yaml NOTE: Yardstick will not interact with VNF. No KPIs from VNF will be collected JIRA: YARDSTICK-1438 Change-Id: I408b3dcb90f29c7b41fcf8f883b5abfc4e9c3dbc Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
2018-09-20Merge changes from topics 'YARDSTICK-1354', 'YARDSTICK-1348', ↵Abhijit Sinha35-0/+7335
'YARDSTICK-1359', 'YARDSTICK-1356' * changes: Add vEPC service request test cases Add network initiated dedicated bearers creation Add UE initiated dedicated bearer creation test Add vEPC infrastructure for Landslide TG Add vEPC default bearers relocation test case Add vEPC default bearers create/delete test case Add Spirent Landslide traffic profile templates Update LSResourceHelper unittests Add Spirent Landslide TG API Added Landslide Resource Helper implementation Add TclClients for Spirent Landslide TG
2018-09-20SA: Fix private key file name generation issueMytnyk, Volodymyr1-1/+1
In current implementation of Yardstick, SA context uses the same SSH key file name for each VM created. E.g. the Yardstick overrides the key file context each time the new VM is created. As a result, the key file is valid only for last VM created by Yardstick causing the connection problem for other VMs. Solution is to generate different key file name for each VM name separately. JIRA: YARDSTICK-1436 Change-Id: I22ae2874695736255aa3314654a2cced9338b53d Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-09-18Add IxNextgen API for creating Vlans configurationsSerhiy Pshyk2-0/+109
New IxNextgen API allows to configure Vlans(vlan id, priority, tp id) to Ethernet layer of device group. JIRA: YARDSTICK-1420 Change-Id: I1c5fe37b5beb11daf1c539933fcad60832553bb9 Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
2018-09-15Add vEPC service request test casesOrest Voznyy4-0/+1014
Add UE and network initiated testcase and related session profiles. JIRA: YARDSTICK-1429 Change-Id: I16d6c065cf346db31e235ac5add57509cfbc8009 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-15Add network initiated dedicated bearers creationOrest Voznyy3-0/+1009
Adding related pod file, session profile and testcase file. JIRA: YARDSTICK-1355 Change-Id: Ic97d59ef3a36ac05c6c369c36f917ae95d18d863 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-15Add UE initiated dedicated bearer creation testOrest Voznyy2-0/+427
Adding session profile and test case file. JIRA: YARDSTICK-1353 Change-Id: Iaa98fd8cb3ed243f9ac157089de32fa23ec0e6b4 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-15Add vEPC infrastructure for Landslide TGOrest Voznyy7-0/+360
1. Add yardstick topology and model for fully emulated environment. 2. Add support for black-box testing of real vEPC VNF vs Spirent Landslide traffic generator: - vEPC VNF pod file - topology and model files aware of vEPC VNF connection to Spirent Landslide TG - stub class for vEPC VNF, and related unit tests JIRA: YARDSTICK-1424 Change-Id: If7b6d19919679347c4360f4a0f2e420716e7fd2d Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-15Add vEPC default bearers relocation test caseOrest Voznyy2-0/+239
This test case runs on Spirent Landslide TG. All vEPC entities are emulated on Landslide test servers. Test case actions sequence: - create default bearers - start sending traffic - partial/full bearers relocation between two ENodeB's, Intra-MME - disconnect all bearers JIRA: YARDSTICK-1354 Change-Id: I95c7b0001ddb363402013d83ae8617b88b2a4bd9 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-15Add vEPC default bearers create/delete test caseOrest Voznyy5-0/+541
This test case runs on Spirent Landslide TG. All vEPC entities are emulated on Landslide test server. Test run configuration is performed in following files: 1. test case file 2. pod file 3. session profile file (landslide_session_default_bearer*.yaml) The structure of these files represents configuration of different parts of test cases in this test session. The order of test cases should always match. Spirent Landslide test types involved: "SGW_Node", "SGW_Nodal". Test case actions sequence: - create default bearers (as per value of "Sessions" key) with specified rate (session profile key "StartRate") - send traffic flow(s) (pre-configured in traffic profile) for a number of seconds (specified in session profile key "duration") - remove N default bearers with specified rate (session profile key "DisconnectRate") Parameters with ALL_CAPS values are placeholders. User should replace them with their own condifuration. Session profile file placeholders do not need to be replaced as long as these parameters are overriden in other files (e.g. pod, test case). Test results are dispatched according to runner configuration options. For fully emulated test runs like this one, best suited runner is "Search" type. In case of testing against real EPC VNFs, user could prefer different runner type. JIRA: YARDSTICK-1348 Change-Id: I5caf4878fb212b38d4148a297c5dba03b8ab98a8 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-15Add Spirent Landslide traffic profile templatesOrest Voznyy4-0/+430
These files are templates of Spirent Landslide data message flow (DMF) representing traffic profile of one or multiple traffic flows. Template files should not be edited directly. Instead, their parameters could be overriden in test case YAML file, in section "dmf". JIRA: YARDSTICK-1359 Change-Id: I4980a784a78ca870c18457227248e6e2ac0b8eda Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-14Fix Yardstick Docker image ARM buildPatrice Buriez1-1/+1
Soften sed pattern, in order to cope with new contents of ansible/install.yaml, recently modified by change 61173. Previous sed pattern does not work anymore against this changed line: https://gerrit.opnfv.org/gerrit/#/c/61173/4/ansible/install.yaml@39 JIRA: YARDSTICK-1411 Change-Id: I1e1ebd454900252c29263e746470e18e3139e016 Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-09-13Merge "Cleanup unittests for test_lmbench"Abhijit Sinha1-29/+27
2018-09-13Assign IXIA ports in one call instead of one by oneMytnyk, Volodymyr2-6/+11
Assigning IXIA ports one by one takes more time than assign all ports by once. Also, assigning ports in one call resolves the assign port issues caused on some IXIA chassis versions. JIRA: YARDSTICK-1425 Change-Id: Ibb08e40ee1a8762573f6752bdd1f583c5a1d9d2a Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com> Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-09-13Update LSResourceHelper unittestsEmma Foley2-11/+201
New unit tests have been added to make sure the right commands are sent to Spirent LS. This helps to ensure that the correct commands are sent to Spirent, so that potentially breaking changes can be flagged. JIRA: YARDSTICK-1356 Change-Id: I23f452355afcec2705da7d4665a34b178e2ba729 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-09-13Merge "Add UT: ArithmeticRunner exception handling"Emma Foley1-1/+227
2018-09-13IXIA traffic mac address is set incorrectlyMytnyk, Volodymyr1-2/+2
IXIA API sets the src/dst mac of the traffic always into default value. The `srcmac`/`dstmac` is never set in `traffic_param` map [1]. Instead the src/dst mac is stored in traffic_param[`outer_l2`]. [1] https://gerrit.opnfv.org/gerrit/gitweb?p=yardstick.git;a=blob;f=yardstick/network_services/traffic_profile/ixia_rfc2544.py;h=44bf2eafc7f8cf03ebd2c8520d316358fe1e5fc2;hb=HEAD#l92 JIRA: YARDSTICK-1423 Change-Id: Iabab7a54404b284d3b95bbea6c8624233c138351 Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-09-13Merge "Update configure_uwsgi run in baremetal/container"Rex Lee2-1/+9
2018-09-13Merge "Revert "Bugfix: ansible version should be the same both in Dockerfile ↵Rex Lee2-2/+2
and requirements.txt""
2018-09-13Merge "Create a sample fio test case base on kubernetes"Rex Lee1-2/+28
2018-09-13Merge "Create a sample lmbench test case base on kubernetes"Rex Lee1-5/+28
2018-09-13Merge "tc046 kill keystone process condition incorrect - dovetail"Rex Lee1-2/+2
2018-09-12Add Spirent Landslide TG APIEmma Foley2-40/+547
New TG class "LandslideTrafficGen" represents an interface to use Spirent Landslide API to configure and execute vEPC test cases on hardware/software Spirent environment. For that purpose this class and its helper classes use Spirent Landslide REST API calls and Landslide TCL API calls via library module lsapi.py. Change-Id: Ib6560d5cd2483c6c9f5c95568345ac39bfebbd4d Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-11Update configure_uwsgi run in baremetal/containerStepan Andrushko2-1/+9
Now "daemonize" prevents from installing Yardstick on baremetal. Configuring: "daemonize" in case "container" installation mode; "logto" in case "baremetal" installation mode; JIRA: YARDSTICK-1395 Change-Id: Ie390464db9c0816583edc711ed6526629d091bd3 Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com>
2018-09-11Merge "Add IxNextgen API for creating PPPoX configurations"Abhijit Sinha2-0/+218
2018-09-11Create a sample fio test case base on kubernetesrexlee87761-2/+28
make samples/fio.yaml to support both Openstack and K8s with context_type specified. It defaultly use Heat context, if for k8s, yardstick -d task start samples/fio.yaml --task-args '{"context_type": "Kubernetes"}' JIRA: YARDSTICK-1309 Change-Id: I2b0d61cf2f197ec8c60bda0ab4ff51594709f482 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-09-11Merge "Add container pull option to install.yaml"Rodolfo Alonso Hernandez3-13/+50
2018-09-11Create a sample lmbench test case base on kubernetesrexlee87761-5/+28
make samples/lmbench.yaml to support both Openstack and K8s with context_type specified. It defaultly use Heat context, if for k8s, yardstick -d task start samples/lmbench.yaml --task-args '{"context_type": "Kubernetes"}' JIRA: YARDSTICK-1310 Change-Id: Ifa542e830fb5ab7823bd4f994ab5d7a8e8a5b415 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-09-11Merge "Add k8s scenario test suite in Gambia"Rex Lee5-0/+90
2018-09-10Revert "Bugfix: ansible version should be the same both in Dockerfile and ↵Rex Lee2-2/+2
requirements.txt" This reverts commit 662de49b9f6c116d15882aed9e678cd3cf67494b. JIRA: YARDSTICK-1411 Change-Id: Id277aa8711c9ff77e085d26c90546eecc3490535 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-09-10tc046 kill keystone process condition incorrect - dovetailrexlee87761-2/+2
Change-Id: If01abfd5a99b4cea74f835e258f3a982af9446cd Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-09-08Added Landslide Resource Helper implementationEmma Foley3-38/+1218
Class "LandslideResourceHelper" provides API for operations needed - to configure Landslide test session - manage test session execution (start/stop/abort) - collect measurements during test run This helper class API is responsible to for configure Landslide test runs: - create test user - create test servers (for emulation of specific vEPC blocks) - create SUTs (actual tested VNFs performing specific vEPC roles) - create test session (contains actual test cases) - create DMFs (pre-defined traffic flows in traffic profile) - operate traffic run execution (start, stop, abort) - monitor test run status - collect KPIs on TG side Some of these features use Landslide REST API. Other ones use Landslide TCL API. JIRA: YARDSTICK-1356 Change-Id: I8fc8a7d85301121da465d054b8d38ae09a541c36 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-07Add IxNextgen API for creating PPPoX configurationsSerhiy Pshyk2-0/+218
New IxNextgen API allows to perform the following configurations on IxNetwork: - add topology - add device group to topology - add ethernet protocol layer to device group - add ipv4 protocol layer to device group - add pppox protocol layer to device group JIRA: YARDSTICK-1408 Change-Id: Icf5bb5f24784cf2c167c515b04c81471ac425e38 Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
2018-09-07Add k8s scenario test suite in Gambiarexlee87765-0/+90
JIRA: YARDSTICK-1410 Change-Id: I23b4c0bcfa9aa431e259930a41d4171ff5b4e8a1 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-09-05Add TclClients for Spirent Landslide TGOrest Voznyy8-0/+1438
Add class "LandslideTclClient" which holds TCL calls wrappers for operations on the LandSlide TG. It also introduces several new parameters in - test case YAML file structure: * session_profile: YAML file holding a template of Landslide test session for specific vEPC test scenario * dmf: test case options member holding traffic flow specific parameters for designated test scenario, that override ones in traffic_profile YAML file * test_cases: test case options member holding parameters specific to designated vEPC test case, overriding those in session_profile template file - pod YAML file structure: * tas_manager: holds parameters used to connect to Spirent Landslide TAS * config: list of entries describing Landslide test servers access, their network interfaces configuration, information of vEPC elements emulated by specific test server Class "LandslideProfile" implements access to traffic profile template from "LandslideResourceHelper" class. JIRA: YARDSTICK-1356 Change-Id: Ia6428072b1e72025be17c94e5141bd23f7993247 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-04Fix CLI argument handling in nsb_setup.shPatrice Buriez1-10/+67
CLI argument $1 was used for both admin-openrc file with OpenStack credentials and Yardstick Docker image, so obviously one of them would be wrong, and NSB setup would fail at some point. Historically, admin-openrc could be supplied as CLI argument $1, but change https://gerrit.opnfv.org/gerrit/56493 broke it, which means specifying Yardstick Docker image as CLI argument never worked. This fix implements support for the following CLI options: -o openrc Specify admin-openrc file with OpenStack credentials -i image Specify Yardstick Docker image -h Provide usage information For backward compatibility reasons, admin-openrc file can be supplied either with the -o option, or as a CLI argument. JIRA: YARDSTICK-1137 Change-Id: I3e9904970560468c24b21738203bc67b7591f9e4 Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-09-04Merge "Merge latest changes to Dashboard from L2FWD to L3FWD dashboard"Abhijit Sinha2-12735/+12491
2018-09-04Merge "NSB NFVi PROX Missing VNF Statistics for CPU Utilization"Abhijit Sinha38-7/+155