aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/network_services/vnf_generic
AgeCommit message (Collapse)AuthorFilesLines
2018-10-05Update Ixia tg to use new API for traffic model creationSerhiy Pshyk1-1/+4
JIRA: YARDSTICK-1464 Change-Id: Ibbced9e0e04e077ec5c23fc15e97308dd11b7004 Signed-off-by: Serhiy Pshyk <serhiyx.pshyk@intel.com>
2018-10-02NSB PROX NFVi Test does not stop after reaching expected precisionDanielMartinBuckley1-1/+1
JIRA: YARDSTICK-1419 When using prox_binsearch algorithm, a binary search is performed, increasing lower bound when step was successful and decreasing upper bound when the step was a failure. This runs until the test_precision (as specified in the traffic profile) is reached. When the test precision is reached, the test is not completed until the runner duration is reached. As runner duration is usually high (e.g. 1800sec), the tests take much too long to execute. This makes it difficult to create test suites. Change-Id: I6cc503a09fb534a556c61c805e6df4786bb8cc4b Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-09-21Merge "[NSB] Adds support for running "black box" test cases for VNFs"Abhijit Sinha1-0/+46
2018-09-21[NSB] Adds support for running "black box" test cases for VNFsChornyi, TarasX1-0/+46
Using AgnosticVNF is now possible to create a "black box" testcase for any VNF. Example rfc2544 test with TRex as trafficgen samples/vnf_samples/nsut/agnostic/tc_baremetal_rfc2544_ipv4_64B_trex.yaml and Ixia samples/vnf_samples/nsut/agnostic/tc_baremetal_rfc2544_latency_ipv4_64B_ixia.yaml NOTE: Yardstick will not interact with VNF. No KPIs from VNF will be collected JIRA: YARDSTICK-1438 Change-Id: I408b3dcb90f29c7b41fcf8f883b5abfc4e9c3dbc Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
2018-09-15Add vEPC infrastructure for Landslide TGOrest Voznyy1-0/+53
1. Add yardstick topology and model for fully emulated environment. 2. Add support for black-box testing of real vEPC VNF vs Spirent Landslide traffic generator: - vEPC VNF pod file - topology and model files aware of vEPC VNF connection to Spirent Landslide TG - stub class for vEPC VNF, and related unit tests JIRA: YARDSTICK-1424 Change-Id: If7b6d19919679347c4360f4a0f2e420716e7fd2d Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-13Update LSResourceHelper unittestsEmma Foley1-1/+1
New unit tests have been added to make sure the right commands are sent to Spirent LS. This helps to ensure that the correct commands are sent to Spirent, so that potentially breaking changes can be flagged. JIRA: YARDSTICK-1356 Change-Id: I23f452355afcec2705da7d4665a34b178e2ba729 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-09-12Add Spirent Landslide TG APIEmma Foley1-3/+179
New TG class "LandslideTrafficGen" represents an interface to use Spirent Landslide API to configure and execute vEPC test cases on hardware/software Spirent environment. For that purpose this class and its helper classes use Spirent Landslide REST API calls and Landslide TCL API calls via library module lsapi.py. Change-Id: Ib6560d5cd2483c6c9f5c95568345ac39bfebbd4d Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-08Added Landslide Resource Helper implementationEmma Foley1-5/+511
Class "LandslideResourceHelper" provides API for operations needed - to configure Landslide test session - manage test session execution (start/stop/abort) - collect measurements during test run This helper class API is responsible to for configure Landslide test runs: - create test user - create test servers (for emulation of specific vEPC blocks) - create SUTs (actual tested VNFs performing specific vEPC roles) - create test session (contains actual test cases) - create DMFs (pre-defined traffic flows in traffic profile) - operate traffic run execution (start, stop, abort) - monitor test run status - collect KPIs on TG side Some of these features use Landslide REST API. Other ones use Landslide TCL API. JIRA: YARDSTICK-1356 Change-Id: I8fc8a7d85301121da465d054b8d38ae09a541c36 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-09-05Add TclClients for Spirent Landslide TGOrest Voznyy1-0/+521
Add class "LandslideTclClient" which holds TCL calls wrappers for operations on the LandSlide TG. It also introduces several new parameters in - test case YAML file structure: * session_profile: YAML file holding a template of Landslide test session for specific vEPC test scenario * dmf: test case options member holding traffic flow specific parameters for designated test scenario, that override ones in traffic_profile YAML file * test_cases: test case options member holding parameters specific to designated vEPC test case, overriding those in session_profile template file - pod YAML file structure: * tas_manager: holds parameters used to connect to Spirent Landslide TAS * config: list of entries describing Landslide test servers access, their network interfaces configuration, information of vEPC elements emulated by specific test server Class "LandslideProfile" implements access to traffic profile template from "LandslideResourceHelper" class. JIRA: YARDSTICK-1356 Change-Id: Ia6428072b1e72025be17c94e5141bd23f7993247 Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Orest Voznyy <orestx.voznyy@intel.com>
2018-08-23Merge "Runners crash on test startup."Abhijit Sinha1-3/+4
2018-08-23Runners crash on test startup.DanielMartinBuckley1-3/+4
JIRA: YARDSTICK-1393 The problem resides in this traffic generator class: this class is inheriting from SampleVNFTrafficGen and overriding the __init__ method but is NOT calling the parent class __init__ one. Change-Id: I206a66e361a3e2eb50bb5fa01ddadae25b4a9f54 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-08-21Merge "Improvie TRex RFC2544 throughput calculation"Rodolfo Alonso Hernandez1-1/+3
2018-08-21Convert IXIA latency statistics to integerRodolfo Alonso Hernandez1-3/+3
JIRA: YARDSTICK-1385 Change-Id: Id50c393da7ded4b3c8e127f7d7a6501832a68446 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-08-20Improvie TRex RFC2544 throughput calculationRodolfo Alonso Hernandez1-1/+3
Improve TRex RFC2544 throughput calculation by using the in/out packets instead of using the throughput samples. JIRA: YARDSTICK-1387 Change-Id: I877e901c3226d31c6435a877a2924bb95deb1cf0 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-08-20Fix RFC2544 IXIA latency informationRodolfo Alonso Hernandez1-16/+15
Do not overwrite the samples information with the latency parameters. JIRA: YARDSTICK-1385 Change-Id: I2f144800ad19d7736eedf5c7b6783731031c4bf4 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-08-17Merge "Improve IXIA TG Rx/TX throughput calculation"Rodolfo Alonso Hernandez1-2/+2
2018-08-17Add RFC2544 IXIA latency informationRodolfo Alonso Hernandez1-9/+8
JIRA: YARDSTICK-1385 Change-Id: Icecceaabc0389279a3a7154884c50259f794eb76 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-08-16Merge "IXIA traffic stops running after first iteration"Emma Foley1-11/+7
2018-08-16Improve IXIA TG Rx/TX throughput calculationMytnyk, Volodymyr1-2/+2
Use float instead of integer to convert TG throughtput IXIA statistic. JIRA: YARDSTICK-1378 Change-Id: I39bd143adc4b10035c6f0986a06d407a18aa0887 Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-08-14Fix TRex RFC2544 traffic profile tolerance definitionRodolfo Alonso Hernandez2-2/+4
In TRex RFC2544 traffic profile, the tolerance limit is not set correctly. The parameters "tol_high" and "tol_low" in "get_drop_percentage" are incorrect. The tolerance limit should be reduced to 0.01. JIRA: YARDSTICK-1382 Change-Id: If5fc1f3aec86effabd7903e0924724fe7eeb42ab Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-08-10IXIA traffic stops running after first iterationMytnyk, Volodymyr1-11/+7
IXIA traffic stops running after first iteration when traffic profile duration is set to 60 sec. JIRA: YARDSTICK-1374 Change-Id: I216b98d43b66aabc72ed88e895b89e28572b264f Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com> Signed-off-by: Myron Sosyak <myronx.sosyak@intel.com>
2018-07-25Merge "Fix typo in classname AclUknownActionTemplate"Rodolfo Alonso Hernandez1-1/+1
2018-07-24Fix typo in classname AclUknownActionTemplateEmma Foley1-1/+1
AclUknownActionTemplate -> AclUnknownActionTemplate JIRA: YARDSTICK-1334 Change-Id: I796e080078c657c5e0e59144d24ec797e244a8f7 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-07-23Add DPDK pktgen traffic generatorRodolfo Alonso Hernandez1-0/+103
Add DPDK pktgen traffic generator. This traffic generator is designed only to work in with a MQ aware runner. Implements two consumer methods: - runner_method_start_iteration - runner_method_stop_iteration "run_traffic" method will only initialize the traffic profile object. This traffic generator uses a socket port (LUA port, default 22022) to send the command messages. Link: http://pktgen-dpdk.readthedocs.io/en/latest/index.html JIRA: YARDSTICK-1344 Change-Id: I89bba0f462fa1c22b33f1253f67f7c41e6e721a5 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-13Create a SampleVNF MQ consumer classRodolfo Alonso Hernandez15-54/+96
This new class is designed to be used in a ``GenericVNF`` object (VNF or traffic generator) in a builder pattern. This consumer class will be instantiated when initializing the ``GenericVNF`` derived object. The ``GenericVNFConsumer`` endpoint is the ``GenericVNF``derived object itself. ``GenericVNFEndpoint`` is an interface class to be inherited in a ``GenericVNF`` child object; the VNF object will implement the endpoints methods. NewVNF(GenericVNF, GenericVNFEndpoint): def __init__(self, name, vnfd, task_id): ... self._consumer = vnf_base.GenericVNFConsumer([task_id], self) self._consumer.start_rpc_server() JIRA: YARDSTICK-1293 Change-Id: I1bcd980352099e9ebefdab0c96d51aa46f6a1e54 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-13Merge "Cleanup TG IxLoad test cases"Rodolfo Alonso Hernandez1-5/+4
2018-07-12Fix "id" retrieval from MQ producer in GenericTrafficGenRodolfo Alonso Hernandez1-1/+1
"GenericTrafficGen.get_mq_producer_id" uses a wrong method to retrieve the MQ producer ID. "MessagingProducer" ID is a property called "id". JIRA: YARDSTICK-1312 Change-Id: Id72836b1cde59059359907bd3a315a1abb55778e Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-12Cleanup TG IxLoad test casesRodolfo Alonso Hernandez1-5/+4
Refactor SSH handling and library mocks. JIRA: YARDSTICK-1236 Change-Id: I591601f8ea8bf4bbd5c43300ff29d7930af5c3c6 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-11Revert "vPE Sample VNF is missing in the installation scripts"Volodymyr Mytnyk1-2/+2
SampleVNF source tree doesn't contain vPE VNF sources, thus the installation of the vPE sampleVNF fails because of the changes. Reverting. This reverts commit 5bb855a82d960a4fb796e801ec9a9a22dc40b1ca. Change-Id: I27e929e813a28847c2ca725031ec767633a36954 Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-07-09Merge changes from topics 'YARDSTICK-1286', 'YARDSTICK-1127'Rodolfo Alonso Hernandez5-13/+71
* changes: Make "IterationIPC" MQ producer for VNF control messages Enable traffic generator PID in "NSPerf" scenario setup Enable the MQ producer in "SampleVNFTrafficGen" class Add "TrafficGeneratorProducer" for "GenericTrafficGen" instances Add "IterationIPC" runner
2018-07-05Merge "vPE Sample VNF is missing in the installation scripts"Rodolfo Alonso Hernandez1-2/+2
2018-07-05Enable traffic generator PID in "NSPerf" scenario setupRodolfo Alonso Hernandez2-11/+15
Now "NSPerf" scenario will inform about the traffic generator PIDs after setup process. With this information, IterationIPC runner will be able to receive the messages sent by those traffic generators and control the main iteration loop. The following example, using vFW as VNF and OpenStack as context, makes use of this new runner implementation: /samples/vnf_samples/nsut/vfw/ tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_iterationipc.yaml JIRA: YARDSTICK-1127 Change-Id: I46b1368bc209680b88ff9fb5c3b9beadf6271ac9 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-05Enable the MQ producer in "SampleVNFTrafficGen" classRodolfo Alonso Hernandez5-11/+44
Now all traffic generators using the default implementation of "ClientResourceHelper.run_traffic" will update the status of the traffic injection using the MQ. The available methods are listed in common.messsaging (VNF_METHOD_*): - tg_method_started: VNF app started - tg_method_finished: VNF app finished - tg_method_iteration: VNF app execution loop started JIRA: YARDSTICK-1127 Change-Id: I12829e1762fc20cc95da3b50767a66f031e25ee8 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-05Add "TrafficGeneratorProducer" for "GenericTrafficGen" instancesRodolfo Alonso Hernandez1-0/+21
Added "TrafficGeneratorProducer" class, a message producer specific for "GenericTrafficGen" derived classes. JIRA: YARDSTICK-1127 Change-Id: Icc87a6920155612e759a1d4d2f29028940c2e040 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-29Cleanup TestIxLoadTrafficGen unit testsRodolfo Alonso Hernandez1-4/+4
Removed unneeded output and mock system calls. JIRA: YARDSTICK-1270 Change-Id: I0e04a67cab592216280481c7145188c4f9045466 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-29NFVI support for standalone, baremetal and heat contextsChornyi, TarasX5-23/+70
JIRA: YARDSTICK-1257 Change-Id: I6733bd49ac91985e8f3d7722e6990e8733bb430e Signed-off-by: Chornyi, TarasX <tarasx.chornyi@intel.com>
2018-06-28Decrease Sampling interval - Invalid Port resultsDanielMartinBuckley1-4/+6
JIRA: YARDSTICK-1219 The computed in/fwd throughputs displayed on the L2FWD, L3FWD and VPE dashboards. The computed value is likely 4x the expected value. It is required to return sampling information form Generator AND VNF at least every 1 second. Change-Id: I4435fd05ba3116ead836843a4c2fce133b767a28 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-27Fix NSB NfVi support for 25 and 40GbpsXavier Simonart1-4/+27
When setting interface_speed option in the test case, NSB/PROX was still considering 100% as 10 Gbps. With this fix, when interface_speed is set to for instance 25 Gbps then NSB/PROX will start generate at 25 Gbps. Yardstick logging info printed on the screen will show performance result as percentage of interface_speed (e.g. 100% will be 25 Gbps). This is done by sending the command "speed 250" to PROX (PROX cannot know the underlying interface speed in all cases e.g. in virtio, and hence considers 100% as 10Gbps). JIRA: YARDSTICK-1217 Change-Id: I4989161b04a7afa5e347935ee7ccef7dfe5a504d Signed-off-by: Xavier Simonart <xavier.simonart@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-27Merge "Decrease Sampling interval"Abhijit Sinha2-23/+81
2018-06-26NSB NFVi BNG test fails to run - stops after one stepDanielMartinBuckley1-4/+16
JIRA: YARDSTICK-1244 When sending the start or stop command to PROX. We sould not send the same core. We should not send "start 1,1" instead we should send "start 1" Change-Id: Ie600dc3ed808fd00341e92f13bd52199e68dc51f Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-26Decrease Sampling intervalDanielMartinBuckley2-23/+81
JIRA: YARDSTICK-1219 Currently NSB for NFVI using PROX returns sampling information every 13 seconds. This is too slow. It is required to return sampling information form Generator AND VNF at least every 1 second. This change is depandant on JIRA: YARDSTICK-1212 and YARDSTICK-1220 Change-Id: Ica7ab795a2919d191d8cd846d028e15739e33fb7 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2018-06-25Change IxNextgen library loadRodolfo Alonso Hernandez1-15/+3
Changed IxNextgen library load in IXIA RFC2544 traffic generator call. JIRA: YARDSTICK-1116 Change-Id: Icbdc5f1255dd2676584f3a1be767287e55e71f4a Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-22Merge "Configure ACL via static file"Abhijit Sinha4-25/+216
2018-06-21Configure ACL via static fileMytnyk, Volodymyr4-25/+216
This patch allows user to configure ACL/vFW SampleVNF ACL via configuration file provided in TC definition. The Yardstick applies the rules to SampleVNF from specified config file + rules generated by Yardstick (default rules). The example of SampleVNF ACL CLI commands generated/applied by Yardstick can be found at (using default ACL config file): ACL VNF ACL CLI commands: acl/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml http://paste.openstack.org/show/723303/ vFW VNF ACL CLI commands: vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml http://paste.openstack.org/show/723304/ Change-Id: I76a630261a982083b628e3985fc3bec14ca495db Signed-off-by: Mytnyk, Volodymyr <volodymyrx.mytnyk@intel.com>
2018-06-18Refactor RFC2455 TRex traffic profile injectionRodolfo Alonso Hernandez3-93/+56
The TRex RFC2455 traffic profile configures the duration of the traffic injection. Once the traffic is started, the traffic profile should poll the traffic injector client for the injection status. When the injection is finished, the traffic execution should end. Instead of this, the traffic profile waits a fixed time (using time.sleep method) and then stops the traffic. This approach is not accurate and may cause shorter injection periods. JIRA: YARDSTICK-1197 Change-Id: I5090df81ef4ec7945ff6c1aff070656b48e0fe77 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-12Improve IXIA IxNetwork library and traffic profile (4)Rodolfo Alonso Hernandez1-29/+14
This patch implements an active wait for the traffic injection. Once the traffic is started, the traffic generator class will poll periodically the IXIA traffic generator chassis to retrieve the status of the traffic ("started", "stopped"). Now the latency statistics are retrieved and reported for each injection period. JIRA: YARDSTICK-1116 Change-Id: I4422e2c88b4fc97b7cac3de8a82b2d75467c4117 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-12Improve IXIA IxNetwork library and traffic profile (2)Rodolfo Alonso Hernandez1-13/+13
This patch modifies the way the packet frame and the flow is configured before the injection. "IxNextgen.update_frame" modifies the L2 frame according to the test case and setup the frame rate, frame size, traffic injection duration and MAC addresses. JIRA: YARDSTICK-1116 Change-Id: Ife08f15a4eda24d7835c92c4172b450854d112ee Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-12Improve IXIA IxNetwork library and traffic profile (1)Rodolfo Alonso Hernandez1-13/+10
This patch introduces a new way to configure the TCL IxNetwork server. All the configuration is done using the TCL API, removing the need of using the pre-saved configuration file. "IxNextgen.assign_ports" creates and assigns the virtual ports for each physical port defined in the test case. "IxNextgen.create_traffic_item" creates one traffic item and two flow groups per pair of ports, in both directions. "IxNextgen.gt_statistics" retrieves only the required statistics to generate the samples blob in the traffic generator. JIRA: YARDSTICK-1116 Change-Id: I8f1c0c55e99c274b2ed8276ed9a385c502e16d93 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-12Merge changes from topics 'YARDSTICK-1218', 'YARDSTICK-1216', ↵Rodolfo Alonso Hernandez1-4/+2
'YARDSTICK-1215', 'YARDSTICK-1214' * changes: Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions Move ErrorClass definition to exceptions module Convert SSH custom exceptions to Yardstick exceptions Remove AnsibleCommon class method mock
2018-06-11Move tests: unit/network_services/vnf_genericEmma Foley1-2/+4
* Fix pylint errors * Add TODOs Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged with a TODO, and should be fixed later. JIRA: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Change-Id: I3d95537a64bde923703421510d27bac0d6dd92bd