summaryrefslogtreecommitdiffstats
path: root/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_machine.py
AgeCommit message (Collapse)AuthorFilesLines
2023-06-26Addition of new test case encrypt.testLuc Provoost1-0/+8
The new test is combining a generator sending plain text traffic to another pod. This second pod is encrypting and then decrypting the packets again before sending them back to the generator for latency measurements. The encryption/decryption is done with a hard code algo and key and can be done in SW or by using QAT devices. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: If85766e546b0c3534d2eaed96a465195c1455d06
2023-06-26Adding QAT supportLuc Provoost1-0/+10
When qat devices are assigned to a pod, the code will now detect that through the qat entry in the env of the pod. The qat devices will be added to the PDK eal command line so they can by PROX. For the PCI network devices, we only search for PCIDEVICE instead of PCIDEVICE_INTEL_COM. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Ic801d20adac4a29e6c44e542121686f42778d61a
2023-06-26Add uuid to DPDK file-prefixLuc Provoost1-2/+3
When running multiple pods, the DPDK file-prefix needs to be unique, so a uuid was added to the file-prefix. This parameter is then available in the parameters.lua file. The eal parameter for setting the simd bitwidth was also added. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Iee04e54f059732a40ebdc7635dbd09631b5f2081
2023-02-01Use proper DPDK EAL parameter name: --allowLuc Provoost1-1/+1
The name of one of the EAL parameters has changed. Depending on the DPDK version, we need to use a different name. While building the PROX image, we need now to create a file /opt/rapid/dpdk_version that contains the DPDK version. When running runrapid.py, we will now check this file to decide if we use the old --whitelist, or the new --allow parameter name for the PCI address to be used. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I3e07c552d014c818138d389abce936e275108776
2023-01-17Allow a different pod spec for each podLuc Provoost1-18/+14
A new parameter is now introduced in the [PODx] sections of the rapid.pods file. This parameter specifies the pod spec yaml file that will be used to create this specif pod. When this parameter is not available, the default pod-rapid.yaml file will be used. The parameter name is spec_file_name. Also cleaned some lines of code exceeding 80 characters. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Id26881517d45baf4bf6be2e82b4f9e049f2b2547
2023-01-09Optional MAC address and other changesLuc Provoost1-11/+42
The MAC address of the dataplane interfaces in the environment file is now optional. If not specified, the HW MAC address of that interface will be used. When using l2 tests, the parameter will be needed anyway and the test will fail if the MAC address is not specified. Starting from DPDK 20.11, the EAL --pci-whitelist parameter changed into --allow. We are now checking the DPDK version of the PROX instance to determine which parameter to use. For pods using cgroup v2, we are now using a different file to determine the cpuset available to the pod. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I7a88458406b9ae3c8cae9c583cc37121d40c5073
2022-09-15Removing the --socket-mem eal parameterLuc Provoost1-25/+1
This parameter is not needed any more for a successful run on a Kubernetes environment. I had to remove the parameter since this was not working anymore on the K8s deployments I am currently using. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I784611d66e003f1a6e344b2d9c7a2c06c5c2b114
2021-09-06Changes from ViNePerfSridhar K. N. Rao1-12/+12
This patch adds changes to support using rapid to run tests only. 1. Fix format.yaml path. Add format.yaml as datafile in setup.cfg 2. Do not call generate lua if starting prox is not required. 3. Move logs to results directory - if present 4. Machines should not be used in del, if not created. Fix syntax error. Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com> Change-Id: I580716573d0be2a8da02035f4d180e81b63b3fc8
2021-07-16Using paramiko for all ssh & scpLuc Provoost1-17/+23
By using paramiko, we can now also use differnt credentials to login into the PROX instances. In the ssh section of the environment file, you can now also add the password parameter. Change-Id: I08d31cbf0d02d7e82b7fbe34268851f73ff5dde0 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-06-21Add support for multisocket memory reservationYury Kylulin1-12/+36
Automatic detection of allowed NUMA nodes for memory allocation added to Kubernetes environment. Signed-off-by: Yury Kylulin <yury.kylulin@intel.com> Change-Id: I4f8b2f64971d43414bbd290f7fcbc6210c1d6d71
2021-05-05New test case: gen versus swap via gatewayLuc Provoost1-4/+11
A new test was added: tests/basicrapid_gw.test. Traffic generated will now go to the reflector (swap instance) via the Gateway. The swap instance will also send the traffic back via the gateway. This use case is created to support GCP testing where the interfaces are defined with a /32 netmask, and hence all traffic is routed via the gateway. Change-Id: I60142d650072ab345cd47a0f32cb6a8275da0542 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-02-25Preparation for packet mis-ordering statsLuc Provoost1-7/+7
Changed PROX and the python scripts so that the "lat all stats" command also returns the mis_ordered, extent and duplicate statistics. In order to make this work properly, we need now to also stop and restart the latency cores, every time we stop and start the generator cores, since the sequence numbers are reinitialised every time we start the generator core. Change-Id: Ic4212a989c8ee9d0f026a34ca63a08b982c32b71 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-02-11Add support for native Kubernetes CPU Manager0.0.1Yury Kylulin1-0/+62
For Kubernetes environment core ids in the config and test files are relative and automatically remapped to the allowed cores allocated for container. There is no change to the baremetal or VM environment. Signed-off-by: Yury Kylulin <yury.kylulin@intel.com> Change-Id: I63e499723e8213de1b05d4175eb3eddc4492ccf5
2021-01-27creation of tests and configs subdirectoriesLuc Provoost1-6/+7
All test files are now in the tests directory. All PROX config files moved to the configs directory. When specifying a test on the runrapid.py command line, you should now also mention the directory: e.g. --test tests/basicrapid.test In these test files, you need now to also include the configs subdirectory when specifying the prox configuration that needs to be used. The --map parameter has also changed. It can still take the file name of the map file, that specifies which PROX instance to use during a test. On top of that, you can now also specify a list of indices of the machines to be used during a test: e.g. --map [1,3]. Change-Id: I887cba59834dd8c903a22bc80959b10da2775398 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-12-28Improved PROX cleanup when exitingLuc Provoost1-4/+6
Sending now a 'quit' command to PROX, so that the thread running PROX in an ssh session is now cleaned up correctly. Also fixed a problem with the logging when running the code more than once. We only create handlers now, when they do not yet exist. Updated the rapid version. Also added some more tests in the testcases.yaml for xtesting. At the end of a run, the PROX.log files are copied in the results directory. The success criterium for a test 'pass_threshold' has been removed from the test files and is to be controlled by the xtesting testcases.yaml file. Change-Id: Ifbbb1c91f32c9176f52025d9ae4c495b432a94c9 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-10-19Support dataplane subnet mask & latency histogramLuc Provoost1-8/+9
There is a new parameter in the config_file: dataplane_subnet_mask. This parameter is added to the dataplane IP addresses and is needed by the Linux stack to deal with some protocols (e.g. ARP). If not present, the default value of 24 will be used. In the end, this will make sure that IP addresses in the rapid.env file will have this mask, e.g. /24 If you use other tools than createrapid.py to create the rapid.env file, make sure to add the mask. This commit also introduces a change when exiting: we only copy the prox.log files from the different prox instances in case we expect a new prox.log file to be created in these instances. When using a valid format.yaml file, the tool will now also send the latency distribution data (histogram) to the URL. The tool is now also checking the value of bucket_size_exp: it needs to be 11 or more. Change-Id: I633cdc64ef687fdb6625be1e7482a5a371f83e93 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-09-29New PROX version, background traffic reportingLuc Provoost1-19/+24
deploycentostools.sh is now installing a recent PROX version. When using background traffic, the tool is also reporting the total traffic that was handled by the NFVi stack, including foreground & background traffic. The use of the prox_socket and prox_launch_exit was also broken in a previous release and is now fixed. When setting prox_socket to false for a PROX instance, we will not create a socket connection to control that PROX instance. Setting this option to false is not meaningful for the generator PROX instances since we would not be able to control the behavior of the PROX generator during the test. You could use the generator against another non-PROX instance and then set prox_socket to false for that non-PROX instance. Default is true. prox_launch_exit is used to start and stop the PROX program inside the instance. Default is true. This is useful in case you want to start PROX manually to be able to inspect the PROX UI, but at the same time, let the rapid scripts control the testing. Change-Id: Ib5aa809f4be201859542769f5f55f4989dad97ef Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-09-17Using python concurrent futuresLuc Provoost1-8/+16
Different PROX instances are now started in parallel. The script is starting multiple threads. Change-Id: Ia8785a792240d4e9b5d5d98174bc4c5d7ae5657c Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-09-16ramp generator traffic & prox.log copyLuc Provoost1-0/+3
A new parameter can be put in the *.test files so that the requested traffic is not generated at once, but slowly increasing up to the requested traffic. This is important for some SW switched that adapt and learn from the applied traffic. When suddenly applying a high traffic, the switch might not have enough time to learn and lose packets. Only after a period of time (up to 10 seconds?), the switch can cope with the traffic without packet loss. In order to get reproducible results in this case with low packet loss, please put the parameter ramp_step in the test section of the *.test file. The step is expressed in percentage of a 10Gb/s traffic. If the parameter is not present, the load will be immediately applied. When the runrapid.py script is now finished, it also copies the prox.log files from all the prox instances. In case of issues, please investigate this log file. When using background traffic, the tool will also print the average background traffic for each step in the DEBUG log. In this way, you can check that the background traffic is actually dealing with the same amount of traffic as the foreground. The tool is printing the packet reception rate of the background generators. A bug was also fixed regarding the latency percentile reporting. When using tests involving l2gen.cfg, the values of the percentile latency were wrong. Change-Id: Id3dd02ed5cab084414c717cf898b30e6980ddf31 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-08-06fix: IPV6 packet generation, packet loss reportingLuc Provoost1-1/+1
UDP and IP packet length and source UDP port and destination UDP port were wrongly set for IPv6 packets. In the *.test files, there is now an optional boolean field (ipv6) in the [TestParameters] section. When set to true, the generator will use the IPv6 packet layout to place the right values at the proper offset in the packet. If not present or set to false, the IPv4 layout will be used. Note also that packet size needs to be at least 84 bytes for IPV6 traffic since we need space for latency and other information in each packet. Packet loss ratio is now printed as a percentage of the total packets sent. Change-Id: I2136e87f3032348ae95b69052be7da8461c6303c Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-07-05Adding vdev support and vfioLuc Provoost1-1/+1
Using vfio in stead of igb_uio. DPDK 2020.5 is NOT compiling the igb_uio driver by default any more. For the l3 mode, we are now using the new PROX vdev feature. This is achieved by adding the vdev parameter in the port section of the PROX config files. One of the advantages is that we can now ping a port managed by PROX: the requests will be forwarded by PROX to the LINUX stack, who will reply to the ping. A tap device is being created by PROX to support this feature. Installing old version of nasm since latest causes some issues. If not pushgateway is being used, it should be commented out in the config_file. vfio is now loaded by specifying the module in /etc/modules-load.d There is also an after_boot.sh script that will be run by check_prox_system_setup.sh, which can be used for further configuration of the instance after boot. There is now also a file created /opt/rapid/system_ready_for_rapid by the check_prox_system_setup.sh script. This file is created once all configuration is done. The runrapid.py script will wait till this file is created before executing any tests. Change-Id: Ic5c41af82642066af42134c3323297f5a06f6f72 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-05-29Fixing some more copyright datesLuc Provoost1-1/+1
New files created in 2020 have now the right date in the copyright notice. Change-Id: I8eb244beec3d793e819cce1f63c39671f18286a1 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-05-28IPV6 testing & minor changes and fixesLuc Provoost1-8/+4
IPV6 test has been added. The generator will now generate IPV6 packets using NDP to resolve MAC addresses. The hex IP addresses in the inline definition of the packets in the cfg files is now calculated by a lua function called in parameters.lua. The hard-coded path /home/centos has been replaced by ~. An error in the speed calculations has been fixed. See the RapidTest class for details. The actual number of generated flows is now reported. If the requested number of flows is not a power of 2, the lowest power of 2 that is larger than the requested number of flows will be used and reported. Change-Id: I36b8d345b4a03c2cf358c19ec353ca8ad2831293 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-05-15Code rewrite and Python3 supportLuc Provoost1-0/+122
The code has been rewritten in multiple classes and files for better readability and maintainability. You should now also use Python3 to run the scripts. Some of the variables in the config files have moved to other sections. The pushgateway parameter moved to the [varia] section of the rapid environment file. Change-Id: Ic61a1bc3baa39447d42b21dfcc0f16c28fcdfa6e Signed-off-by: Luc Provoost <luc.provoost@intel.com>