summaryrefslogtreecommitdiffstats
path: root/VNFs/DPPD-PROX/helper-scripts/rapid
AgeCommit message (Collapse)AuthorFilesLines
2024-11-25specify kubeconfig fileHEADmasterLuc Provoost2-3/+7
When running createrapidk8s.py, you can now specify the kubeconfig file to be used. Check the help option for this parameter. The parameter is optional. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Ic4b5fcb68cfda1cf39651600d7d40311ffe56c72
2024-11-25add current path for python modulesLuc Provoost15-38/+38
Added a dot in front of the python files that are imported, so the code can also be called from other directories Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I18d4925fcf68c27c91d5e489178bac79edfcf5f4
2024-11-12fix some default values in parsingLuc Provoost1-5/+3
'prox_socket' 'prox_launch_exit' and 'monitor' will be set to 'True' if not in the configration file for that specific test machine. The code is now cleaner. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I8f05a4eac884896139b2fb6deabbce4d3edfd2ae
2024-11-11Allow to collect powerLuc Provoost1-0/+4
In order to collect pwoer of the system under test, one can now define a power helper function that will retrieve System, CPU, and RAM power. This function will then be called at the end of a test iteration. The actual function to collect the power is not provided in this github. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Ic33881326f9318f1669fdef69a872596cc8e6dbc
2024-11-11New determination of the cpu setLuc Provoost1-11/+1
Check out https://github.com/kubernetes/kubernetes/issues/119669 That describes the problem. The solution can be found in this change. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Id4d47e5d36b1c4acefea8dbe13d7a1e7c499c70d
2024-11-10simplify core parameter parsingLuc Provoost3-32/+13
Simplified some of the code to deal with the differrent core options Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I4f33563b51370efe807e8ace807acc3fdcd2bfc4
2024-11-09fix namespace log errorLuc Provoost1-1/+1
Logging wrong parameter Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I7ca4b1b0eba93eadbd5289d51a2a3fb259af7938
2024-11-09Nodeport service and modified ssh and socket portsLuc Provoost6-21/+65
We support now the use of K8s Nodeport services and non standard ssh and prox socket ports. In order to use Nodeport service, you need to specify the name of the service for each prox pod in the rapid.pods file. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I2873b7e22fee042a14b575e772267a9128b7c4f2
2023-06-29Fix compilation for DPDK22.11Luc Provoost1-0/+4
ETH_LINK_SPEED_AUTONEG changed into RTE_ETH_LINK_SPEED_AUTONEG Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Iababab5ce51b618cd97f8158de2e2979747b141c
2023-06-26Addition of new test case encrypt.testLuc Provoost4-1/+127
The new test is combining a generator sending plain text traffic to another pod. This second pod is encrypting and then decrypting the packets again before sending them back to the generator for latency measurements. The encryption/decryption is done with a hard code algo and key and can be done in SW or by using QAT devices. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: If85766e546b0c3534d2eaed96a465195c1455d06
2023-06-26Adding QAT supportLuc Provoost4-1/+43
When qat devices are assigned to a pod, the code will now detect that through the qat entry in the env of the pod. The qat devices will be added to the PDK eal command line so they can by PROX. For the PCI network devices, we only search for PCIDEVICE instead of PCIDEVICE_INTEL_COM. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Ic801d20adac4a29e6c44e542121686f42778d61a
2023-06-26Add uuid to DPDK file-prefixLuc Provoost1-2/+3
When running multiple pods, the DPDK file-prefix needs to be unique, so a uuid was added to the file-prefix. This parameter is then available in the parameters.lua file. The eal parameter for setting the simd bitwidth was also added. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Iee04e54f059732a40ebdc7635dbd09631b5f2081
2023-02-07fix issue with fixed rate testingLuc Provoost1-0/+2
When running fixed rate tests, we are getting KeyError: 'mis_ordered'. This is now fixed. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I8df7f53a6d8c5d8d5b88215d543918ebea4d45a1
2023-02-07Start ssh properly for UbuntuLuc Provoost1-2/+5
The command to start ssh (on CentOS) seems not be working on Ubunutu. Now we start ssh with 'service ssh start' Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Ief8cd2e10443ffe99da1ed494d154edc6d8e175e
2023-02-02Use ubuntu for containersLuc Provoost2-16/+75
Changed the Dockerfile to install DPDK 22.07 and to run on Ubuntu 20.04. runrapid.py will now use the 'rapid' user to connect to the PROX instances. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I2df48f435e6d572343a02b2ab2c1db116c83ecad
2023-02-02fix machine name in DEBUG loggingLuc Provoost1-1/+1
We are logging the content of all test parameters in DEBUG mode. Since one of the test parameters (machine name) can still change to make sure we have unique machines names, the logging is done later, after the machines names are guaranteed to be unique. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I06cbdbcfd9044798ebad03ead0eca567e7f24a9d
2023-02-02Change pod names: refer to prox in stead of rapidLuc Provoost1-1/+1
In rapid_k8s_deployment.py, we are creating the pods. The name used to refer to rapid. Actually, the pods that we are creating are the prox instances. The rapid scripts need to run elsewhere. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Idcc842c4aa3a85f2232f7fa71b6482c6081cce1c
2023-02-02Remove key file checkLuc Provoost1-6/+0
A check on the name of the ssh key file for user rapid has been removed. We are now just using the name of the key file as it shows in the rapid.env file. In the past, the file name was always replaced by a hardcoded name for the rapid user. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Ie2219b33c75c54e937182e9307e16f290c82513e
2023-02-01Use meson to build port_info_appLuc Provoost1-0/+101
Adding a meson.build file for the port_info_app. This application is a dpdk app that queries the dpdk ports for its MAC address. That MAC address will then be stored in rapid.env. If needed, it will then be used by PROX e.g. in l2 test cases. The app is only used if the vim parameter in the rapid.env file is set to kubernetes. In case of OpenStack, the MAC address is obtained through the heatclient API, see stackdeployment.py for more details. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Iba2b0b98748807a0d33374644b75b4ea1bfb91cc
2023-02-01Use proper DPDK EAL parameter name: --allowLuc Provoost2-3/+19
The name of one of the EAL parameters has changed. Depending on the DPDK version, we need to use a different name. While building the PROX image, we need now to create a file /opt/rapid/dpdk_version that contains the DPDK version. When running runrapid.py, we will now check this file to decide if we use the old --whitelist, or the new --allow parameter name for the PCI address to be used. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I3e07c552d014c818138d389abce936e275108776
2023-01-30Create a unique machine nameLuc Provoost1-0/+9
Using containers, we need a unique --file-prefix parameter in the EAL DPDK configuration. The file-prefix name is the name of the PROX machine. This is defined by the name parameter in the rapid.env file. When the name parameter is not specified in the rapid.env file, the name will be defined in the test file. The machine name will still be defined in the same way, but when a name is not unique, an number will be appended to the name to make it unique. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I0b4e3be60f8ddef6c7eb30bca25c041ea270e238
2023-01-17Allow a different pod spec for each podLuc Provoost3-23/+37
A new parameter is now introduced in the [PODx] sections of the rapid.pods file. This parameter specifies the pod spec yaml file that will be used to create this specif pod. When this parameter is not available, the default pod-rapid.yaml file will be used. The parameter name is spec_file_name. Also cleaned some lines of code exceeding 80 characters. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Id26881517d45baf4bf6be2e82b4f9e049f2b2547
2023-01-17Fix issue with system_ready_for_rapid fileLuc Provoost2-3/+5
In the /opt/rapid directory, we used to check for the existance of the file system_ready_for_rapid. This file was created when all the initialization is done properly. This check was however broken and always resulted succesfully, even if the file did not exist. When executing runrapid.py, the system will now wait to connect to PROX, till this file exists. The file is created when running start.sh Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Idca52549aae40070a0b6ea1d7ccbf6c322dc631d
2023-01-09Optional MAC address and other changesLuc Provoost3-16/+50
The MAC address of the dataplane interfaces in the environment file is now optional. If not specified, the HW MAC address of that interface will be used. When using l2 tests, the parameter will be needed anyway and the test will fail if the MAC address is not specified. Starting from DPDK 20.11, the EAL --pci-whitelist parameter changed into --allow. We are now checking the DPDK version of the PROX instance to determine which parameter to use. For pods using cgroup v2, we are now using a different file to determine the cpuset available to the pod. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I7a88458406b9ae3c8cae9c583cc37121d40c5073
2023-01-09Add entry in the log file when copying filesLuc Provoost1-0/+3
Added some extra logging in the rapid log file when we copy files to or from the PROX instances. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: If9947ce87cec34f8336c69a243e9d5340ca37aef
2023-01-09fix regex expression to find IPV4 addressLuc Provoost1-1/+1
helper.lua contains code to parse the IP addresses in the parameters.lua file. The regular expression to find the IPV4 address is fixed. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I9baf2b059fa281dc408bd0ea59b77d294750c1ef
2022-09-15Fix thresholds for increment_till_failLuc Provoost1-1/+1
The thresholds that define a successful step of the binary search, were not read from the test file or initialised to the default value in case of the increment_till_fail test. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I8347a683f35f8fd2ff26161709bf0bd16387f2e0
2022-09-15Removing the --socket-mem eal parameterLuc Provoost1-25/+1
This parameter is not needed any more for a successful run on a Kubernetes environment. I had to remove the parameter since this was not working anymore on the K8s deployments I am currently using. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I784611d66e003f1a6e344b2d9c7a2c06c5c2b114
2022-09-14Change search success criteriumLuc Provoost2-4/+12
When running a step in the binary search, we can now change the behaviour by adding a new parameter in the [test] section of the test file: generator_threshold. The default value for this parameter is set to inf, which results in the same behaviour as before this change was implemented. The relative difference between what the cores can generate and what is actually sent into the generator's interface must be lower than the generator_threshold, for the test to be successful. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: Id3e787b3571bdb9f241763fceaffccb9ea81a264
2022-09-13Fix the core to check number of bucketsLuc Provoost1-1/+1
We need to check the number of buckets PROX is using for the irq duration distribution. In the past, the scripts sent a command to PROX to check this on core 1. This was typically a core running the the IRQ task in case of VMs. With containers, this is not the case any more and we need to query which core will run this IRQ task. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: If06295774b05d94549955cfbb0b1bb5f4eaf03e2
2022-02-07Fix the search algorithmLuc Provoost1-9/+14
In case, we are sending less packets through than the requested amount of packets, we will now stop the binary search. Signed-off-by: Luc Provoost <luc.provoost@gmail.com> Change-Id: I7ddb8860d47104beffdd5ff05c0359f91f4908e9
2022-02-06Added sleep_time optional configuration in rapidXavier Simonart5-3/+13
This can for instance be useful to ensure that inter-iteration time is higher than SUT switch max_idle time. Signed-off-by: Xavier Simonart <simonartxavier@gmail.com> Change-Id: I3a94b022f4d38e0c97d5cbd343b36a9a3ed5fd9e
2022-02-06Added support for no warmupXavier Simonart1-0/+4
If warmup time is set to 0, there will be no warmup at all. Before this patch, generator was started and immediately stopped which could result in some packets sent. Signed-off-by: Xavier Simonart <simonartxavier@gmail.com> Change-Id: I7acada86c527bcde70ab0117b79f95208a382cdd
2022-02-06Added support for mis-ordered packets in rapidXavier Simonart3-11/+25
- mis-ordered packets are reported/displayed in an extra column - a treshold can be specified for mis-ordered packets, as a criterium Signed-off-by: Xavier Simonart <simonartxavier@gmail.com> Change-Id: I3e17f86c44e54abcec2412ec57ba5303105b7b91
2021-12-30Prevent rapid considering delayed packets as lostXavier Simonart1-0/+1
In case of high latency in SUT, rapid was considering those packets as lost Signed-off-by: Xavier Simonart <simonartxavier@gmail.com> Change-Id: Ifd52bb0f41e1995105bf216ed1068a50250403d8
2021-12-20Add support for preventing rapid to change flowsXavier Simonart2-5/+8
By default, in flowsize and warmup tests, rapid will set the flows by changing some fields. However, rapid only support some types of flows. Using this patch, one can let PROX define the flows, and prevent rapid to do it. This is done by specifying number of flows to 0 in rapid configuration. Signed-off-by: Xavier Simonart <simonartxavier@gmail.com> Change-Id: I69c7e223aaf536ebcfe9fb52e8305d5625e79fa6
2021-11-02Create combined image and add helm chartYury Kylulin11-4/+131
Changes in this commit: 1) Create self contained image, rapid scripts can be executed from the management pod and create other testing pods in the cluster. 2) Helm chart to automate rapid deployment and proper service account configuration to be able to start management pod. 3) Fix for migration to paramiko. Signed-off-by: Yury Kylulin <yury.kylulin@intel.com> Change-Id: I0281871cce392b3003e0274602a5c90c29af9b23
2021-09-06Changes from ViNePerfSridhar K. N. Rao4-15/+31
This patch adds changes to support using rapid to run tests only. 1. Fix format.yaml path. Add format.yaml as datafile in setup.cfg 2. Do not call generate lua if starting prox is not required. 3. Move logs to results directory - if present 4. Machines should not be used in del, if not created. Fix syntax error. Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com> Change-Id: I580716573d0be2a8da02035f4d180e81b63b3fc8
2021-08-20Fix ssh timeout issueLuc Provoost1-1/+1
Moving to paramiko for ssh introduced a bug: when starting PROX through ssh, a default timeout of 15 seconds was started. The timer is now disable since we need to run PROX as long as the test will take. Change-Id: I65162fff33beb6d30d2f5ee2b83f9bef7880ce4b Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-07-16Using paramiko for all ssh & scpLuc Provoost6-118/+118
By using paramiko, we can now also use differnt credentials to login into the PROX instances. In the ssh section of the environment file, you can now also add the password parameter. Change-Id: I08d31cbf0d02d7e82b7fbe34268851f73ff5dde0 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-06-21Add support for multisocket memory reservationYury Kylulin3-16/+38
Automatic detection of allowed NUMA nodes for memory allocation added to Kubernetes environment. Signed-off-by: Yury Kylulin <yury.kylulin@intel.com> Change-Id: I4f8b2f64971d43414bbd290f7fcbc6210c1d6d71
2021-06-08fix container image and use latest codeLuc Provoost1-10/+4
The container image was not containing a working prox executable any more. Also took out the commit id, so we always have the latest prox code in the latest container image Change-Id: I4336e06031d5bf5b3f08f4bc32b76703fde34040 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-05-12Fix latency buckets with multiple lat coresLuc Provoost3-66/+98
The bucket data of the last latency core was taken into account when collecting the latency stats. We are now adding the bucket data for all cores. Also changed the error reporting when PROX instance is breaking the pipe connection (e.g. when the dataplane network is overloaded). Cleaning up some trailing spaces in pox_ctrl.py Change-Id: I09ba01ac65e7e4e9ff03ad47da83aa4f83250a67 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-05-10Reinstate installation of igb_uioLuc Provoost2-1/+9
For good performance on some platforms, we need the igb_uio driver. With this change, the driver is compiled and insmod done during boot time. When running the rapid scripts, the devbind.sh script is sent to the instances (after replacing the string "MACADDRESS"). Make sure you uncomment the proper line to chose which driver you want to use. Change-Id: I69549400a97f29e06add6ab44515e96ffa9ce0e8 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-05-10Fixing rapidxt for XtestingLuc Provoost5-4/+25
Changes were needed for installing the python rapid python package by adding 2 new files: pyproject.toml & setup.cfg. Also fixed a bug in rapidxt.py Change-Id: Iac98b6068afef4a5f1a97459ea1109211607d53d Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-05-05New test case: gen versus swap via gatewayLuc Provoost4-10/+134
A new test was added: tests/basicrapid_gw.test. Traffic generated will now go to the reflector (swap instance) via the Gateway. The swap instance will also send the traffic back via the gateway. This use case is created to support GCP testing where the interfaces are defined with a /32 netmask, and hence all traffic is routed via the gateway. Change-Id: I60142d650072ab345cd47a0f32cb6a8275da0542 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-04-30Do not run after_boot.sh by defaultLuc Provoost1-10/+12
In the definition of the stack in openstack-rapid.yaml, we define a CloudConfig resource. after_boot.sh is called automatically by a service (check-prox-system-setup.service), after each boot. In this file, defined in the CloudConfig resource, you can put instructions needed for succesful operation and to fix certain issues in your environment. By default, this file should NOT be executed, hence we should not have this in the CloudConfig. Change the name back to after_boot.sh if you need to execute this code. Change-Id: Ifd3f86fe6dfd31444b799833d6b435a310708f25 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-04-28Default use of rapid_rsa_keyLuc Provoost6-23/+35
A secret and public key pair is available in the rapid directory: rapid_rsa_key and rapid_rsa_key.pub. createrapid.py is now modified to upload this key into OpenStack if the keypair does not already exist in OpenStack. Which keypair to use is now only defined in params_rapid.yaml. You can still define a key pair with a different name in params_rapid.yaml. If the secret and public key files are not available, a new keypair will be created and the secret key file will be saved. The name of the secret key file is the name as the OpenStack keypair name. The name of the public file is the same, with the pub extension. Change-Id: Ib8b5e4bf3a3d3138216698dc588a6908248220c0 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-04-22Fix error in sut throughput measurementLuc Provoost1-1/+1
In the fomula to calculate the sut throughput measurement, we used the frequency of the wrong machine: the generator. This issue showed up when testing with generator and sut on a different type of compute node, with different processor frequency. Change-Id: Ib42bb8a30924d6bc6bb2773aed5dcc9145bb50fc Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-04-16Adding documentation in the test filesLuc Provoost16-15/+239
a README file in the tests subdirectory is now describing the parameters used in the test description files. Each test files has a comment pointing to the README file for more information. Change-Id: I49b9b3e1fac0a65c43e7f2d7dc90612923e28f78 Signed-off-by: Luc Provoost <luc.provoost@intel.com>