summaryrefslogtreecommitdiffstats
path: root/VNFs/DPPD-PROX/helper-scripts/rapid/createrapid.py
AgeCommit message (Collapse)AuthorFilesLines
2021-04-28Default use of rapid_rsa_keyLuc Provoost1-3/+1
A secret and public key pair is available in the rapid directory: rapid_rsa_key and rapid_rsa_key.pub. createrapid.py is now modified to upload this key into OpenStack if the keypair does not already exist in OpenStack. Which keypair to use is now only defined in params_rapid.yaml. You can still define a key pair with a different name in params_rapid.yaml. If the secret and public key files are not available, a new keypair will be created and the secret key file will be saved. The name of the secret key file is the name as the OpenStack keypair name. The name of the public file is the same, with the pub extension. Change-Id: Ib8b5e4bf3a3d3138216698dc588a6908248220c0 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-03-15Fix the tuned optimizationsLuc Provoost1-1/+1
tuned-adm is now run through check_prox_system_setup.sh at the first system startup/reboot. This will result in the correct grub cmdline parameters. We are also coying the default rapid_rsa_key in the authorized_keys of the rapid user, which is now the default user to run the tests. Fixed also an issue in handle_mirror.c: using now ETYPE_IPv4. Changed deploycentostools.sh to reflect the changes done when building the image with disk-image-create. Change-Id: I5933d21faef9fe02b56c01d0b96e2c143cc5476d Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2021-01-27creation of tests and configs subdirectoriesLuc Provoost1-1/+1
All test files are now in the tests directory. All PROX config files moved to the configs directory. When specifying a test on the runrapid.py command line, you should now also mention the directory: e.g. --test tests/basicrapid.test In these test files, you need now to also include the configs subdirectory when specifying the prox configuration that needs to be used. The --map parameter has also changed. It can still take the file name of the map file, that specifies which PROX instance to use during a test. On top of that, you can now also specify a list of indices of the machines to be used during a test: e.g. --map [1,3]. Change-Id: I887cba59834dd8c903a22bc80959b10da2775398 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-10-19Support dataplane subnet mask & latency histogramLuc Provoost1-1/+4
There is a new parameter in the config_file: dataplane_subnet_mask. This parameter is added to the dataplane IP addresses and is needed by the Linux stack to deal with some protocols (e.g. ARP). If not present, the default value of 24 will be used. In the end, this will make sure that IP addresses in the rapid.env file will have this mask, e.g. /24 If you use other tools than createrapid.py to create the rapid.env file, make sure to add the mask. This commit also introduces a change when exiting: we only copy the prox.log files from the different prox instances in case we expect a new prox.log file to be created in these instances. When using a valid format.yaml file, the tool will now also send the latency distribution data (histogram) to the URL. The tool is now also checking the value of bucket_size_exp: it needs to be 11 or more. Change-Id: I633cdc64ef687fdb6625be1e7482a5a371f83e93 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-09-29New PROX version, background traffic reportingLuc Provoost1-1/+1
deploycentostools.sh is now installing a recent PROX version. When using background traffic, the tool is also reporting the total traffic that was handled by the NFVi stack, including foreground & background traffic. The use of the prox_socket and prox_launch_exit was also broken in a previous release and is now fixed. When setting prox_socket to false for a PROX instance, we will not create a socket connection to control that PROX instance. Setting this option to false is not meaningful for the generator PROX instances since we would not be able to control the behavior of the PROX generator during the test. You could use the generator against another non-PROX instance and then set prox_socket to false for that non-PROX instance. Default is true. prox_launch_exit is used to start and stop the PROX program inside the instance. Default is true. This is useful in case you want to start PROX manually to be able to inspect the PROX UI, but at the same time, let the rapid scripts control the testing. Change-Id: Ib5aa809f4be201859542769f5f55f4989dad97ef Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-08-03Improved IRQ measurements and pushing resultsLuc Provoost1-5/+1
The IRQ test has been reworked to increase the accuracy of the IRQ measurements. Results can now also be pushed to a Prometheus push gateway or to OPNFV's Xtesting. In order to do so, a new file format.yaml has been introduced. Please use this file now to specify the details of the PushGateway or the Xtesting server. Added new test: increment_till_fail.test Change-Id: I111aae3e099bc03e3d2ddd1014a0301bac356e0b Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-07-05Adding vdev support and vfioLuc Provoost1-0/+2
Using vfio in stead of igb_uio. DPDK 2020.5 is NOT compiling the igb_uio driver by default any more. For the l3 mode, we are now using the new PROX vdev feature. This is achieved by adding the vdev parameter in the port section of the PROX config files. One of the advantages is that we can now ping a port managed by PROX: the requests will be forwarded by PROX to the LINUX stack, who will reply to the ping. A tap device is being created by PROX to support this feature. Installing old version of nasm since latest causes some issues. If not pushgateway is being used, it should be commented out in the config_file. vfio is now loaded by specifying the module in /etc/modules-load.d There is also an after_boot.sh script that will be run by check_prox_system_setup.sh, which can be used for further configuration of the instance after boot. There is now also a file created /opt/rapid/system_ready_for_rapid by the check_prox_system_setup.sh script. This file is created once all configuration is done. The runrapid.py script will wait till this file is created before executing any tests. Change-Id: Ic5c41af82642066af42134c3323297f5a06f6f72 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-05-15createrapid.py is now using heat templates yamlLuc Provoost1-389/+48
yaml files have been added to the repo as an example. Please check the README explaining the output section reqs for this yaml file. There is also a new file (config_file): it also specifies which yaml files to use. multiple dataplane interfaces per VM can now be specified and will appear in the <STACK>.env file. An error in setting the packet size has been fixed (see set_udp_packet_size for packet size setting details) Change-Id: Ie89a4940521dac7dd3652acca477739abb9f5497 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-01-31Add support for kubernetes deployment in rapid scripts.Yury Kylulin1-5/+5
In case of kubernetes based deployments procedure for test environment creation is different. Please refer to README.k8s file for more details. Dockerfile and dockerimage.sh files can be used to create a docker image with prox. There is separate createrapidk8s.py script which is used to create appropriate number of PODs for testing based on the rapid.pods configuration. port_info is a helper application to identify port MAC address. Change-Id: I73611f066ac54260aa11dd17c173379df06f46a5 Signed-off-by: Yury Kylulin <yury.kylulin@intel.com> Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2019-11-26rapid november release combining changesLuc Provoost1-4/+9
-Renamed monitorswap.test into corestats.test. This is a more generic test case that can monitor the core statistics of all PROX instances -New portstats.test which can monitor the port statistics of all PROX instances -Removed the tasks parameter in the [DEFAULT] section of the *.test files. Runrapid.py is now scaning the PROX configuration files to automatically detect the PROX tasks that are being used. -There is now an option to add the PushGateway parameter in the [Default] section of some *.test files. This will push the measurements to the Gateway, so they can be used by tools like Grafana. The logging of the data in a csv file has also been improved. Results can now be found on the screen, in the RUN*.log files, in the *.csv files and are being pushed to the PushGateway. -In the *.test files, replaced the swapcores, irqcores and impaircores by the cores parameter since they are not treated differently by the code. For a generator, we are still using the gencores and latcores parameters since we need to treat them differently in runrapid.py -devbind.sh is removed from the repository since this file is now created by runrapid.py -renamed the rapidVMs.vms file into rapid.vms -image is now built without ssh access for root. Only using centos user. runrapid.py is now using centos user for ssh sessions. PROX tools are strted using sudo -using new DPDK, mb_lib and PROX versions as defined in deploycentostools.sh -installing missing yum-utils package in image -added flowsize parameter to the impairtest -added list of flowsizes to the fixed_rate testing (l3framerate.test) -the irqtest can now loop through multiple Test Machines in stead of only checking the irq of one. Note that we should have the same cores parameter for all Test machines -changes in prox_ctrl.py to support setting the speed of a generator on a combination of mulitple cores and tasks. Also added the multi_port_stats function to support the new portstats.test -Updated version to 19.11.21 -Fixed some power of 2 in the flow table Change-Id: Ia14543bf3195a2b1bd4184b455da6940f1777fc9 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-07-05Some fixes after code reviewLuc Provoost1-17/+19
Taking into account comments from Patrice and Xavier Change-Id: Ifdabd1945e074c9ee97b059956f107901392c020 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-06-30Multiple changes for June releaseLuc Provoost1-149/+130
- Change inittest in warmuptest. This has now a warmuptime, warmupspeed, packet size and flow size. - Change in centos.json. This will now also copy deploycentostools.sh during image building so that it can be used to re-compile prox in the VM by typing "./deploycentostools.sh compile" - runrapid.py parameters needing a file name, need to be complete file names: no extensions will be added any more by the scripts. - Changes in createrapid.py to handel the openstack CLI output in a simpler way. - The management interface of the VMs can now also be an SRIOV interface. We have now an extra optional parameter in the VM sections of the rapiVMs.vms file: SRIOV_mgmt_port - Changed the name of some sections and keys in the environment file since runrapid.py will not always communicate to OpenStack VMs. This could be containers or any other (virtual) machine. - The previous MachineMap.cfg has been renamed to machine.map - A new test has been added: monitorswap is just showing the statistics of a swap (virtual) machine, without generating any packets. This is useful in situations where an external tester is used. - Latency and core statistics can now be measured even if there are multiple PROX cores and tasks running. A new parameter has been added in the test files with following default value: tasks=[0]. During the collection of statistics, all tasks in this list will be queried. It is ok to have a non-existing task in such a query since it will be ignored. - A --screenlog parameter was added for runrapid.py allowing for more detailed output on the screen during debugging. No need to check the log file. - Previous tests to run multiple packet size with a given flow size and to run multiple flow size, with a given packet size are now combined by specifiying 2 lists: packetsizes & flows - The screen output of this test has also been reworked with more meaningful column names and the test result is now in the field "core received". This allows also for faster termination of the test: When the all packets sent by Gen NIC are received by the cores within the thresholds for packet loss and latency, the test is now stopped, even we were requesting more packets to be sent. Change-Id: I3307e7a972f2140e739f376f146fe875df0303e6 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-06-28Change of openstackrapid directory nameLuc Provoost1-0/+418
The scripts are orrignally written to support an OpenStack deployment. We are now making changes to support container environments, hence renaming the directory to rapid. Change-Id: Icaa2fad83c6da451eee733934151e98e9cd07de4 Signed-off-by: Luc Provoost <luc.provoost@intel.com>