summaryrefslogtreecommitdiffstats
path: root/VNFs/DPPD-PROX/helper-scripts
AgeCommit message (Collapse)AuthorFilesLines
2020-01-27Code review feedback & back-ground traffic supportLuc Provoost1-117/+162
Various code review comments taken into account. We also support now background traffic, meaning the scripts will start a test between machines and monitor packet loss, latency, etc., but at the same time, the script will also start traffic on background machines. That traffic will not be monitored. Each generator will generate the same ammoung of traffic. Change-Id: I5cd495be230a526f91de965f3d4dd0581681f5ca Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-01-23Fix run_iteration when not using the sut_sockLuc Provoost4-9/+33
When no sut_sock monitoring the swap, the while loop would never come to an end. Also added heartbeat timeout in the generator config files. Change-Id: I3c36991c65b9b826faf3d051a1354ca7eb34c24e Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-01-17Specify latency bucket size & correct calculationLuc Provoost4-20/+61
A new parameter bucket_size_exp in the *.test file specifies the bucket size for latency measurement. The default is 11 which corresponds to a bucket size of 1 usec for a 2GHz processor. This fix is also calculating the bucket size based on the reported frequency. Before it was assuming a 2GHz processor. Change-Id: I70d15e90e151a7e474cd05c9ac91b71a11fd5c78 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-01-13Initial support for percentile latency measurementsLuc Provoost3-38/+130
We are now reporting a percentile number for the round-trip latency. The numbers are only correct for processors running at 2GHz. Will fix this next so it runs for any freqency. We only report 1-127 usec. If the latency goes higher, we are now reporting "VERY HIGH". Change-Id: If9fda63cea8616febcc0f24dc52590976cf39f3d Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2020-01-08Combine code for fixed rate and variable rate testsLuc Provoost2-241/+251
Previously, there were 2 functions: run_flow_size_test and run_fixed_rate. The functionality of run_fixed_rate is now added to run_flow_size_test. Change-Id: Ifda70b8088b9131cc6c034d8007825e6d0917682 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-12-31Better Latency accuray warning messageLuc Provoost1-9/+9
The lacency accuracy warning is now given only taking into account packets that were not dropped. Also fixed an error with the value of r in the run_iteration function Change-Id: Ic05fd2e4b8c59a4b7bcc131e981c08e0001436a6 Signed-off-by: Luc Provoos <luc.provoost@intel.com>
2019-12-30Support for ETSI GS NFV-TST 009Luc Provoost3-101/+249
New test added TST009_Throughput.test allowing to run the throughput benchmark as defined in ETSI GS NFV-TST 009. This change adds the binary search algorithm as defined in the specification document. Change-Id: I00e81a811a9c42b06136694fc1387bc59e954a28 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-12-02Remove UUIDs in centos.jsonLuc Provoost2-13/+12
Removed the remaining UUIDs from the centos.json file, making it easier to reuse this file. This does require a recent version of the packer tool. Updated the README accordingly and also added the --env option in the packer alias, so that the OS_CLOUD variable is set when running packer in the container. Updated rapid.vms with the security group name used in the centos.json file. Change-Id: I4f6815a3250811e336c14857e0ec5e7793a77acf Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-12-02Fix mempool size for swap.cfgLuc Provoost2-2/+2
Fixing an error re-introduced in the swap.cfg file. The mempool size needs to be bigger than the number of rx and tx descriptors (we need some extra buffers in the mempool for other things than rx and tx descriptors). The default number of tx and rx descriptors has been changed in PROX to 2K. This initially went through the testing since this is just giving a warning in the prox.log file for some PMDs. For other PMDs, PROX will not start. Also fixed an inconsistency in rapid.vms: name of the security group. Change-Id: If95677bd6169a16da1b3b2a56d13936a854c385e Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-11-26rapid november release combining changesLuc Provoost23-316/+473
-Renamed monitorswap.test into corestats.test. This is a more generic test case that can monitor the core statistics of all PROX instances -New portstats.test which can monitor the port statistics of all PROX instances -Removed the tasks parameter in the [DEFAULT] section of the *.test files. Runrapid.py is now scaning the PROX configuration files to automatically detect the PROX tasks that are being used. -There is now an option to add the PushGateway parameter in the [Default] section of some *.test files. This will push the measurements to the Gateway, so they can be used by tools like Grafana. The logging of the data in a csv file has also been improved. Results can now be found on the screen, in the RUN*.log files, in the *.csv files and are being pushed to the PushGateway. -In the *.test files, replaced the swapcores, irqcores and impaircores by the cores parameter since they are not treated differently by the code. For a generator, we are still using the gencores and latcores parameters since we need to treat them differently in runrapid.py -devbind.sh is removed from the repository since this file is now created by runrapid.py -renamed the rapidVMs.vms file into rapid.vms -image is now built without ssh access for root. Only using centos user. runrapid.py is now using centos user for ssh sessions. PROX tools are strted using sudo -using new DPDK, mb_lib and PROX versions as defined in deploycentostools.sh -installing missing yum-utils package in image -added flowsize parameter to the impairtest -added list of flowsizes to the fixed_rate testing (l3framerate.test) -the irqtest can now loop through multiple Test Machines in stead of only checking the irq of one. Note that we should have the same cores parameter for all Test machines -changes in prox_ctrl.py to support setting the speed of a generator on a combination of mulitple cores and tasks. Also added the multi_port_stats function to support the new portstats.test -Updated version to 19.11.21 -Fixed some power of 2 in the flow table Change-Id: Ia14543bf3195a2b1bd4184b455da6940f1777fc9 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-11-25Update RAPID readme about packer detailsIllia Polliul2-8/+34
Change-Id: Ieff72073fff06818a5c5bc8fb9116461c95c2901 Signed-off-by: Illia Polliul <ipolliul@juniper.net>
2019-10-13Increase the default number of rx and tx descriptors to 2KXavier Simonart1-1/+1
A high number of RX descriptor can be helpful in avoiding packet loss due to the core being interrupted. This commit increases the default number of descriptors from 256 to 2K. Those values are checked versus the minimum and maximum supported by the pmd. Number of mbufs (mempool size) have been updated n config files to support this default number of dsescriptors. The number of descriptors can still be overwritten by the config file. However those values must always remain within the limits reported by the pmd. Change-Id: I5f7999eca886dad68f3c0733da0d796bf4c06a56 Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
2019-07-05Some fixes after code reviewLuc Provoost25-91/+92
Taking into account comments from Patrice and Xavier Change-Id: Ifdabd1945e074c9ee97b059956f107901392c020 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-07-01Cosmetic change when printing warningsProvoost1-1/+3
Warnings can be printed in the following cases: - When not enough packets are taken into account for measuring packet latency accuracy. - When there us a potential network throughput issue, meaning the generator is generating more packets than the the NIC can handle - When the generator cannot generate the requested load If there are no warnings for a certain measurement, nothing gets printed, suppressing an empty line Change-Id: Iee07c12142e28dcc0ac406bfed7626731ab08f98 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-06-30Multiple changes for June releaseLuc Provoost25-713/+861
- Change inittest in warmuptest. This has now a warmuptime, warmupspeed, packet size and flow size. - Change in centos.json. This will now also copy deploycentostools.sh during image building so that it can be used to re-compile prox in the VM by typing "./deploycentostools.sh compile" - runrapid.py parameters needing a file name, need to be complete file names: no extensions will be added any more by the scripts. - Changes in createrapid.py to handel the openstack CLI output in a simpler way. - The management interface of the VMs can now also be an SRIOV interface. We have now an extra optional parameter in the VM sections of the rapiVMs.vms file: SRIOV_mgmt_port - Changed the name of some sections and keys in the environment file since runrapid.py will not always communicate to OpenStack VMs. This could be containers or any other (virtual) machine. - The previous MachineMap.cfg has been renamed to machine.map - A new test has been added: monitorswap is just showing the statistics of a swap (virtual) machine, without generating any packets. This is useful in situations where an external tester is used. - Latency and core statistics can now be measured even if there are multiple PROX cores and tasks running. A new parameter has been added in the test files with following default value: tasks=[0]. During the collection of statistics, all tasks in this list will be queried. It is ok to have a non-existing task in such a query since it will be ignored. - A --screenlog parameter was added for runrapid.py allowing for more detailed output on the screen during debugging. No need to check the log file. - Previous tests to run multiple packet size with a given flow size and to run multiple flow size, with a given packet size are now combined by specifiying 2 lists: packetsizes & flows - The screen output of this test has also been reworked with more meaningful column names and the test result is now in the field "core received". This allows also for faster termination of the test: When the all packets sent by Gen NIC are received by the cores within the thresholds for packet loss and latency, the test is now stopped, even we were requesting more packets to be sent. Change-Id: I3307e7a972f2140e739f376f146fe875df0303e6 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-06-28Change of openstackrapid directory nameLuc Provoost27-0/+0
The scripts are orrignally written to support an OpenStack deployment. We are now making changes to support container environments, hence renaming the directory to rapid. Change-Id: Icaa2fad83c6da451eee733934151e98e9cd07de4 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-04-12Improved image building & new PROXLuc Provoost25-551/+764
Bug fixes for the script for building an image via the packer tool. This also installs a newer version of PROX with some bug fixes in PROX. The *.test files have been simplified. Also introduced a new file MachineMap.cfg to map test machines on actual VMs. Change-Id: I69445327ef0a907bc8c1566aaa60c733418c541e Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2019-02-07Added use case "Forwaring Rate At Maximum Offered Load"Xavier Simonart2-0/+148
With this use case, the generator is generating at maximum speed We measure the forwarding rate. Throughput, latency, number of packets sent and received are printed every seconds. Change-Id: I05eb874d6ecefd58fb0c876bccf6a32872306076 Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
2019-01-07Improved image build & packet loss measurementsLuc Provoost22-84/+139
Reworked deploycentos1.sh and deploycentos2.sh which is installing a PROX version that deals with non data plane packet counting for improved packet loss measurements. Fixed some issues with the packer scripts. Change-Id: I64aab09d364ab5abbbc5fc75c73afcbe11fed681 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2018-10-11test improvements and fixes for image creationLuc Provoost13-69/+150
Updated and corrected the creation of an image containing the PROX tool with packer. Test will now only be declared successful when average and maximum latency are below a certain threshold. The start speed is now also a parameter of the *.test files. Change-Id: I1112555c87e7a857e1c699921ae08f61281642e1 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2018-06-17Adding centos.json to be used with packer to generate a VM with PROXLuc Provoost16-84/+328
Based on a CentOS cloud image downloadable from the web, you can now use packer to create an image that has DPDK and PROX installed in the proper directories. The VM is also tuned for guest VNFs. A service is also created to deal with the proper isolated_cpu settings for tuned. The service will also make sure uio and igb_uio are loaded. The prox_user_data.sh is not needed any more and hence deleted. ctrl-c of runrapid.py will now also stop prox in all the VMs so a new test can be started. Change-Id: I16cc59878e2d4912757f42e05f98d51dff5feb89 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2018-04-03Added missing file: rapidVMs.vmsLuc Provoost1-0/+31
This file describes flavors, flavor meta data and boot info. All this info can be defined in the [Default] section and will then be used by to create all VMs. This data can also be specified in individual VM sections to overule the deault values per VM. Change-Id: I9d59208360eacec2790e5c467d7afcbeba301251 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2018-03-27Replaced heat stack yaml files by individual server creationLuc Provoost14-928/+188
There is now a new file (rapidVMs.vms) describing how many VMs need to be created. You can specify the same flavor values in the [DEFAULT] section or specify specific flavors per VM. The heat stack yaml files are removed. Change-Id: Ic5ec485e8344a35e9ac9f6e8347adbc70520d10a Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2018-02-21Fixed the calculation of dropped packets and other changesLuc Provoost11-93/+142
* Calculation of aboslute and relative dropped packets was wrong and is now fixed. * ./createrapid.py has now an extra option to select custom availability zones for the different VMs. * Fixed a typo to create the flavor: hw:cpu_thread_policy * The generator config files have now the packet id & signature fields. See PROX for more details on this. * ./runrapid has now a new option --configonly. Default is False. If set to True, all configuration files will be copied to the VMS, but the tests are not started. This allows you to go to the VMs and manually start PROX. * In addition to the average latency, maximum latency is now also displayed. Change-Id: I3bbd7897deb6e452068c07ef68d3959ffa4a4b50 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2018-02-12Zero packet loss testing has been added.Luc Provoost12-72/+277
Using l2 testing, where no ARP messages are being used, you can now run a zero packet loss test. The binary search will continue to find the speed where absolutely no packets have been lost during the test. On top of that, the output has been reformatted to also show the absolute and relative packet loss. The *.test files have now 2 new parameters: drop_rate_treshold and accuracy. When drop_rate_treshold is set to 0, this test will conclude successfull when no packet has been lost whatsoever. If this is set to any other number, the test will be successfull when the relative packet loss is lower than drop_rate_treshold. The new accuray parameter defines when the binary search will stop: when the relative difference between the min and max of the search window is lower than accuracy. The irq testing is slightly modified to print the total number of interrupts in a certain time bucket, not only as the number per second, but when there is no interrupt whatsowever, we print '0' in stead of '0.0' Change-Id: I8fafded0a225cbe1ba7ac61cd28e94d72099e4a0 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2018-02-04Fixed issues with the pkt_size commandLuc Provoost5-70/+219
Fixed issues with pkt_size. For a 64 byte packet, we need to issue pkt_size <core> <task> 60 4 extra bytes will be calculated and added automatically by PROX Also added the missing irq.test and irq.cfg files Added support for OpenStack systems not using floating IP addresses Change-Id: I05d775eb858a58dd349ea370a0e34ac6448590c9 Signed-off-by: Luc Provoost <luc.provoost@intel.com>
2018-01-29Rework of the test cases in the *.test filesProvoost18-107/+211
Cores can now be specified in the test files. An irq test has been added to verify if the VMs are not suffering from long interrupts. Change-Id: I647fda3ebf76c7c2b36a8df9a63d9fb83563c249 Signed-off-by: Provoost <luc.provoost@intel.com>
2017-12-18Adding Security gateway testingProvoost18-367/+1225
Using now .test files describing the test to be executed. Also desribed in the README file how to deal with SRIOV ports. Change-Id: I3eb1551d7f205fef3fa1914bc6959c2126d63425 Signed-off-by: Provoost <luc.provoost@intel.com>
2017-09-25Remove default section in the createrapidDeepak S1-1/+0
Change-Id: Id9c22f92955b64242971ff768a64132451c8350e Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-09-19Splitting the script in 2 parts: createrapid.py and runrapid.pyLuc Provoost11-528/+1098
The split of the scripts is documented in the README file. The VM also runs now devbind.sh with the proper MAC address to bind the dataplane interface with the DPDK poll mode driver. Change-Id: I3e7a6235379dc80530934376c7799ac3396b5325 Luc Provoost <luc.provoost@intel.com> Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-07-14Adding PROX(Packet pROcessing eXecution engine) VNF to sampleVNFDeepak S42-0/+6351
JIRA: SAMPLEVNF-55 PROX is a DPDK-based application implementing Telco use-cases such as a simplified BRAS/BNG, light-weight AFTR... It also allows configuring finer grained network functions like QoS, Routing, load-balancing... (We are moving PROX version v039 to sampleVNF https://01.org/intel-data-plane-performance-demonstrators/prox-overview) Change-Id: Ia3cb02cf0e49ac5596e922c197ff7e010293d033 Signed-off-by: Deepak S <deepak.s@linux.intel.com>