aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2021-07-28NFVBENCH-216 Fix broken flake8 tests in CIGwenael Lambrouin3-6/+26
- remove version constraints on the "hacking" python package (flake8 plugin that enforces openstack coding style) because this was the root cause holding flake8 in the past. - require flake8>=3.3.0 to have python 3.6 support - ignore all the new errors introduced by newer versions of the tools. The point here is to have flake8 back with equal functionality. If wanted, some of the new error codes could be re-enabled, but this would require to fix the code and could lead to accidental regressions. - align minimum package version numbers in requirements-dev.txt and test-requirements.txt Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I2e85aded58f1b2b9b5e750dba66ac68b13e150a5
2021-07-28tox: fix docs-linkcheck behind a proxyGwenael Lambrouin1-0/+1
Change-Id: I3718f3276f145a64e97968065512fe9c1f7cb91f Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: change packet rate for latency non regression testsGwenael Lambrouin3-2/+278
Base the rate of the latency test on the latest characterization max throughput test instead of the latest non regression max throughput test. The goal is to use the same packet rate for all latency tests and to avoid variations of the latency result due to the variation of the max throughput results, ie to decouple max throughput and latency test results. This is achieved with a new "Given" behave phrase: Given packet rate equal to {percentage} of max throughput of last characterization This new phrase is now used by default in non-regression.feature, but it is still possible to use the previous behaviour with the phrase: Given <throughput> rate of previous scenario Change-Id: I15b5d7a68cd57c67d01d2119781f65114e6d41ce Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: doc and log start nfvbench serverGwenael Lambrouin1-1/+21
Change-Id: I36b7a32525f75bf1dc2b7ec150428afa5298d478 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: log latency test (fixed threshold)Gwenael Lambrouin1-3/+10
Change-Id: I8285829a854f146fb9736d44655a7e848923203e Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: log nfvbench API testGwenael Lambrouin1-5/+7
Change-Id: I67bfba22393f2f324b3c052b443b24c520231172 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: refactor max result searchGwenael Lambrouin1-19/+16
Remove duplicate code introduced by logging and make the max result search easier to read. Change-Id: If88c6d5a8b57ae9e26edab206e0f61526a98d09d Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: log nfvbench traffic runsGwenael Lambrouin1-1/+19
Change-Id: I791b57c78f98252f01c08a6539762725888a3514 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: add secondary logging infrastructureGwenael Lambrouin1-3/+19
In addition to the main logger managed by behave framework to show the test progress and the results summary, add a second logger to provide insight on what's going on internally. That second logger logs to a file in the results dir. Change-Id: Ie2e8012ea54b153a2e661a06a2c521f48e7040b2 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: code cleaning (TEST_DB_EXT_URL)Gwenael Lambrouin2-6/+0
Remove all reference to the TEST_DB_EXT_URL env variable which is not used. Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I552255f11c04da344aac1f2d9dd9f3da4293e553
2021-07-22behave_tests: create results dir if neededGwenael Lambrouin1-5/+11
Change-Id: I122607f24844eda15920b514a29e4efba9a7c4e6 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: configure nfvbench ip/port with env varsGwenael Lambrouin2-7/+17
It is now possible to configure nfvbench server IP address and port number with environment variables: NFVBENCH_SERVER_HOST and NFVBENCH_SERVER_PORT. It is still possible to configure them in feature files, and the values found in feature files take precedence. This allows to have behave tests and nfvbench server running on different machines without changing feature files, which is especially useful for testing. Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I98dc7f87a1a233b90b44dfc8b26a1e63961fff3c
2021-07-22Compare the latency result with a fixed threshold of 1msGwenael Lambrouin2-2/+31
Change-Id: I2b4ea4ee6e6442d4ceac268e7bf3c6bf9277ff54 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-08NFVBENCH-215 Fix wrong throughput ratio in latency testsfmenguy7-12/+13
Change-Id: I5c976dd49a2c17b47559b1d6a565a6e78f7cfd0e Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-09Add Gwenael as committerfmenguy1-0/+4
Change-Id: I56883e7ae5c4057a38ca43b6f1199b810e2ac947 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-08Fix pps error message in behave testsfmenguy1-1/+1
Change-Id: I2f050f8a6f193c4e04ac8a427aedb7c241633b73 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-07Add missing file for clouds.yaml managementfmenguy1-0/+1
Change-Id: Ic9be998451dceb314b2fc46e936604402a27e925 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04Increase loopVM version in Dockerfile5.0.3fmenguy1-1/+1
Change-Id: I4496b90a5b49b0c1e9c2c841240bbe9c0dca5f45 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04Merge "Increase VM version to 0.15"Fran�ois-R�gis Menguy1-1/+1
2021-06-04Merge "behave_tests: increase nfvbench_test_api timeout"Fran�ois-R�gis Menguy1-1/+1
2021-06-04Merge "behave_tests: add feature quick-test-10kpps"Fran�ois-R�gis Menguy2-1/+35
2021-06-04Merge "Increase Xtesting version in requirements and remove behaveframework ↵Fran�ois-R�gis Menguy3-126/+1
temporary patch"
2021-06-04Increase VM version to 0.15fmenguy1-1/+1
Change-Id: I5ab68c6eadd4cbb91753870c10b53ead0b973bca Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04NFVBENCH-213 Add clouds.yaml management in generator VMfmenguy3-21/+37
Change-Id: Ia741847f886a291f0b0d7276e36d5c18f7819185 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04NFVBENCH-212 Add clouds.yaml file as a config file to use for openstack API ↵fmenguy5-57/+125
access Change-Id: If855ffda1070ed9c9c4544230e4efec185a93f45 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04NFVBENCH-211 Fix VPP driver for loop VMfmenguy2-2/+3
Change-Id: I6e55b4c0ad41d773b5303b8dc8c09cee8482c646 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-02behave_tests: increase nfvbench_test_api timeoutGwenael Lambrouin1-1/+1
In some cases, the 50 seconds timeout to wait for nfvbench HTTP server to be ready is too short. (Convoluted but real example: when the DNS servers are not properly configured and nfvbench tries to reach the OpenStack APIs while it does not need them because we just want to do a loopback test without loop VM) The new timeout is 120 seconds. Change-Id: I4932eff7c9a100370e7ceaaa2a467efbbceb5993 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-06-02behave_tests: add feature quick-test-10kppsGwenael Lambrouin2-1/+35
Add a quick test at low packet rate for troubleshooting purpose: - duration: 10s - fixed packet rate: 10kpps - frame size: 64 - flow count: 128 Change-Id: Ib736dbe7ade6d0b6e4d484a6d8bfca34bb5c4c5a Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-06-02Increase Xtesting version in requirements and remove behaveframework ↵fmenguy3-126/+1
temporary patch Change-Id: Id52abffc67a17066c9282bcdcab04ecfd6c568cb Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-01xtesting: change nfvbench config file path for test case 10kpps-pvp-runGwenael Lambrouin3-22/+38
The test case 10kpps-pvp-run now looks for nfvbench config file at the same place as the behave tests: /etc/nfvbench/nfvbench.cfg Change-Id: I260dbf6ddefc6d683e79cc3afb03ead8fb408218 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-05-25NFVBENCH-210 Remove TRex restart from behave test scenariosfmenguy3-5/+0
Change-Id: I19895eae9a3e68999cdde15f3f21520dcf6e6897 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-05-25Update loop VM image to v0.145.0.2fmenguy1-1/+1
Change-Id: I1cf8a36544c5f9205391e916eaa00311f8808a46 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-05-25NFVBENCH-209 Fix NFVbench loopvm build failed on testpmd stepfmenguy5-49/+20
Change-Id: I0715b46b34b382176f2e06798deeede402b4c2fd Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-05-17Revert cleanup and allow force push image to GS (tmp)fmenguy1-34/+11
Change-Id: Ic742d0f538b62a101d3c3752fd2008163bb3bbe0 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-05-17NFVBENCH-208 Fix VM imagesfmenguy5-0/+23
Change-Id: I9014b36b4f2e044f807a9b8a997e8c455511c046 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-04-28Add generator VM Xtesting call example in documentation5.0.1fmenguy1-1/+15
Change-Id: I789caa7f46bd2b21af15ec5ffe4d857e10a7734c Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-04-28NFVBENCH-196: New NFVbench image for generator part (nfvbench and TRex codes ↵5.0.0fmenguy23-102/+859
inside VM) Change-Id: Id919a1b9dbf5e04b198d022432ed5c64232a27ba Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-04-28NFVBENCH-206 Fix TRex patches remaining codefmenguy1-15/+0
Change-Id: Ifebb89d9d0aa2a21257c8e925cd51faa07b88353 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-04-27NFVBENCH-190: Add a 'i40e_mixed' option, trex accepts other i40e driven ↵Pierrick Louin4-3/+53
ports to run in kernel mode on the same board Change-Id: I33b1d848f409e0431bd4e247456b1d697d8b9d04 Signed-off-by: Pierrick Louin <pierrick.louin@orange.com>
2021-04-27NFVBENCH-205 - Add behave tests for characterization and non-regressionfmenguy20-19/+965
Change-Id: I708eee21a9fd11e7a276707fb0b43d8598381ce7 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-03-30NFVBENCH-163: Add gratuitous ARP in case of L3 router modefmenguy10-11/+125
Change-Id: Iec2b186176285f723eb2685319c55e6cd6d33a8a Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-03-11Add Kibana visualizations examples for NDR resultfmenguy20-1/+1015
Change-Id: I652dc2c3d69ae5f6bff5019b0868d1becaceec63 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-03-05NFVBENCH-204 Remove some piece of code depending on a non merged patch ↵Pierrick Louin1-3/+1
(NFVBENCH-190) Signed-off-by: Pierrick Louin <pierrick.louin@orange.com> Change-Id: I34476f743f9ab0620f866e93ac068941a3a031af
2021-03-05NFVBENCH-203: Make --show-config really print the actual running configPierrick Louin2-8/+25
Add a --show-pre-config cmd to print the config before CLI override. Signed-off-by: Pierrick Louin <pierrick.louin@orange.com> Change-Id: I708498cf10eee043242cd975ab05bc44883bca4e
2021-03-05NFVBENCH-202: Generalize auto T-Rex restart on every command line related ↵Pierrick Louin1-14/+25
parameter change. Signed-off-by: Pierrick Louin <pierrick.louin@orange.com> Change-Id: Ia8e572e8ebea37bb7fb2c898d6a311b522dec0b4 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-03-04NFVBENCH-201 Fix port creation with accurate subnet_idfmenguy2-0/+26
Change-Id: I419cfeb82d16ff1ac1145df5d52fe6f88d890a49 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-03-04NFVBENCH-200 Cleanup refactoringfmenguy4-34/+54
Change-Id: I380a9efecdce8d525a9ae41949828c73e9cdac5f Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-03-04NFVBENCH-199 add exception to avoid API locked with pending runfmenguy1-3/+9
Change-Id: I12635d3f002bbac38e6c063ed3ca972ad02685ce Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-03-03NFVBENCH-198 Error in File logger management (REST API)fmenguy1-6/+7
Change-Id: I9c798b3bcbf416c0d3f107856aca0872c650a580 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-03-03NFVBENCH-197 Upgrade NFVbench packaging (Ubuntu version and requirements)fmenguy2-21/+17
Change-Id: I79c8bc86272435bd4fa9ed9d1129eb964513ec1a Signed-off-by: fmenguy <francoisregis.menguy@orange.com>