aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2023-01-18nfvbenchvm: fix image URL in build logGwenael Lambrouin1-2/+2
Update the link to access to the qcow2 VM image in Google storage: use https instead of http Change-Id: I315fec57ae12256753bc9abf561c29aafa9aa325 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2023-01-06tox: fix testing issues after VM buildGwenael Lambrouin1-2/+2
When the traffic generator VM is built with option -s (use local nfvbench code), the code and the unit test files are copied to nfvbenchvm/dib/elements/nfvbenchvm/static/opt/nfvbench. This can lead to two problems: 1) if pytest is invoked after that, the test collection will fail because of duplicated test files; 2) if flake8 is invoked after that, the same files will be checked twice. To fix this this issue that can occur on a developper's machine, we tell pytest to look for test files in the test/ directory rather than everywhere. This is fine because test/ is the base directory for all test files. Similarly, we tell flake8 to skip the nfvbenchvm/ directory. Change-Id: Idb7f62828db4290f16221bb72bd9c6d5d958f649 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-12-15nfvbenchvm: release 0.16Gwenael Lambrouin5-9/+108
- separate loop VM and generator VM version numbers (a first step towards using nfvbench version number for the generator VM) - increase loop VM version number - Add a release note page dedicated to nfvbench loop VM Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I2bc651eb315bfa889e5dd1d56f69e2c3d9219646
2022-12-15Update to py38Cédric Ollivier4-61/+50
py38 is the oldest py version support vs Ubuntu 22.04 python third-party. It updates requirements to yoga too. Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> Change-Id: I9d39f3600a4d0c432b7caf2f0b2bd51554e12318
2022-11-10nfvbenchvm: refactor wait for VPP serviceGwenael Lambrouin1-6/+39
Sleeping for 10s and hoping that VPP service will be ready to accept vppctl commands does not work all the times. Instead of increasing the sleep time, poll every second until "vppctl show int" returns something. Wait at most 30 seconds. By the way, exit with a clear error message if "vppctl show int" does not report two network interfaces as expected. Change-Id: I6f0f053378c0bee8fd28a231dc1041c94642f5b3 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-10-27behave_tests: refactor TestAPI DB lookupGwenael Lambrouin5-57/+8353
- use testapi.TestapiClient everywhere - relax search constraints: match only project name (nfvbench), test case name (characterization or non-regression), scenario tag (throughput or latency) and user_label (test chain identifier: identifies, among other things, the platform, the compute class under test, ...) - add unit tests for some of the related behave steps Change-Id: I26763f845c2286601cb958b326525b29320a1627 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-10-17behave_tests: refactor TestAPI HTTP requestGwenael Lambrouin1-6/+17
- raise a requests.exceptions.HttpError if the HTTP request returned an unsuccessful status code - document exceptions that can occur during the communication with TestAPI database Change-Id: Iea81e76ef8938cb34a06529105ac6c282e664479 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-10-17behave_tests: add unit tests for TestAPI clientGwenael Lambrouin6-5/+74003
Change-Id: I31a8d774df406c993f6fb605eb09025d643c6e39 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-10-17Fix Jenkins buildGwenael Lambrouin2-2/+2
- docs: fix broken link - pep8: fix missing space Change-Id: I421d5a8e10b4e63c379c99f306813ade15f1f7a2 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-05-31behave_tests: fix infinite recursionGwenael Lambrouin1-1/+1
Fix bug in behave_tests that occurs when behave looks for a previous result (either non-regression or characterization) in testapi database. When the result cannot be found on the first results page, behave enters an infinite recursion. Eventually, the operating system kills the python interpreter leading to a return code equal to 137. When behave is run by xtesting run_tests command, run_tests is also killed so the error is not reported by xtesting. This is now fixed. Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I6b0feafb5ebadf7d0d1df6d0ee03fd22cbe6899d
2022-03-10nfvbenchvm: set mgmt interface MTU=1500 by defaultGwenael Lambrouin3-2/+50
By default, configure the MTU of the management interface to the conservative value of 1500: this will reduce the risk to get an unmanageable VM. Else the MTU we will get will depend on the IaaS setup and it could be high. And this will lead to troubles when we try to manage the VM from a machine configured with a high MTU and when the network path contains a link with a low MTU. This can typically happen when we try to manage the VM from another VM in the same IaaS and when the management traffic has to go outside the IaaS. To set the MTU to a different value, configure the INTF_MGMT_MTU variable in /etc/nfvbenchvm.conf. If INTF_MGMT_MTU is set to the special value "auto", the MTU will not be configured and it will keep the value set by the hypervisor ("legacy" nfvbenchvm behavior). If INTF_MGMT_MTU is unset, the MTU will be set to 1500. In other cases, the MTU will be set to the value of INTF_MGMT_MTU. Change-Id: I5816113bbacec5fcdcc9672e20d8a4e2ac0c599a Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-03-08nfvbenchvm: keep only 1 copy of behave_tests codeGwenael Lambrouin1-0/+2
Have multiple versions of the code in the VM can be confusing during troubleshooting sessions, so we only keep one copy of behave_tests code as we did for nfvbench code. Change-Id: I5af6cd960486bafaaddfcbac11504c6dcbd1848d Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-03-04nfvbenchvm: add option to use local nfvbench codeGwenael Lambrouin3-29/+122
build-image.sh has a new option -s to build the generator VM image from the local nfvbench code (Git working copy) instead of the master branch of nfvbench repository on OPNFV Gerrit. This is useful for testing during the development phases. When this option is used, the version of the generator VM image is extended with nfvbench development version number to be able to distinguish the development images from the latest published image. Change-Id: If21cbf2be59ff4f4110ebc157af0f934b98bddcb Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-03-01nfvbenchvm: add 2 debug features to build-image.shGwenael Lambrouin11-24/+97
- The new option -t (enable debug traces) allows to show in the build log the commands run in the shell scripts, including the commands defined in the disk image builder elements. - The new option -d (debug) instructs disk-image-create to drop the developer in a shell inside the chroot in case an error occurs. This makes troubleshooting of the image possible (inspect files, run commands, ...) Change-Id: I5f76e7dec64fc4c86b1515f6b81a16e11e03eadf Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-02-28nfvbenchvm: abort build on errorGwenael Lambrouin11-2/+35
Make sure a VM image build fails if any step fails. Else we can end up with a bad image not containing all that we want, and we discover this later at run time. For instance, without the current patch, we can get a generator VM image without nfvbench installed because "pip install" failed, but this is not detected by the build process because the last step in post-install.d/52-nfvbench-script is an "echo" command that returns a success code. We fix this by making all the Bash scripts fail if any command whose return code is not tested fails (set -euo pipefail). Change-Id: Ic5ec9eb85a8d6e7e4d1dcbac8ebcac5931e4808e Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-02-28nfvbenchvm: fix broken build after dib updateGwenael Lambrouin6-14/+30
Attempting to build nfvbench VM images with diskimage_builder (dib) 3.16.0 leads to the following error: diskimage_builder.element_dependencies.MissingElementException: Element 'block-device' not found Something changed at some point of diskimage_builder history, and the updated doc states: "When using the vm element, an element that provides block-device should be included". To fix this and prevent future similar issues, this patch: - adds the "block-device-mbr" element to the dependency list - sets diskimage_builder version By the way: - replace the deprecated "centos7" element with the more generic "centos" element. - add missing dependencies in the pre-requisites section of nfvbenchvm doc - install pip with package manager (the current latest version of get-pip.py is not compatible with python 3.6, so let's use the occasion to stop getting pip directly from the Internet) - use Python 3 in build-image.sh Change-Id: I3198a1042eca04224b2a62db443c39a76903cf22 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-02-21.gitlab-ci.yml: fix vm image jobsGwenael Lambrouin1-6/+6
1) use CentOS 7 hosts to build VM images: CentOS 8 is not supported as a build host according to diskimage-builder documentation: https://docs.openstack.org/diskimage-builder/latest/user_guide/supported_distros.html (page checked on 2022-02-08) 2) add missing packages for diskimage-builder: qemu-img kpartx sudo e2fsprogs 3) use "python3 -m venv" instead of virtualenv (else the pipeline fails because of https://github.com/pypa/virtualenv/issues/1029) Change-Id: I14daf76dded1f230e12c47218ddcb1fe9029bcd8 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2022-02-10Set pylint version to 2.10.2Gwenael Lambrouin2-2/+2
If we don't specify a version of pylint, jenkins jobs in gerrit sometimes fail for a reason unrelated to the submitted change because a newer version of pylint adding new checks is used for the first time during the job run. This is very annoying. Let's stick to the latest version of pylint known to work with nfvbench, and we will consider an update later on. Change-Id: Ia2f87d0730ec1e733905a002a651ad6bcdf7edbb Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-09-06Merge "behave_tests: tweak nfvbench results JSON filename"Gwenael Lambrouin2-4/+13
2021-09-03Switch VPP package repository to packagecloud.ioGwenael Lambrouin1-3/+9
VPP install guide [1] now refers to packagecloud.io instead of nexus.fd.io as the source for VPP binary packages. [1] https://fd.io/docs/vpp/v2106/gettingstarted/installing/centos.html I could not find when the switch took place, but I recently got some intermittent issues when trying to use nexus.fd.io (Bad Gateway errors), so it seemed like a good opportunity to make the move. Additionally, this will make it possible to get vpp releases higher than 19.08 when building nfvbench loop VM image. Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I9cff7289e426e0ed8c8a7df8322a4a4baeb79f47
2021-08-31behave_tests: tweak nfvbench results JSON filenameGwenael Lambrouin2-4/+13
In case nfvbench packet rate is expressed as a percentage of the max throughput rather than a value in pps or bps, use that percentage to build nfvbench results JSON filename. This is sometimes needed for results post-processing because the percentage information cannot be found in nfvbench results itself. Change-Id: I7d16dba16a733a8ee58a6f80ce4df40cb40e9843 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-08-25Add Gitlab-CI ConfigurationTrevor Bramwell1-0/+76
Adds jobs for tox verification, ReadTheDocs builds on changes to docs/, docker image builds, and disk image verification and builds. Change-Id: Ie8e579a12ae6a0efd325a04de670486cafd50f46 Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
2021-08-24Be explicit about text file encodingGwenael Lambrouin7-10/+10
Python PEP 597 (https://www.python.org/dev/peps/pep-0597) recommends to use an explicit encoding for text files instead of the default locale encoding. Pylint 2.10 adds a new checker named unspecified-encoding for that. The present patch adds explicit utf-8 encoding to open() calls in nfvbench and fixes pylint unspecified-encoding warnings. Remark: this patch does not change nfvbench behaviour on systems where utf-8 is the locale encoding, which is generally the case on Linux systems. Change-Id: Ic4dfb37e1ea958452a0173f7630a68f0d95071ae Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-30docs: how to use tox on a developer's machineGwenael Lambrouin3-6/+100
Change-Id: I0acbf6a89b1126336862a008bbe4331fa94ef0a5 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-28NFVBENCH-216 Fix broken flake8 tests in CIGwenael Lambrouin3-6/+26
- remove version constraints on the "hacking" python package (flake8 plugin that enforces openstack coding style) because this was the root cause holding flake8 in the past. - require flake8>=3.3.0 to have python 3.6 support - ignore all the new errors introduced by newer versions of the tools. The point here is to have flake8 back with equal functionality. If wanted, some of the new error codes could be re-enabled, but this would require to fix the code and could lead to accidental regressions. - align minimum package version numbers in requirements-dev.txt and test-requirements.txt Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I2e85aded58f1b2b9b5e750dba66ac68b13e150a5
2021-07-28tox: fix docs-linkcheck behind a proxyGwenael Lambrouin1-0/+1
Change-Id: I3718f3276f145a64e97968065512fe9c1f7cb91f Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: change packet rate for latency non regression testsGwenael Lambrouin3-2/+278
Base the rate of the latency test on the latest characterization max throughput test instead of the latest non regression max throughput test. The goal is to use the same packet rate for all latency tests and to avoid variations of the latency result due to the variation of the max throughput results, ie to decouple max throughput and latency test results. This is achieved with a new "Given" behave phrase: Given packet rate equal to {percentage} of max throughput of last characterization This new phrase is now used by default in non-regression.feature, but it is still possible to use the previous behaviour with the phrase: Given <throughput> rate of previous scenario Change-Id: I15b5d7a68cd57c67d01d2119781f65114e6d41ce Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: doc and log start nfvbench serverGwenael Lambrouin1-1/+21
Change-Id: I36b7a32525f75bf1dc2b7ec150428afa5298d478 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: log latency test (fixed threshold)Gwenael Lambrouin1-3/+10
Change-Id: I8285829a854f146fb9736d44655a7e848923203e Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: log nfvbench API testGwenael Lambrouin1-5/+7
Change-Id: I67bfba22393f2f324b3c052b443b24c520231172 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: refactor max result searchGwenael Lambrouin1-19/+16
Remove duplicate code introduced by logging and make the max result search easier to read. Change-Id: If88c6d5a8b57ae9e26edab206e0f61526a98d09d Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: log nfvbench traffic runsGwenael Lambrouin1-1/+19
Change-Id: I791b57c78f98252f01c08a6539762725888a3514 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: add secondary logging infrastructureGwenael Lambrouin1-3/+19
In addition to the main logger managed by behave framework to show the test progress and the results summary, add a second logger to provide insight on what's going on internally. That second logger logs to a file in the results dir. Change-Id: Ie2e8012ea54b153a2e661a06a2c521f48e7040b2 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: code cleaning (TEST_DB_EXT_URL)Gwenael Lambrouin2-6/+0
Remove all reference to the TEST_DB_EXT_URL env variable which is not used. Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I552255f11c04da344aac1f2d9dd9f3da4293e553
2021-07-22behave_tests: create results dir if neededGwenael Lambrouin1-5/+11
Change-Id: I122607f24844eda15920b514a29e4efba9a7c4e6 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-22behave_tests: configure nfvbench ip/port with env varsGwenael Lambrouin2-7/+17
It is now possible to configure nfvbench server IP address and port number with environment variables: NFVBENCH_SERVER_HOST and NFVBENCH_SERVER_PORT. It is still possible to configure them in feature files, and the values found in feature files take precedence. This allows to have behave tests and nfvbench server running on different machines without changing feature files, which is especially useful for testing. Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com> Change-Id: I98dc7f87a1a233b90b44dfc8b26a1e63961fff3c
2021-07-22Compare the latency result with a fixed threshold of 1msGwenael Lambrouin2-2/+31
Change-Id: I2b4ea4ee6e6442d4ceac268e7bf3c6bf9277ff54 Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>
2021-07-08NFVBENCH-215 Fix wrong throughput ratio in latency testsfmenguy7-12/+13
Change-Id: I5c976dd49a2c17b47559b1d6a565a6e78f7cfd0e Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-09Add Gwenael as committerfmenguy1-0/+4
Change-Id: I56883e7ae5c4057a38ca43b6f1199b810e2ac947 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-08Fix pps error message in behave testsfmenguy1-1/+1
Change-Id: I2f050f8a6f193c4e04ac8a427aedb7c241633b73 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-07Add missing file for clouds.yaml managementfmenguy1-0/+1
Change-Id: Ic9be998451dceb314b2fc46e936604402a27e925 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04Increase loopVM version in Dockerfile5.0.3fmenguy1-1/+1
Change-Id: I4496b90a5b49b0c1e9c2c841240bbe9c0dca5f45 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04Merge "Increase VM version to 0.15"Fran�ois-R�gis Menguy1-1/+1
2021-06-04Merge "behave_tests: increase nfvbench_test_api timeout"Fran�ois-R�gis Menguy1-1/+1
2021-06-04Merge "behave_tests: add feature quick-test-10kpps"Fran�ois-R�gis Menguy2-1/+35
2021-06-04Merge "Increase Xtesting version in requirements and remove behaveframework ↵Fran�ois-R�gis Menguy3-126/+1
temporary patch"
2021-06-04Increase VM version to 0.15fmenguy1-1/+1
Change-Id: I5ab68c6eadd4cbb91753870c10b53ead0b973bca Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04NFVBENCH-213 Add clouds.yaml management in generator VMfmenguy3-21/+37
Change-Id: Ia741847f886a291f0b0d7276e36d5c18f7819185 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04NFVBENCH-212 Add clouds.yaml file as a config file to use for openstack API ↵fmenguy5-57/+125
access Change-Id: If855ffda1070ed9c9c4544230e4efec185a93f45 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>
2021-06-04NFVBENCH-211 Fix VPP driver for loop VMfmenguy2-2/+3
Change-Id: I6e55b4c0ad41d773b5303b8dc8c09cee8482c646 Signed-off-by: fmenguy <francoisregis.menguy@orange.com>