aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-06-18Stop running tests from tests/unitEmma Foley6-47/+7
Since all tests have been moved into yardstick/tests, temperary code to run tests in both tests and yardstick/tests should be removed. JIRA: YARDSTICK-837 Change-Id: I3bf3c9ece9bc8457d992e53ee8cc5cadb1dec8f0 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-18Merge "Cleanup BaseMonitor unit tests"Rodolfo Alonso Hernandez1-7/+11
2018-06-18Cleanup BaseMonitor unit testsRodolfo Alonso Hernandez1-7/+11
Stop multiprocessing queues to avoid BrokenPipe exception. Assert correctly the raise of exceptions. JIRA: YARDSTICK-1229 Change-Id: I1a1c92c49a439283a35c9a69b59d6866f35ec09a Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-15Replace MagicMocks with Mocks from SearchRunner UTsMiikka Koistinen1-27/+16
This patch replaces unneeded MagicMocks with Mocks from TestSearchRunner and TestSearchRunnerHelper. JIRA: YARDSTICK-1169 Change-Id: I30a7bbc71c0b1062e6c0dfe634a5a8f1ecb7c684 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Convert SLA asserts to raisesMiikka Koistinen46-156/+220
This commit converts Python assertions to a custom exception in all places where SLA validation is checked with an assertion. This commit also fixes all emerged pylint errors. JIRA: YARDSTICK-966 Change-Id: If771ed03b2cbc0a43a57fcfb9293f18740b3ff80 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Add vsperf_dpdk SLA validation unit test casesMiikka Koistinen1-0/+45
JIRA: YARDSTICK-1228 Change-Id: Ic32f2bcd7f8bdf718c9c266666409d32ecab4924 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Cleanup vsperf unit testsMiikka Koistinen1-60/+139
Remove unneeded mock return values and improve the test execution. Add unit test cases about SLA validation. JIRA: YARDSTICK-1227 Change-Id: I90b8b9c4cb6a57ae4b0869e84af0da43ee22a670 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-06-15Merge "assert[Greater,Equal] -> assert_{,not_}called"Rodolfo Alonso Hernandez10-51/+52
2018-06-15Bugfix: heat conext of test case yamls jinja2 bypass sriov typerexlee877622-25/+22
the "if provider" condition bypass when provider = "sriov", related to https://gerrit.opnfv.org/gerrit/#/c/32789/ networks: test: cidr: '10.0.1.0/24' {% if provider == "vlan" %} provider: {{provider}} physical_network: {{physical_network}} {% if segmentation_id %} segmentation_id: {{segmentation_id}} {% endif %} {% endif %} JIRA: YARDSTICK-1242 Change-Id: I6a611e839654f401648d9bd6188589816f7b4946 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-06-14Merge "Add "host_name_separator" variable to Context class"Rodolfo Alonso Hernandez9-39/+78
2018-06-14Merge "Install dependencies: bare-metal, standalone"Emma Foley5-31/+97
2018-06-14Install dependencies: bare-metal, standaloneStepan Andrushko5-31/+97
This will install all dependencies needed on the bare-metal and standalone physical nodes. JIRA: YARDSTICK-1177 Change-Id: I719ca64d4da098d6dd60597e5fc4ad6f4715f79e Signed-off-by: Stepan Andrushko <stepanx.andrushko@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-14Stop time.sleep mocks in unit testsRodolfo Alonso Hernandez2-2/+12
tVpeApproxVnf and TestProxSocketHelper classes mock "time.sleep" in the setUp method without deleting it at the end of the test execution. JIRA: YARDSTICK-1243 Change-Id: Iff31d9c7b400ad8a47f37792aeb0d20328b9d9e1 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add send socket commands functionRodolfo Alonso Hernandez3-0/+87
Pktgen provides a TCP socket connection to allow the user to control it from a remote console or program [1]. This new method will provide Yardstick the ability to send string commands to a port in a remote host. [1] http://pktgen-dpdk.readthedocs.io/en/latest/socket.html JIRA: YARDSTICK-1186 Change-Id: I9d64ccad662fa3599de65654c5dab02833fcc91d Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Create user defined "NodePort" services per podRodolfo Alonso Hernandez3-22/+79
This feature will give the user the ability to create user defined "NodePort" services per pod, a part from the default SSH port created, by default, for each pod created. Example of Kubernetes yaml definition: apiVersion: v1 kind: Servicemeta data: name: pod-1-service-nodeport spec: type: NodePort ports: - name: web port: 80 targetPort: 8888 nodePort: 33333 clusterIP: 10.254.0.8 selector: app: pod-1 Example of Yardstick test case definition: context: type: Kubernetes servers: host: containers: - image: ... securityContext: allowPrivilegeEscalation: false - image: ... node_ports: - port: <port number> # Mandatory name: <port name> # Optional targetPort: <port name> # Optional, default: targetPort=port nodePort: <port number> # Optional, assigned by Kubernetes Kubernetes service, type "NodePort" [1] [1] https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport JIRA: YARDSTICK-1181 Change-Id: Ia7730e4569d5526d573402a98c27d55c5c82cdef Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Change "KubernetesObject" class name to "ReplicationController"Rodolfo Alonso Hernandez2-19/+22
The class name "KubernetesObject" is not accurate. This class holds the definition of a Kubernetes replication controller. The suggested name is therefore "ReplicationControllerObject". JIRA: YARDSTICK-1180 Change-Id: I9ee9228f9528cb779e54a9bc178760b6e24c85c2 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Specify the networks to be used per podRodolfo Alonso Hernandez2-1/+28
If CRD "Network" is defined and network items are created, each pod (server) can have access to one or several networks. This is defined in the metadata section, as "annotations.networks" [1]. Example of Kubernetes pod definition with networks: apiVersion: v1 kind: Pod metadata: name: test-pod annotations: networks: '[{"name": "flannel"}]' Example of Yardstick server definition with networks: context: type: Kubernetes servers: host: containers: - name: ... networks: - flannel # These names must be defined in # context.networks ... networks: - name: flannel plugin: flannel Kubernetes annotations [2]. [1]https://github.com/intel/multus-cni/tree/b9446232cdf4f1b6f2bea583291973cc97e963f4#configuring-multus-to-use-kubeconfig-and-a-default-network [2]https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/ JIRA: YARDSTICK-1178 Change-Id: I6e7b4bacf10810833ec733c14d44e5db613675e3 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add new Kubernetes resource kind: "Network"Rodolfo Alonso Hernandez6-4/+370
Add a new Kubernetes resource kind: "Network" [1] [2] Kubernetes network plugins (alpha definition): [3] Network definition example in Kubernetes: apiVersion: "kubernetes.com/v1" kind: Network metadata: name: flannel plugin: flannel args: '[ {delegate": {"isDefaultGateway": true}} ]' Proposed Kubernetes context network definition: context: networks: - name: flannel plugin: flannel args: (string) - name: sriov_upload plugin: sriov args: (string) [1]https://builders.intel.com/docs/networkbuilders/multiple-network-interfaces-in-kubernetes-application-note.pdf [2]http://cdn.opensourcecloud.cn/zt/k8s/01.pdf [3]https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/ JIRA: YARDSTICK-1160 Change-Id: I71a49ac14e3d28ded91d2ed3cd9cc527e40303f7 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add new Kubernetes resource kind: "CustomResourceDefinition"Rodolfo Alonso Hernandez7-3/+245
Custom resource definition example in Kubernetes: apiVersion: "apiextensions.k8s.io/v1beta" kind: CustomResourceDefinition metadata: name: networks.kubernetes.com spec: group: kubernetes.com version: v1 scope: Namespaced names: plural: networks singular: network kind: Network Proposed Kubernetes context network definition: context: custom_resources:     - name: network        # name of the resource (singular)       version: v1          # optional, "v1" by default       scope: Namespaced    # optional, "Namespaced" by default From this definition, we will extract the Kubernetes parameters: - metadata.name: custom_resources.name + "s" + context_name + ".com" - spec.group: context_name + ".com" - spec.scope: custom_resources.scope - spec.version: custom_resources.version - spec.names.plural: custom_resources.name + "s" - spec.names.singular: custom_resources.name - spec.names.kind: custom_resources.name with first capital letter [1] https://kubernetes.io/docs/concepts/api-extension/custom-resources/ JIRA: YARDSTICK-1163 Change-Id: If8980dc3f6ddf9c6949bf15be8011aa98482ddc9 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add "securityContext" parameter in Kubernetes contextRodolfo Alonso Hernandez2-5/+60
This new parameter, "securityContext", will allow the user to define the privilege and access control settings for a pod or a container [1]. Example of "securityContext" definition in the pod (if only one container is defined): context:   type: Kubernetes   servers:     host:       image: ...       securityContext:         runAsUser: 1000         fsGroup: 2000  Example of "securityContext" definition in the pod and the container (if several containers are defined):      context:   type: Kubernetes   servers:     host:       securityContext:         runAsUser: 1000         fsGroup: 2000        containers:         - image: ...           securityContext:             allowPrivilegeEscalation: false         - image: ... [1] https://kubernetes.io/docs/tasks/configure-pod-container/security-context/ JIRA: YARDSTICK-1156 Change-Id: I597a300c68cd834522a284b1cca0faa918493342 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Define several containers per pod in a replication controllerRodolfo Alonso Hernandez2-50/+102
Add the ability to define not only one but many containers per pod in a replication controller descriptor. This feature must be backwards compatible; all current test cases using the "single container" server definition must be accepted. Example of single container pod definition: context: type: Kubernetes servers: host: image: ... commands: ... volumes: - name: volume1 # mandatory <volume type definition> # mandatory Example of several container pod definition: context: type: Kubernetes servers: host: containers: # if this key is present, all container specific parameters (image, commands, args, volumeMounts, etc.) must be defined per container - image: ... commands: ... - image: ... commands: ... volumes: - name: volume1 # mandatory <volume type definition> # mandatory NOTE: other parameters, like "volumes" or "nodeSelector", are common to all containers in the pod. JIRA: YARDSTICK-1155 Change-Id: Ib95668c68e9c09e6de3f1aa41c903cc52e6809ad Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add "volumeMounts" parameter in Kubernetes contextRodolfo Alonso Hernandez2-28/+89
This new parameter, "volumeMounts", will allow the user to automatically create new volumes mounted inside the container. Example of Kubernetes context definition: context: type: Kubernetes servers: host: volumeMounts: - name: volume1 # mandatory mountPath: /dev/hugepages # mandatory readOnly: True # optional, default=False JIRA: YARDSTICK-1151 Change-Id: Ic00b45e6d603a9f85e3f0e25becdf0bce14e637c Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add "volumes" parameter in Kubernetes contextRodolfo Alonso Hernandez6-40/+118
This new parameter, "volumes", will allow the user to automatically create new volumes. Example of Kubernetes context definition: context: type: Kubernetes servers: host: image: ... commands: ... volumes: - name: volume1 # mandatory <volume type definition> # mandatory The volume type and the definition must be one of the supported ones in Kubernetes [1]. [1] https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes JIRA: YARDSTICK-1152 Change-Id: I44a91c605f047de4f286407e28fb5aa2e921b00a Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-13assert[Greater,Equal] -> assert_{,not_}calledEmma Foley10-51/+52
assertEqual(mock_xxx.call_count, 1) -> mock_xxx.assert_called_once assertEqual(mock_xxx.call_count, 0) -> mock_xxx.assert_not_called assertGreater(mock.call_count, 0) -> mock.assert_called() assertGreaterEqual(mock.call_count, 1) -> mock.assert_called() JIRA: YARDSTICK-1069 Change-Id: I890084d120c8e78304e169e2a0e5d30011a41525 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-12Improve IXIA IxNetwork library and traffic profile (4)Rodolfo Alonso Hernandez10-277/+324
This patch implements an active wait for the traffic injection. Once the traffic is started, the traffic generator class will poll periodically the IXIA traffic generator chassis to retrieve the status of the traffic ("started", "stopped"). Now the latency statistics are retrieved and reported for each injection period. JIRA: YARDSTICK-1116 Change-Id: I4422e2c88b4fc97b7cac3de8a82b2d75467c4117 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-12Improve IXIA IxNetwork library and traffic profile (3)Rodolfo Alonso Hernandez3-96/+157
This patch modifies IP packet parameters. "IxNextgen.update_ip_packet" modifies the L3 packet according to the test case and setup the IP addresses. JIRA: YARDSTICK-1116 Change-Id: I46ff75ab1989d0e6f5cc876418a015386717e06f Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-12Improve IXIA IxNetwork library and traffic profile (2)Rodolfo Alonso Hernandez7-1158/+621
This patch modifies the way the packet frame and the flow is configured before the injection. "IxNextgen.update_frame" modifies the L2 frame according to the test case and setup the frame rate, frame size, traffic injection duration and MAC addresses. JIRA: YARDSTICK-1116 Change-Id: Ife08f15a4eda24d7835c92c4172b450854d112ee Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-12Improve IXIA IxNetwork library and traffic profile (1)Rodolfo Alonso Hernandez4-244/+432
This patch introduces a new way to configure the TCL IxNetwork server. All the configuration is done using the TCL API, removing the need of using the pre-saved configuration file. "IxNextgen.assign_ports" creates and assigns the virtual ports for each physical port defined in the test case. "IxNextgen.create_traffic_item" creates one traffic item and two flow groups per pair of ports, in both directions. "IxNextgen.gt_statistics" retrieves only the required statistics to generate the samples blob in the traffic generator. JIRA: YARDSTICK-1116 Change-Id: I8f1c0c55e99c274b2ed8276ed9a385c502e16d93 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-06-12Merge changes from topics 'YARDSTICK-1218', 'YARDSTICK-1216', ↵Rodolfo Alonso Hernandez14-178/+161
'YARDSTICK-1215', 'YARDSTICK-1214' * changes: Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptions Move ErrorClass definition to exceptions module Convert SSH custom exceptions to Yardstick exceptions Remove AnsibleCommon class method mock
2018-06-11Cleanup CGNAPT unit testsRodolfo Alonso Hernandez1-123/+70
Remove unneeded mocks, clean console output and improve the test execution. JIRA: YARDSTICK-1226 Change-Id: I83e91028c17208a39d3813e85b44c4df1961e01e Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-11Cleanup process test casesRodolfo Alonso Hernandez1-4/+6
Remove logging console output during test execution. JIRA: YARDSTICK-1234 Change-Id: I8ecab0219ffd1919f2fa97a90d5be11018b11322 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-11Cleanup OpenStack utils test casesRodolfo Alonso Hernandez1-32/+35
Remove logging console output during test execution. JIRA: YARDSTICK-1230 Change-Id: I1b61f0f18bfb6e5a63c9027537cfb736261c4926 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-11Move tests: unit/network_services/vnf_genericEmma Foley29-251/+306
* Fix pylint errors * Add TODOs Some errors are ignored locally, as they were a symptom of other problems. These issues have been flagged with a TODO, and should be fixed later. JIRA: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Change-Id: I3d95537a64bde923703421510d27bac0d6dd92bd
2018-06-08Refactor "utils.parse_ini_file" testingRodolfo Alonso Hernandez1-17/+93
JIRA: YARDSTICK-1218 Change-Id: I5136b22110d215beb7af72998ad47a8dba27f63b Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-08Merge "Remove print out of logger exception in TestUtils"Rodolfo Alonso Hernandez1-7/+10
2018-06-08Merge "Cleanup TestYardstickNSCli class"Rodolfo Alonso Hernandez1-18/+32
2018-06-08Merge "Remove __init__ method overriding in HeatContextTestCase"Rodolfo Alonso Hernandez1-7/+4
2018-06-08Merge "Standalone XML machine type is not longer valid"Rodolfo Alonso Hernandez1-1/+1
2018-06-08Merge "Error when adding SR-IOV interfaces in SR-IOV context"Rodolfo Alonso Hernandez2-12/+18
2018-06-08Remove print out of logger exception in TestUtilsRodolfo Alonso Hernandez1-7/+10
Remove print out of logger exception while executing "TestUtils.test_parse_ini_file_missing_section_header" test case. JIRA: YARDSTICK-1225 Change-Id: I2a4e447c5d119fb9585b477d74f3d0a002c8278e Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-07Cleanup TestYardstickNSCli classRodolfo Alonso Hernandez1-18/+32
Remove the console output during the test execution. Refactor the "parse_arguments" test case. JIRA: YARDSTICK-1224 Change-Id: I68954b986dbea70752adb121e0445e162bdfddcb Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-07Merge "Fix latency in l2 and l3 PROX generator config files"Abhijit Sinha4-20/+85
2018-06-07Merge "Use TRex release v2.41 to support both x86 and aarch64"Rodolfo Alonso Hernandez2-2/+3
2018-06-07Merge "Remove instantiated contexts in "test_task""Rodolfo Alonso Hernandez1-0/+9
2018-06-07Fix latency in l2 and l3 PROX generator config filesXavier Simonart4-20/+85
JIRA: YARDSTICK-1211 Change-Id: I0899d470ca5a0ec7d42d36a9ff9c39cc7369310c Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
2018-06-06Merge "Add NSB PROX Baremetal dashboard for L3"Abhijit Sinha2-1151/+9973
2018-06-06Merge "Fix NSB NFVi metrics accuracy"Abhijit Sinha6-62/+82
2018-06-05Merge "Bump oslo.messaging version to 5.36.0"Rodolfo Alonso Hernandez1-1/+1
2018-06-05Fix NSB NFVi metrics accuracyXavier Simonart6-62/+82
JIRA: YARDSTICK-1212 Change-Id: Ic4d4a3c00e4e278f4de06cc176ff663892895569 Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
2018-06-01Move IncorrectConfig, IncorrectSetup and IncorrectNodeSetup to exceptionsRodolfo Alonso Hernandez7-67/+50
JIRA: YARDSTICK-1218 Change-Id: I804065e9bce3e728f5bf9e756a78df8fd28f74ac Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>