aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/orchestrator
AgeCommit message (Collapse)AuthorFilesLines
2018-07-10Merge "Add network elements as a dict in Kubernetes context"Rodolfo Alonso Hernandez1-5/+6
2018-07-10Add "tolerations" parameter in Kubernetes contextRodolfo Alonso Hernandez1-1/+17
This new parameter, "torelations", is applied to pods and allow (but not require) the pods to schedule onto node with matching tains [1]. Example of "tolerations" definition in a Kubernetes pod: spec: containers: [...] tolerations: - key: <key defined in the taint> value: <key value to match with the taint> effect: <effect in case of match> operator: <matching operator> - key: ... Example of "tolerations" definition in a Yardstick test case: context: type: Kubernetes servers: host: containers: [...] tolerations: - key: ... value: ... effect: ... operator: ... NOTE: if any toleration is defined, a default one will be applied in order to allow any replication controller to create the pods in any Kubernetes node. This default toleration is defined as: spec: tolerations: - operator: "Exists" [1] https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ JIRA: YARDSTICK-1254 Change-Id: I32fb9c7086b4218c323218738057f634eb6ffff4 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-10Add "ports" parameters in Kubernetes contextJohn O'Loughlin1-1/+12
This new parameter, "ports", will allow the user to automatically to add port variables in a pod definition: context: type: Kubernetes servers: host: image: ... commands: ... ports: - containerPort: <port value> # Mandatory hostIP: <host IP> # Optional hostPort: <host port> # Optional name: <port name> # Optional protocol: <protocol name> # Optional The port type and the definition must be one of the supported ones in Kubernetes [1]. Container port definition, V1 API, Kubernetes Python library v6.0.0 [2] [1] https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/ [2] https://github.com/kubernetes-client/python/blob/6.0.0/kubernetes/client/models/v1_container_port.py JIRA: YARDSTICK-1183 Change-Id: I3673dbeec8d380cb2f8b1dacf6c25c7cdbc37e28 Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
2018-07-10Add "restartPolicy" parameter in Kubernetes policyJohn O Loughlin1-1/+7
This new parameter, "restartPolicy", will allow define the restart policy per pod. Example of yaml definition in Kubernetes: apiVersion: v1 kind: Pod metadata: name: frontend spec: containers: - name: db ... restartPolicy: Always # Possible values: "Always", "OnFailure" and "Never" Example of definition in a Yardstick context: context: type: Kubernetes servers: host: containers: - name: ... restartPolicy: Always # Default value: "Always" Restart policy [1]. [1] https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy JIRA: YARDSTICK-1175 Change-Id: Id4317b909f98422e89d6d4553e8cfb0e1f593355 Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
2018-07-06Add network elements as a dict in Kubernetes contextRodolfo Alonso Hernandez1-5/+6
Like in Heat context, network elements should be defined in a dictionary, using the network name as the entry key. Previous implementation: context: networks: - name: ... plugin: ... args: ... New implementation: context: networks: net_name_1: plugin: ... args: ... net_name_2: ... JIRA: YARDSTICK-1160 Change-Id: I786b06555af509440604305e2c3c2c9fe521bef8 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-07-06Add "resources" parameter in Kubernetes contextJohn O Loughlin1-0/+7
This new parameter, "resources", will allow the user to automatically add resources in a pod definition per container. Example of yaml definition in Kubernetes: apiVersion: v1 kind: Pod metadata: name: frontend spec: containers: - name: db image: mysql resources: requests: memory: "64Mi" cpu: "250m" limits: memory: "128Mi" cpu: "500m" Example of definition in a Yardstick context: context: type: Kubernetes servers: host: resources: # There are two possible keys in this dict: # "requests" and "limits". limits: <dictionary> requests: <dictionary> Resources in containers [1]. [1] https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/ JIRA:YARDSTICK-1171 Change-Id: I163d72d9d3e88b538fca73eb4dbf38613aa23ff4 Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
2018-07-06Add "env" parameter in Kubernetes contextJohn O Loughlin1-0/+6
This new parameter, "env", will allow the user to automatically to add environment variables in a pod definition Example of definition in a context: context: type: Kubernetes servers: host: image: ... commands: ... env: - <variable name>: <variable value> - <variable name>: <variable value> The volume type and the definition must be one of the supported ones in Kubernetes [1]. [1] https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes JIRA: YARDSTICK-1161 Change-Id: Icf372a98c090063f10c9217755519ab92ba9b0ba Signed-off-by: John O Loughlin <john.oloughlin@intel.com>
2018-06-14Create user defined "NodePort" services per podRodolfo Alonso Hernandez1-20/+37
This feature will give the user the ability to create user defined "NodePort" services per pod, a part from the default SSH port created, by default, for each pod created. Example of Kubernetes yaml definition: apiVersion: v1 kind: Servicemeta data: name: pod-1-service-nodeport spec: type: NodePort ports: - name: web port: 80 targetPort: 8888 nodePort: 33333 clusterIP: 10.254.0.8 selector: app: pod-1 Example of Yardstick test case definition: context: type: Kubernetes servers: host: containers: - image: ... securityContext: allowPrivilegeEscalation: false - image: ... node_ports: - port: <port number> # Mandatory name: <port name> # Optional targetPort: <port name> # Optional, default: targetPort=port nodePort: <port number> # Optional, assigned by Kubernetes Kubernetes service, type "NodePort" [1] [1] https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport JIRA: YARDSTICK-1181 Change-Id: Ia7730e4569d5526d573402a98c27d55c5c82cdef Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Change "KubernetesObject" class name to "ReplicationController"Rodolfo Alonso Hernandez1-6/+5
The class name "KubernetesObject" is not accurate. This class holds the definition of a Kubernetes replication controller. The suggested name is therefore "ReplicationControllerObject". JIRA: YARDSTICK-1180 Change-Id: I9ee9228f9528cb779e54a9bc178760b6e24c85c2 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Specify the networks to be used per podRodolfo Alonso Hernandez1-1/+18
If CRD "Network" is defined and network items are created, each pod (server) can have access to one or several networks. This is defined in the metadata section, as "annotations.networks" [1]. Example of Kubernetes pod definition with networks: apiVersion: v1 kind: Pod metadata: name: test-pod annotations: networks: '[{"name": "flannel"}]' Example of Yardstick server definition with networks: context: type: Kubernetes servers: host: containers: - name: ... networks: - flannel # These names must be defined in # context.networks ... networks: - name: flannel plugin: flannel Kubernetes annotations [2]. [1]https://github.com/intel/multus-cni/tree/b9446232cdf4f1b6f2bea583291973cc97e963f4#configuring-multus-to-use-kubeconfig-and-a-default-network [2]https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/ JIRA: YARDSTICK-1178 Change-Id: I6e7b4bacf10810833ec733c14d44e5db613675e3 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add new Kubernetes resource kind: "Network"Rodolfo Alonso Hernandez1-4/+95
Add a new Kubernetes resource kind: "Network" [1] [2] Kubernetes network plugins (alpha definition): [3] Network definition example in Kubernetes: apiVersion: "kubernetes.com/v1" kind: Network metadata: name: flannel plugin: flannel args: '[ {delegate": {"isDefaultGateway": true}} ]' Proposed Kubernetes context network definition: context: networks: - name: flannel plugin: flannel args: (string) - name: sriov_upload plugin: sriov args: (string) [1]https://builders.intel.com/docs/networkbuilders/multiple-network-interfaces-in-kubernetes-application-note.pdf [2]http://cdn.opensourcecloud.cn/zt/k8s/01.pdf [3]https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/ JIRA: YARDSTICK-1160 Change-Id: I71a49ac14e3d28ded91d2ed3cd9cc527e40303f7 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add new Kubernetes resource kind: "CustomResourceDefinition"Rodolfo Alonso Hernandez1-0/+45
Custom resource definition example in Kubernetes: apiVersion: "apiextensions.k8s.io/v1beta" kind: CustomResourceDefinition metadata: name: networks.kubernetes.com spec: group: kubernetes.com version: v1 scope: Namespaced names: plural: networks singular: network kind: Network Proposed Kubernetes context network definition: context: custom_resources:     - name: network        # name of the resource (singular)       version: v1          # optional, "v1" by default       scope: Namespaced    # optional, "Namespaced" by default From this definition, we will extract the Kubernetes parameters: - metadata.name: custom_resources.name + "s" + context_name + ".com" - spec.group: context_name + ".com" - spec.scope: custom_resources.scope - spec.version: custom_resources.version - spec.names.plural: custom_resources.name + "s" - spec.names.singular: custom_resources.name - spec.names.kind: custom_resources.name with first capital letter [1] https://kubernetes.io/docs/concepts/api-extension/custom-resources/ JIRA: YARDSTICK-1163 Change-Id: If8980dc3f6ddf9c6949bf15be8011aa98482ddc9 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add "securityContext" parameter in Kubernetes contextRodolfo Alonso Hernandez1-5/+17
This new parameter, "securityContext", will allow the user to define the privilege and access control settings for a pod or a container [1]. Example of "securityContext" definition in the pod (if only one container is defined): context:   type: Kubernetes   servers:     host:       image: ...       securityContext:         runAsUser: 1000         fsGroup: 2000  Example of "securityContext" definition in the pod and the container (if several containers are defined):      context:   type: Kubernetes   servers:     host:       securityContext:         runAsUser: 1000         fsGroup: 2000        containers:         - image: ...           securityContext:             allowPrivilegeEscalation: false         - image: ... [1] https://kubernetes.io/docs/tasks/configure-pod-container/security-context/ JIRA: YARDSTICK-1156 Change-Id: I597a300c68cd834522a284b1cca0faa918493342 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Define several containers per pod in a replication controllerRodolfo Alonso Hernandez1-34/+53
Add the ability to define not only one but many containers per pod in a replication controller descriptor. This feature must be backwards compatible; all current test cases using the "single container" server definition must be accepted. Example of single container pod definition: context: type: Kubernetes servers: host: image: ... commands: ... volumes: - name: volume1 # mandatory <volume type definition> # mandatory Example of several container pod definition: context: type: Kubernetes servers: host: containers: # if this key is present, all container specific parameters (image, commands, args, volumeMounts, etc.) must be defined per container - image: ... commands: ... - image: ... commands: ... volumes: - name: volume1 # mandatory <volume type definition> # mandatory NOTE: other parameters, like "volumes" or "nodeSelector", are common to all containers in the pod. JIRA: YARDSTICK-1155 Change-Id: Ib95668c68e9c09e6de3f1aa41c903cc52e6809ad Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add "volumeMounts" parameter in Kubernetes contextRodolfo Alonso Hernandez1-23/+35
This new parameter, "volumeMounts", will allow the user to automatically create new volumes mounted inside the container. Example of Kubernetes context definition: context: type: Kubernetes servers: host: volumeMounts: - name: volume1 # mandatory mountPath: /dev/hugepages # mandatory readOnly: True # optional, default=False JIRA: YARDSTICK-1151 Change-Id: Ic00b45e6d603a9f85e3f0e25becdf0bce14e637c Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-06-14Add "volumes" parameter in Kubernetes contextRodolfo Alonso Hernandez1-19/+35
This new parameter, "volumes", will allow the user to automatically create new volumes. Example of Kubernetes context definition: context: type: Kubernetes servers: host: image: ... commands: ... volumes: - name: volume1 # mandatory <volume type definition> # mandatory The volume type and the definition must be one of the supported ones in Kubernetes [1]. [1] https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes JIRA: YARDSTICK-1152 Change-Id: I44a91c605f047de4f286407e28fb5aa2e921b00a Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-05-09Avoid "volumeMounts" with "configMap" fixed permissionsRodolfo Alonso Hernandez1-1/+1
To access to the container without using a password, the jumphost RSA public key is copied to each container, using "volumeMounts" defined as "configMap", to /root/.ssh/authorized_keys. To work properly, the following permissions must be set: - /root/.ssh: 700 - /root/.ssh/authorized_keys: 600 Because of [1][2], the mounted folders have fixed permissions and cannot be modified. [1]https://groups.google.com/forum/#!topic/kubernetes-dev/eTnfMJSqmaM [2]https://github.com/kubernetes/kubernetes/issues/28317 JIRA: YARDSTICK-1149 Change-Id: I821064da56699c5b4f509d233c33e55af119fd56 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-05-03Add "os_cloud_config" as a new context flag parameterRodolfo Alonso Hernandez1-7/+10
This new parameter will contain the OpenStack cloud specific configuration used by Shade client. This new flag is used only in Heat context. By default, this new parameter (dict) will contain this content: 'os_cloud_config': {'verify': False} This field will be used by HeatStack [1] to create a Shade cloud. Shade retrieves, if not defined, the OpenStack configuration from "os_client_config". This configuration is used to generate the cloud configuration, which is the description of the OpenStackCloud returned. The default parameter defined, "verify", refers to the related bug. By default, in case of using SSL certificate it will not be verified. [1] https://github.com/opnfv/yardstick/blob/b338d3091bb0beb89d4ad9f7c144f43a31a19a74/yardstick/orchestrator/heat.py#L47 JIRA: YARDSTICK-1139 Change-Id: I875a7018401b84e51dab775b8194174645d27e06 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-03-27restore Heat stack failure logs for CIRoss Brattain1-0/+9
We need to see what Heat failures occured so that we can debug CI failures. The only way to do this seems to be to use the private event_utils.get_events function to query for all failures Also since we had a failure go ahead and dump the Heat template that failed. Sample output: 2018-03-25 23:50:08,765 [INFO] yardstick.orchestrator.heat heat.py:629 Creating stack 'yardstick-460ed969' START 2018-03-25 23:50:21,932 [ERROR] yardstick.orchestrator.heat heat.py:644 Resource CREATE failed: BadRequest: resources.yardstick-460ed969-xe1: Invalid input for operation: physical_network 'nosuch' unknown for flat provider network. Neutron server returns request_ids: ['req-6f981f1e-a9e2-4114-af84-1ee528aed51b'] 2018-03-25 23:50:21,933 [ERROR] yardstick.orchestrator.heat heat.py:644 BadRequest: resources.yardstick-460ed969-xe1: Invalid input for operation: physical_network 'nosuch' unknown for flat provider network. Neutron server returns request_ids: ['req-6f981f1e-a9e2-4114-af84-1ee528aed51b'] 2018-03-25 23:50:21,972 [ERROR] yardstick.orchestrator.heat heat.py:645 {'description': '\n' 'All referred generated resources are prefixed with the ' 'template\n' 'name (i.e. yardstick-460ed969).\n', 'heat_template_version': '2013-05-23', 'outputs': {'trafficgen_1.yardstick-460ed969': {'description': 'VM UUID', 'value': {'get_resource': 'trafficgen_1.yardstick-460ed969'}}, 'trafficgen_1.yardstick-460ed969-fip': {'description': 'floating ' 'ip ' JIRA: YARDSTICK-998 Change-Id: Ia8f4e5ba7e280fb9086519680d5ee90a2b442e6b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-03-06Merge "Enable heat context to support existing network"Jack Chan1-8/+14
2018-03-01Merge "Add methods to get an existing stack"Ross Brattain1-5/+22
2018-03-01Merge "Use context name instead of key_uuid"Ross Brattain1-8/+2
2018-03-01Add methods to get an existing stackEmma Foley1-5/+22
* Add yardstick/orchestrator/heat.py:HeatStack.get * Add yardstick/benchmark/contexts/heay.py:HeatContext.retrieve_existing_stack JIRA: YARDSTICK-886 Change-Id: I6974b79a25f98066a49b1bc8ccd11383e7962091 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-03-01Use context name instead of key_uuidEmma Foley1-8/+2
context.key_uuid is only used as a suffix for the key files. There is already a unique ID associated with a the context; the context name includes a task ID, which is a UUID. This patch sets context.key_uuid to to context.name instead of generating a separate UUID As a result, get_short_key_uuid() is not needed. JIRA: YARDSTICK-1028 Change-Id: Id175061d6cfe23a068bb3d12ce176c1f176e8236 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2018-02-27add egress security rulesmting1-1/+30
JIRA: YARDSTICK-1034 Change-Id: I613b8bff669fceeb0c7c59dec2b6f1ab22b09247 Signed-off-by: mting <dianfeng.du@ericsson.com>
2018-02-27Enable heat context to support existing networkrexlee87761-8/+14
JIRA: YARDSTICK-970 Change-Id: I593af4b3a30e52521e3d77fd84bec16ab4733483 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2018-02-15Workaround to fix Heat stack deletion bug in ShadeRodolfo Alonso Hernandez1-1/+8
There is a bug in Shade, in the function "event_utils.poll_for_events". While Shade is waiting for the Heat stack to be deleted, retrieves periodically the status of this Heat stack. When the stack is deleted, the function "get_stack" [1] returns None; then L90 tries to access to a key of this None object. [1]https://github.com/openstack-infra/shade/blob/e5bcb705c5b32334cdc1dd9807126298c87b0622/shade/_heat/event_utils.py#L89-L90 Shade function JIRA: YARDSTICK-1015 Change-Id: I56ea853be41bfd7f63e094d7a3696d97ba078ba4 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-02-12Heatstack create: fix str/bytes error in NamedTemporaryFile.write()Ross Brattain1-1/+1
we need to use jsonutils.dump_as_bytes for NamedTemporaryFile.write() 2018-02-11 19:15:01,366 [INFO] yardstick.orchestrator.heat heat.py:563 Creating stack 'yardstick-4359f33e' START 2018-02-11 19:15:01,935 [ERROR] yardstick.benchmark.contexts.heat heat.py:317 stack failed Traceback (most recent call last): File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/heat.py", line 313, in deploy timeout=self.heat_timeout) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/orchestrator/heat.py", line 567, in create stack.create(self._template, self.heat_parameters, block, timeout) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/orchestrator/heat.py", line 56, in create template_file.write(jsonutils.dumps(template)) File "/home/rbbratta/yardstick_venv3/lib/python3.5/tempfile.py", line 622, in func_wrapper return func(*args, **kwargs) TypeError: a bytes-like object is required, not 'str' JIRA: YARDSTICK-1005 Change-Id: I504f23b86119b62f3aea5b83a445b97bf810220c Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-02-09Replace "python-heatclient" with "shade" clientRodolfo Alonso Hernandez1-144/+67
Replaced "python-heatclient" with "shade" client. The "python-heatclient" library is removed from requirements file. "shade" client version 1.22.2 is the required one for OpenStack Pike release. shade is the recommended client to interact with OpenStack clouds [1] [1] https://github.com/openstack-infra/shade JIRA: YARDSTICK-880 Change-Id: Ibbdfc42a543d2d9ed4cf70d78de3de8cbecd0076 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-11-10Merge "Heat: add availability_zone"Ross Brattain1-5/+6
2017-11-09Addition of IPv6 icpm, udp and tcp to Neutron Security GroupDanielMartinBuckley1-2/+15
When creating a Neutron security group for a heat template. Add IPv6 security group rules to access the resouces. Change-Id: I0b541dfe400121e35908a70778665ee1576f0622 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2017-11-01Heat: add availability_zoneRoss Brattain1-5/+6
Users would like to ensure placement of VMs on specific compute nodes so that the measurements are meaningful. Examples: Measure network performance in different scenarios (VMs in same host, in different hosts, across fabric, across tenants) Example: context: name: yardstick placement_groups: pgrp1: policy: "availability" servers: tg_0: floating_ip: true placement: "pgrp1" availability_zone: "zone2" vnf_0: floating_ip: true placement: "pgrp1" availability_zone: "zone1" Change-Id: I28a757c25ae3f5b3571ab3edd82d51ceba32c302 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-26Merge "Add nodeSelector to enable selecting the desired Kubernetes running ↵Rex Lee1-1/+9
node when creating containers for Kubernetes context"
2017-09-21Add service in kubernetes contextchenjiankun1-1/+32
JIRA: YARDSTICK-803 Currently kubernetes test case can only run in master node. We need to support it run in jump server. So I add service and use nodePort type. Then we can login the pod using nodePort. Change-Id: Ia7900d263f1c5323f132435addec27ad10547ef9 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-09-21Add nodeSelector to enable selecting the desiredTrevor Tao1-1/+9
Kubernetes running node when creating containers for Kubernetes context For example, a yaml file may looks like: servers: host: image: xxx command: /bin/bash nodeSelector: xxx: yyy Synchronously change the unit test for this function Change-Id: If74c9dad9b1a70395bb79f34708a0fde04e7e650 Signed-off-by: Trevor Tao <trevor.tao@arm.com>
2017-09-14improve logging, clear using printrexlee87761-7/+7
Change-Id: I744353f631cf1771d75f750543e8612f81be71ee Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-08-25Enabling ACL test for provider network.Maciej Skrocki1-2/+4
Properly re-using network_type from model. Change-Id: I40180eb53080a360b4099c034038eac67716f8d1 Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com>
2017-08-08NSB updateDeepak S1-1/+3
Refactored main NSB VNF classes accroding to class diagram https://wiki.opnfv.org/display/yardstick/NSB+class+diagram All the SampleVNFs have been separated and placed under the SampleVNF class. Added AutoConnectSSH to automatically create SSH conneciton on demand. Added VnfdHelper class to wrap the VNFD dictionary in prepartion for class-based modeling. Extracted DpdkVnfSetupEnvHelper for DPDK based VNF setup. Extracted Stats and other client config to ResourceHelper Had to replace dict_key_flatten with deepgetitem due to Python 2.7 Jinja2 infinite recursion. Change-Id: Ia8840e9c44cdbdf39aab6b02e6d2176b31937dc9 Signed-off-by: Deepak S <deepak.s@linux.intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-03fix heatclient importRoss Brattain1-1/+1
for some reason, maybe apexlake, we didn't do the correct import we need to import heatclient.client directly Change-Id: I682c88ae780845adb0b5aa898390697197e3af5d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-29Heat: support vnic_type in heat type contextJingLu51-4/+5
JIRA: YARDSTICK-757 Some test scenarios require VM with specific vnic type. This work is about supporting using different vnic types in heat type context. context: name: demo image: cirros-0.3.5 flavor: yardstick-flavor user: cirros placement_groups: pgrp1: policy: "availability" servers: athena: floating_ip: true placement: "pgrp1" ares: placement: "pgrp1" networks: test: cidr: '10.0.1.0/24' vnic_type: "normal" Change-Id: Ia229fda72f47b04288ea107e2d58fd3e8ac91dd9 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-29Heat: support create and attach volume in heat type contextJingLu51-0/+34
JIRA: YARDSTICK-756 Some test scenarios require VM with volume attached. This work is about supporting create and attach volume in heat type context. context: name: demo image: cirros-0.3.5 flavor: yardstick-flavor user: cirros placement_groups: pgrp1: policy: "availability" servers: athena: floating_ip: true # per-vm inline volume definition. if no volume size specified, then this # volume should be an existing volume in the openstack environment volume: yardstick-volume placement: "pgrp1" ares: # per-vm inline volume definition. if volume size is specified, then this # volume will be crated and attach to the vm volume: name: test-volume size: 10 # volume mountpoint is also configurable volume_mountpoint: /dev/vdb placement: "pgrp1" networks: test: cidr: '10.0.1.0/24' Change-Id: Ief87b313980a59eac229eb4780d93ffc929ceb66 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-27leave port_security_enabled undefined by defaultRoss Brattain1-3/+5
some deployments don't have port security ML2 plugin enabled, so we have to leave port security undefined by default RuntimeError: Resource CREATE failed: BadRequest: resources.yardstick-TC072-19be2f60-test: Unrecognized attribute(s) 'port_security_enabled' https://build.opnfv.org/ci/user/narindergupta/my-views/view/joid/job/yardstick-joid-baremetal-daily-master/1276/console Change-Id: I69178f869f6215f42d90ea4cfb297dad3d799f83 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-19heat: port_security_enabled, allowed_address_pairs, disable gateway_ipRoss Brattain1-12/+24
xe0: cidr: '10.0.2.0/24' vld_id: public allowed_address_pairs: - ip_address: '0.0.0.0/0' xe1: cidr: '10.0.3.0/24' vld_id: private allowed_address_pairs: - ip_address: '0.0.0.0/0' We need to disable gateways on data plane interfaces With TGs and VFNs we probably don't need gateway on data interfaces Change-Id: I276a7d591a0a3ff03877c4f525303fc2874de683 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-17Merge "Kubernetes (k8s) support"Jing Lu1-0/+130
2017-07-17Kubernetes (k8s) supportchenjiankun1-0/+130
JIRA: YARDSTICK-682 We decide to support k8s in E release. We need to discuss with openretriver team and then rewrite the ping test case under k8s as the first step. Change-Id: I3f81ebca8de5c1f3a8b7d42581cd7342dc320239 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-13add network info to topologyRoss Brattain1-2/+6
we need to know which network each port is connected to so we can find VLAN or VXLAN ID. To do this we implement a new method for Contexts, Context.get_network(). This method is similar to Context.get_server(), it searches for a given network name in all the contexts. From this we generate a context_cfg["networks"] dict that stores all the network info for the nodes in the scenario. Then when we generate the topology for VNFD, we can lookup a given network by the vld_id and get the network_type, segmentation_id, etc. Then if we need to for example generated traffic on a given VLAN or VXLAN, we have this info available. Define default nd_route_tbl for ACL VNF we need default empty nd_route_tbl for IPv6 route. Change-Id: I9f9cfbd6acabeb4ae4675ca7354390efa57b29e7 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-07-06Improvement: delete stack if create stack failedJingLu51-2/+3
JIRA: YARDSTICK-709 If the stack is CREATE_FAILED during the context deployment, the failed stack should be deleted. Change-Id: I35b28ffae0f4d1f3ea9809d21add5d718dae5c9d Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-04support segmentation_id for vlan provider networkJingLu51-2/+6
If a network specified in the test case is a vlan provider network, new attribute "segmentation_id" can be used to specify the desired vlan tag in the test case network section: networks: test-net: cidr: '192.168.1.0/24' provider: "vlan" physical_network: 'physnet1' segmentation_id: "1000" If the "segmentation_id" attribute is absent, a random vlan tag will be allocated to the network. Change-Id: Ic53852447a3c1bd8feb9ebd42d35f1ade3684be1 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-20Acquire NSB specific data from Heat.Edward MacGillivray1-42/+80
First we add mac_address, subnet_cidr to Heat template outputs Then we convert those into a form for NSB and add vld_id. NSB also requires PCI Bus ID, kernel driver and dpdk_port_num. We get this by ssh-ing into instance and dumping sysfs We also need to fix allow for ssh key auth, and implement relative path file loading so NSB can find all its YAML files JIRA: YARDSTICK-580 Change history: don't hide heat create tracebacks we need tracebacks for debug vnf_generic: add task_path to scenario so we can load relative paths for vnf_generic we want to be able to load yaml relative to the task path For example: traffic_profile: ../../traffic_profiles/fixed.yaml topology: ping_tg_topology.yaml # TODO: look in relative path where the tc.yaml is found These need to be relative to samples/vnf_samples/nsut/ping/tc_ping_heat_context.yaml Add a scenario["task_path"] entry heat: log actual exception vnf_generic: replace list with set and iterate over values() some general refactors to remove redundact lookups and type conversions heat: provide mac_address, device_id and network_id from outputs We may need more information to dynamically determine test topology. Towards this end return more info in the heat template. We can return mac_address, device_id and network_id. Once we have this info we can add it to the context_cfg as an interfaces dict. add sample vnf ping multi-network test this test requires 3 network, one for mgmt and the other two for NSB traffic tests We have to make sure we don't use DPDK on mgmt interface because DPDK unbinds the driver heat: convert networks to OrderedDict so we can lookups networks as well as iterate over them in consisitent order heat: and vld_id to networks for vnf_generic vnf_generic uses vld_id Virtual Link Descriptor ID to identify interfaces Add the key to the networks dict and store in Networks object implement relative path file loading in vnf_generic in multiple places we need to load a file relative to the task path, so add open_relative_file_path and modify load_vnf_model to include the scenario_cfg parameter so we have access to task_path DRAFT: heat timeout support Heat stack in CI job failed due to some Nova issue. But then apparently yardstick kept running and took 180mins to timeout https://build.opnfv.org/ci/view/bottlenecks/job/bottlenecks-compass-posca_stress_ping-baremetal-daily-master/16/console We can add a Heat create timeout and fail faster if there is an error. The question is how long should we wait for a Heat stack to deploy. We can set a default and allow override in the heat context config, if users make complicated stacks heat: get netmask and gateway from heat outputs we have do some tricky business with finding the subnet cidr and converting it into netmask vnf_generic: get vpci, driver and dpdk_port_num use a big old find command to dump all the sysfs netdev info nicely. This was re-used from autotest FCoE tests. r"""find /sys/devices/pci* -type d -name net -exec sh -c '{ grep -sH ^ \ +$1/ifindex $1/address $1/operstate $1/device/vendor $1/device/device \ +$1/device/subsystem_vendor $1/device/subsystem_device ; \ +printf "%s/driver:" $1 ; basename $(readlink -s $1/device/driver); } \ +' sh \{\}/* \; This finds all PCI devices that are network devices, then dumps all the relevant info using /bin/sh. Then we parse this into a 'netdevs' dict inside the node_dict and also convert into VNF fields we need. vnf_generic: set node name for kpis node is a dict, so we have to use node_name vnfdgen: we CANNOT use TaskTemplate.render because it does not allow for missing variables, we need to allow password for key_filename to be undefined remove default ssh password hack, once rendering is fixed add new example tc_external_ping_heat_context Change-Id: If1fe0c1a2ab0a5be17e40790a66f28f706fa44d6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-05-12Create flavor from heat contextDanielMartinBuckley1-11/+55
JIRA: YARDSTICK-582 Create a customizable flavor via heat context. All heat parameters are configurable including Core Affinity. The default flavor name is XXXX-flavor where XXXX is stackname. Flavor attributes are taken from the heat context file. If a flavor attribute is not used it takes default attribute value. If flavor name is not specified it uses the server name + "-flavor" or stack-name + "-flavor". Compute node specific attributes are configurable via "extra_specs" attribute. See https://docs.openstack.org/admin-guide/compute-flavors.html for details. Change-Id: If4015970b889b0b95bfa8eba9491ebf31e92f2c7 Signed-off-by: DanielMartinBuckley <daniel.m.buckley@intel.com>