Introduction ============= Yardstick is a project dealing with performance testing. Yardstick produces its own test cases but can also be considered as a framework to support feature project testing. Yardstick developed a test API that can be used by any OPNFV project. Therefore there are many ways to contribute to Yardstick. You can: * Develop new test cases * Review codes * Develop Yardstick API / framework * Develop Yardstick grafana dashboards and Yardstick reporting page * Write Yardstick documentation This developer guide describes how to interact with the Yardstick project. The first section details the main working areas of the project. The Second part is a list of “How to” to help you to join the Yardstick family whatever your field of interest is. Where can I find some help to start? -------------------------------------- .. _`user guide`: http://artifacts.opnfv.org/yardstick/danube/1.0/docs/stesting_user_userguide/index.html .. _`wiki page`: https://wiki.opnfv.org/display/yardstick/ This guide is made for you. You can have a look at the `user guide`_. There are also references on documentation, video tutorials, tips in the project `wiki page`_. You can also directly contact us by mail with [Yardstick] prefix in the title at opnfv-tech-discuss@lists.opnfv.org or on the IRC chan #opnfv-yardstick. Yardstick developer areas ========================== Yardstick framework -------------------- Yardstick can be considered as a framework. Yardstick is release as a docker file, including tools, scripts and a CLI to prepare the environement and run tests. It simplifies the integration of external test suites in CI pipeline and provide commodity tools to collect and display results. Since Danube, test categories also known as tiers have been created to group similar tests, provide consistant sub-lists and at the end optimize test duration for CI (see How To section). The definition of the tiers has been agreed by the testing working group. The tiers are: * smoke * features * components * performance * vnf How Todos? =========== How Yardstick works? --------------------- The installation and configuration of the Yardstick is described in the `user guide`_. How to work with test cases? ---------------------------- **Sample Test cases** Yardstick provides many sample test cases which are located at "samples" directory of repo. Sample test cases are designed as following goals: 1. Helping user better understand yardstick features(including new feature and new test capacity). 2. Helping developer to debug his new feature and test case before it is offical released. 3. Helping other developers understand and verify the new patch before the patch merged. So developers should upload your sample test case as well when they are trying to upload a new patch which is about the yardstick new test case or new feature. **OPNFV Release Test cases** OPNFV Release test cases which are located at "tests/opnfv/test_cases" of repo. those test cases are runing by OPNFV CI jobs, It means those test cases should be more mature than sample test cases. OPNFV scenario owners can select related test cases and add them into the test suites which is represent the scenario. **Test case Description File** This section will introduce the meaning of the Test case description file. we will use ping.yaml as a example to show you how to understand the test case description file. In this Yaml file, you can easily find it consists of two sections. One is “Scenarios”, the other is “Context”.:: --- # Sample benchmark task config file # measure network latency using ping schema: "yardstick:task:0.1" {% set provider = provider or none %} {% set physical_network = physical_network or 'physnet1' %} {% set segmentation_id = segmentation_id or none %} scenarios: - type: Ping options: packetsize: 200 host: athena.demo target: ares.demo runner: type: Duration duration: 60 interval: 1 sla: max_rtt: 10 action: monitor context: name: demo image: yardstick-image flavor: yardstick-flavor user: ubuntu placement_groups: pgrp1: policy: "availability" servers: athena: floating_ip: true placement: "pgrp1" ares: placement: "pgrp1" networks: test: cidr: '10.0.1.0/24' {% if provider == "vlan" %} provider: {{provider}} physical_network: {{physical_network}} {% if segmentation_id %} segmentation_id: {{segmentation_id}} {% endif %} {% endif %} "Contexts" section is the description of pre-condition of testing. As ping.yaml shown, you can configure the image, flavor , name ,affinity and network of Test VM(servers), with this section, you will get a pre-condition env for Testing. Yardstick will automatic setup the stack which are described in this section. In fact, yardstick use convert this section to heat template and setup the VMs by heat-client (Meanwhile, yardstick can support to convert this section to Kubernetes template to setup containers). Two Test VMs(athena and ares) are configured by keyword "servers". "flavor" will determine how many vCPU, how much memory for test VMs. As "yardstick-flavor" is a basic flavor which will be automatically created when you run command "yardstick env prepare". "yardstick-flavor" is "1 vCPU 1G RAM,3G Disk". "image" is the image name of test VMs. if you use cirros.3.5.0, you need fill the username of this image into "user". the "
heat_template_version: 2015-10-15

description: >
  Creates a port on the storage_mgmt API network.

parameters:
  StorageMgmtNetName:
    description: Name of the storage_mgmt API neutron network
    default: storage_mgmt
    type: string
  PortName:
    description: Name of the port
    default: ''
    type: string
  ControlPlaneIP: # Here for compatibility with noop.yaml
    description: IP address on the control plane
    default: ''
    type: string
  FixedIPs:
    description: >
        Control the IP allocation for the VIP port. E.g.
        [{'ip_address':'1.2.3.4'}]
    default: []
    type: json
  IPPool: # Here for compatibility with from_pool.yaml
    default: {}
    type: json
  NodeIndex: # Here for compatibility with from_pool.yaml
    default: 0
    type: number

resources:

  StorageMgmtPort:
    type: OS::Neutron::Port
    properties:
      network: {get_param: StorageMgmtNetName}
      name: {get_param: PortName}
      fixed_ips: {get_param: FixedIPs}
      replacement_policy: AUTO

outputs:
  ip_address:
    description: storage_mgmt network IP
    value: {get_attr: [StorageMgmtPort, fixed_ips, 0, ip_address]}
  ip_address_uri:
    description: |
        storage_mgmt network IP (for compatibility with storage_mgmt_v6.yaml)
    value: {get_attr: [StorageMgmtPort, fixed_ips, 0, ip_address]}
  ip_subnet:
    description: IP/Subnet CIDR for the storage_mgmt network IP
    value:
          list_join:
            - ''
            - - {get_attr: [StorageMgmtPort, fixed_ips, 0, ip_address]}
              - '/'
              - {str_split: ['/', {get_attr: [StorageMgmtPort, subnets, 0, cidr]}, 1]}
'--amend' option on the ``git commit`` command:: git commit --amend If the commit was successful, the ``git status`` command should not return the updated files as about to be commited. The final step consists in pushing the newly modified commit to Gerrit:: git review Plugins ========== For information about Yardstick plugins, refer to the chapter **Installing a plug-in into Yardstick** in the `user guide`_.