diff options
Diffstat (limited to 'docs/testing')
| -rw-r--r-- | docs/testing/user/userguide/14-nsb_installation.rst | 3 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc056.rst | 149 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc057.rst | 165 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc058.rst | 148 |
4 files changed, 464 insertions, 1 deletions
diff --git a/docs/testing/user/userguide/14-nsb_installation.rst b/docs/testing/user/userguide/14-nsb_installation.rst index 3eb17bbca..7c5327964 100644 --- a/docs/testing/user/userguide/14-nsb_installation.rst +++ b/docs/testing/user/userguide/14-nsb_installation.rst @@ -103,7 +103,7 @@ Config yardstick conf cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf vi /etc/yardstick/yardstick.conf -Add trex_path and bin_path in 'nsb' section. +Add trex_path, trex_client_lib and bin_path in 'nsb' section. :: @@ -121,6 +121,7 @@ Add trex_path and bin_path in 'nsb' section. [nsb] trex_path=/opt/nsb_bin/trex/scripts bin_path=/opt/nsb_bin + trex_client_lib=/opt/nsb_bin/trex_client/stl Config pod.yaml describing Topology diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc056.rst b/docs/testing/user/userguide/opnfv_yardstick_tc056.rst new file mode 100644 index 000000000..01aa99ac2 --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc056.rst @@ -0,0 +1,149 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Yin Kanglin and others. +.. 14_ykl@tongji.edu.cn + +************************************* +Yardstick Test Case Description TC056 +************************************* + ++-----------------------------------------------------------------------------+ +|OpenStack Controller Messaging Queue Service High Availability | ++==============+==============================================================+ +|test case id | OPNFV_YARDSTICK_TC056:OpenStack Controller Messaging Queue | +| | Service High Availability | ++--------------+--------------------------------------------------------------+ +|test purpose | This test case will verify the high availability of the | +| | messaging queue service(RabbitMQ) that supports OpenStack on | +| | controller node. When messaging queue service(which is | +| | active) of a specified controller node is killed, the test | +| | case will check whether messaging queue services(which are | +| | standby) on other controller nodes will be switched active, | +| | and whether the cluster manager on attacked the controller | +| | node will restart the stopped messaging queue. | ++--------------+--------------------------------------------------------------+ +|test method | This test case kills the processes of messaging queue | +| | service on a selected controller node, then checks whether | +| | the request of the related Openstack command is OK and the | +| | killed processes are recovered. | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, an attacker called "kill-process" is | +| | needed. This attacker includes three parameters: | +| | 1) fault_type: which is used for finding the attacker's | +| | scripts. It should be always set to "kill-process" in this | +| | test case. | +| | 2) process_name: which is the process name of the specified | +| | OpenStack service. If there are multiple processes use the | +| | same name on the host, all of them are killed by this | +| | attacker. | +| | In this case, this parameter should always set to "rabbitmq".| +| | 3) host: which is the name of a control node being attacked. | +| | | +| | e.g. | +| | -fault_type: "kill-process" | +| | -process_name: "rabbitmq-server" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|monitors | In this test case, two kinds of monitor are needed: | +| | 1. the "openstack-cmd" monitor constantly request a specific | +| | Openstack command, which needs two parameters: | +| | 1) monitor_type: which is used for finding the monitor class | +| | and related scritps. It should be always set to | +| | "openstack-cmd" for this monitor. | +| | 2) command_name: which is the command name used for request. | +| | | +| | 2. the "process" monitor check whether a process is running | +| | on a specific node, which needs three parameters: | +| | 1) monitor_type: which used for finding the monitor class | +| | and related scripts. It should be always set to "process" | +| | for this monitor. | +| | 2) process_name: which is the process name for monitor | +| | 3) host: which is the name of the node runing the process | +| | In this case, the command_name of monitor1 should be | +| | services that will use the messaging queue(current nova, | +| | neutron, cinder ,heat and ceilometer are using RabbitMQ) | +| | , and the process-name of monitor2 should be "rabbitmq", | +| | for example: | +| | | +| | e.g. | +| | monitor1-1: | +| | -monitor_type: "openstack-cmd" | +| | -command_name: "openstack image list" | +| | monitor1-2: | +| | -monitor_type: "openstack-cmd" | +| | -command_name: "openstack network list" | +| | monitor1-3: | +| | -monitor_type: "openstack-cmd" | +| | -command_name: "openstack volume list" | +| | monitor2: | +| | -monitor_type: "process" | +| | -process_name: "rabbitmq" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there are two metrics: | +| | 1)service_outage_time: which indicates the maximum outage | +| | time (seconds) of the specified Openstack command request. | +| | 2)process_recover_time: which indicates the maximum time | +| | (seconds) from the process being killed to recovered | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Developed by the project. Please see folder: | +| | "yardstick/benchmark/scenarios/availability/ha_tools" | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI NFV REL001 | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1) test case file:opnfv_yardstick_tc056.yaml | +| | -Attackers: see above "attackers" description | +| | -waiting_time: which is the time (seconds) from the process | +| | being killed to stoping monitors the monitors | +| | -Monitors: see above "monitors" description | +| | -SLA: see above "metrics" description | +| | | +| | 2)POD file: pod.yaml | +| | The POD configuration should record on pod.yaml first. | +| | the "host" item in this test case will use the node name in | +| | the pod.yaml. | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | start monitors: | +| | each monitor will run with independently process | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | do attacker: connect the host through SSH, and then execute | +| | the kill process script with param value specified by | +| | "process_name" | +| | | +| | Result: Process will be killed. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | stop monitors after a period of time specified by | +| | "waiting_time" | +| | | +| | Result: The monitor info will be aggregated. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | verify the SLA | +| | | +| | Result: The test case is passed or not. | +| | | ++--------------+--------------------------------------------------------------+ +|post-action | It is the action when the test cases exist. It will check | +| | the status of the specified process on the host, and restart | +| | the process if it is not running for next test cases. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | Fails only if SLA is not passed, or if there is a test case | +| | execution problem. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc057.rst b/docs/testing/user/userguide/opnfv_yardstick_tc057.rst new file mode 100644 index 000000000..2a4ce40c0 --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc057.rst @@ -0,0 +1,165 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Yin Kanglin and others. +.. 14_ykl@tongji.edu.cn + +************************************* +Yardstick Test Case Description TC057 +************************************* + ++-----------------------------------------------------------------------------+ +|OpenStack Controller Cluster Management Service High Availability | ++==============+==============================================================+ +|test case id | | ++--------------+--------------------------------------------------------------+ +|test purpose | This test case will verify the quorum configuration of the | +| | cluster manager(pacemaker) on controller nodes. When a | +| | controller node , which holds all active application | +| | resources, failed to communicate with other cluster nodes | +| | (via corosync), the test case will check whether the standby | +| | application resources will take place of those active | +| | application resources which should be regarded to be down in | +| | the cluster manager. | ++--------------+--------------------------------------------------------------+ +|test method | This test case kills the processes of cluster messaging | +| | service(corosync) on a selected controller node(the node | +| | holds the active application resources), then checks whether | +| | active application resources are switched to other | +| | controller nodes and whether the Openstack commands are OK. | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, an attacker called "kill-process" is | +| | needed. This attacker includes three parameters: | +| | 1) fault_type: which is used for finding the attacker's | +| | scripts. It should be always set to "kill-process" in this | +| | test case. | +| | 2) process_name: which is the process name of the load | +| | balance service. If there are multiple processes use the | +| | same name on the host, all of them are killed by this | +| | attacker. | +| | 3) host: which is the name of a control node being attacked. | +| | | +| | In this case, this process name should set to "corosync" , | +| | for example | +| | -fault_type: "kill-process" | +| | -process_name: "corosync" | +| | -host: node1 | ++--------------+--------------------------------------------------------------+ +|monitors | In this test case, a kind of monitor is needed: | +| | 1. the "openstack-cmd" monitor constantly request a specific | +| | Openstack command, which needs two parameters: | +| | 1) monitor_type: which is used for finding the monitor class | +| | and related scripts. It should be always set to | +| | "openstack-cmd" for this monitor. | +| | 2) command_name: which is the command name used for request | +| | | +| | In this case, the command_name of monitor1 should be services| +| | that are managed by the cluster manager. (Since rabbitmq and | +| | haproxy are managed by pacemaker, most Openstack Services | +| | can be used to check high availability in this case) | +| | | +| | (e.g.) | +| | monitor1: | +| | -monitor_type: "openstack-cmd" | +| | -command_name: "nova image-list" | +| | monitor2: | +| | -monitor_type: "openstack-cmd" | +| | -command_name: "neutron router-list" | +| | monitor3: | +| | -monitor_type: "openstack-cmd" | +| | -command_name: "heat stack-list" | +| | monitor4: | +| | -monitor_type: "openstack-cmd" | +| | -command_name: "cinder list" | +| | | ++--------------+--------------------------------------------------------------+ +|checkers | In this test case, a checker is needed, the checker will | +| | the status of application resources in pacemaker and the | +| | checker have three parameters: | +| | 1) checker_type: which is used for finding the result | +| | checker class and related scripts. In this case the checker | +| | type will be "pacemaker-check-resource" | +| | 2) resource_name: the application resource name | +| | 3) resource_status: the expected status of the resource | +| | 4) expectedValue: the expected value for the output of the | +| | checker script, in the case the expected value will be the | +| | identifier in the cluster manager | +| | 3) condition: whether the expected value is in the output of | +| | checker script or is totally same with the output. | +| | (note: pcs is required to installed on controller node in | +| | order to run this checker) | +| | | +| | (e.g.) | +| | checker1: | +| | -checker_type: "pacemaker-check-resource" | +| | -resource_name: "p_rabbitmq-server" | +| | -resource_status: "Stopped" | +| | -expectedValue: "node-1" | +| | -condition: "in" | +| | checker2: | +| | -checker_type: "pacemaker-check-resource" | +| | -resource_name: "p_rabbitmq-server" | +| | -resource_status: "Master" | +| | -expectedValue: "node-2" | +| | -condition: "in" | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there are two metrics: | +| | 1)service_outage_time: which indicates the maximum outage | +| | time (seconds) of the specified Openstack command request. | ++--------------+--------------------------------------------------------------+ +|test tool | None. Self-developed. | ++--------------+--------------------------------------------------------------+ +|references | ETSI NFV REL001 | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1) test case file: opnfv_yardstick_tc057.yaml | +| | -Attackers: see above "attackers" description | +| | -Monitors: see above "monitors" description | +| | -Checkers: see above "checkers" description | +| | -Steps: the test case execution step, see "test sequence" | +| | description below | +| | | +| | 2)POD file: pod.yaml | +| | The POD configuration should record on pod.yaml first. | +| | the "host" item in this test case will use the node name in | +| | the pod.yaml. | ++--------------+------+----------------------------------+--------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | start monitors: | +| | each monitor will run with independently process | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | do attacker: connect the host through SSH, and then execute | +| | the kill process script with param value specified by | +| | "process_name" | +| | | +| | Result: Process will be killed. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | do checker: check whether the status of application | +| | resources on different nodes are updated | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | stop monitors after a period of time specified by | +| | "waiting_time" | +| | | +| | Result: The monitor info will be aggregated. | +| | | ++--------------+--------------------------------------------------------------+ +|step 5 | verify the SLA | +| | | +| | Result: The test case is passed or not. | +| | | ++--------------+------+----------------------------------+--------------------+ +|post-action | It is the action when the test cases exist. It will check the| +| | status of the cluster messaging process(corosync) on the | +| | host, and restart the process if it is not running for next | +| | test cases | ++--------------+------+----------------------------------+--------------------+ +|test verdict | Fails only if SLA is not passed, or if there is a test case | +| | execution problem. | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc058.rst b/docs/testing/user/userguide/opnfv_yardstick_tc058.rst new file mode 100644 index 000000000..fb9a4c2d1 --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc058.rst @@ -0,0 +1,148 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Yin Kanglin and others. +.. 14_ykl@tongji.edu.cn + +************************************* +Yardstick Test Case Description TC058 +************************************* + ++-----------------------------------------------------------------------------+ +|OpenStack Controller Virtual Router Service High Availability | ++==============+==============================================================+ +|test case id | OPNFV_YARDSTICK_TC058:OpenStack Controller Virtual Router | +| | Service High Availability | ++--------------+--------------------------------------------------------------+ +|test purpose | This test case will verify the high availability of virtual | +| | routers(L3 agent) on controller node. When a virtual router | +| | service on a specified controller node is shut down, this | +| | test case will check whether the network of virtual machines | +| | will be affected, and whether the attacked virtual router | +| | service will be recovered. | ++--------------+--------------------------------------------------------------+ +|test method | This test case kills the processes of virtual router service | +| | (l3-agent) on a selected controller node(the node holds the | +| | active l3-agent), then checks whether the network routing | +| | of virtual machines is OK and whether the killed service | +| | will be recovered. | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, an attacker called "kill-process" is | +| | needed. This attacker includes three parameters: | +| | 1) fault_type: which is used for finding the attacker's | +| | scripts. It should be always set to "kill-process" in this | +| | test case. | +| | 2) process_name: which is the process name of the load | +| | balance service. If there are multiple processes use the | +| | same name on the host, all of them are killed by this | +| | attacker. | +| | 3) host: which is the name of a control node being attacked. | +| | | +| | In this case, this process name should set to "l3agent" , | +| | for example | +| | -fault_type: "kill-process" | +| | -process_name: "l3agent" | +| | -host: node1 | ++--------------+--------------------------------------------------------------+ +|monitors | In this test case, two kinds of monitor are needed: | +| | 1. the "ip_status" monitor that pings a specific ip to check | +| | the connectivity of this ip, which needs two parameters: | +| | 1) monitor_type: which is used for finding the monitor class | +| | and related scripts. It should be always set to "ip_status" | +| | for this monitor. | +| | 2) ip_address: The ip to be pinged. In this case, ip_address | +| | will be either an ip address of external network or an ip | +| | address of a virtual machine. | +| | 3) host: The node on which ping will be executed, in this | +| | case the host will be a virtual machine. | +| | | +| | 2. the "process" monitor check whether a process is running | +| | on a specific node, which needs three parameters: | +| | 1) monitor_type: which used for finding the monitor class | +| | and related scripts. It should be always set to "process" | +| | for this monitor. | +| | 2) process_name: which is the process name for monitor. In | +| | this case, the process-name of monitor2 should be "l3agent" | +| | 3) host: which is the name of the node running the process | +| | | +| | e.g. | +| | monitor1-1: | +| | -monitor_type: "ip_status" | +| | -host: 172.16.0.11 | +| | -ip_address: 172.16.1.11 | +| | monitor1-2: | +| | -monitor_type: "ip_status" | +| | -host: 172.16.0.11 | +| | -ip_address: 8.8.8.8 | +| | monitor2: | +| | -monitor_type: "process" | +| | -process_name: "l3agent" | +| | -host: node1 | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there are two metrics: | +| | 1)service_outage_time: which indicates the maximum outage | +| | time (seconds) of the specified Openstack command request. | +| | 2)process_recover_time: which indicates the maximum time | +| | (seconds) from the process being killed to recovered | ++--------------+--------------------------------------------------------------+ +|test tool | None. Self-developed. | ++--------------+--------------------------------------------------------------+ +|references | ETSI NFV REL001 | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1) test case file: opnfv_yardstick_tc058.yaml | +| | -Attackers: see above "attackers" description | +| | -Monitors: see above "monitors" description | +| | -Steps: the test case execution step, see "test sequence" | +| | description below | +| | | +| | 2)POD file: pod.yaml | +| | The POD configuration should record on pod.yaml first. | +| | the "host" item in this test case will use the node name in | +| | the pod.yaml. | ++--------------+------+----------------------------------+--------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | The test case image needs to be installed into Glance | +|conditions | with cachestat included in the image. | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | Two host VMs are booted, these two hosts are in two different| +| | networks, the networks are connected by a virtual router | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | start monitors: | +| | each monitor will run with independently process | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | do attacker: connect the host through SSH, and then execute | +| | the kill process script with param value specified by | +| | "process_name" | +| | | +| | Result: Process will be killed. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | stop monitors after a period of time specified by | +| | "waiting_time" | +| | | +| | Result: The monitor info will be aggregated. | +| | | ++--------------+--------------------------------------------------------------+ +|step 5 | verify the SLA | +| | | +| | Result: The test case is passed or not. | +| | | ++--------------+------+----------------------------------+--------------------+ +|post-action | It is the action when the test cases exist. It will check | +| | the status of the specified process on the host, and restart | +| | the process if it is not running for next test cases. | +| | Virtual machines and network created in the test case will | +| | be destoryed. | +| | | ++--------------+------+----------------------------------+--------------------+ +|test verdict | Fails only if SLA is not passed, or if there is a test case | +| | execution problem. | ++--------------+--------------------------------------------------------------+ |