From d4fce3f39f4f637b4e0bff03851a3f252aee6e1a Mon Sep 17 00:00:00 2001 From: QiLiang Date: Tue, 12 Jan 2016 22:53:22 +0000 Subject: Rename tc019/tc025 & fix description for tc005 Added tc019 and tc025 to index of test cases in 03-list-of-tcs. JIRA: - Change-Id: I02696dc408d2d5c20becba29c49b3cff2a7049a9 Signed-off-by: QiLiang (cherry picked from commit 25095fb4e6d6a0e71581eee18f50c7d61f6a270d) --- .../yardstick_testcases/03-list-of-tcs.rst | 2 + .../yardstick_testcases/opnfv_yardstick_tc005.rst | 4 +- .../yardstick_testcases/opnfv_yardstick_tc019.rst | 129 +++++++++++++++++++++ .../yardstick_testcases/opnfv_yardstick_tc025.rst | 99 ++++++++++++++++ docs/yardstick/opnfv_yardstick_tc019.rst | 129 --------------------- docs/yardstick/opnfv_yardstick_tc025.rst | 99 ---------------- 6 files changed, 232 insertions(+), 230 deletions(-) create mode 100644 docs/configguide/yardstick_testcases/opnfv_yardstick_tc019.rst create mode 100644 docs/configguide/yardstick_testcases/opnfv_yardstick_tc025.rst delete mode 100644 docs/yardstick/opnfv_yardstick_tc019.rst delete mode 100644 docs/yardstick/opnfv_yardstick_tc025.rst diff --git a/docs/configguide/yardstick_testcases/03-list-of-tcs.rst b/docs/configguide/yardstick_testcases/03-list-of-tcs.rst index d00025840..ac01b3de2 100644 --- a/docs/configguide/yardstick_testcases/03-list-of-tcs.rst +++ b/docs/configguide/yardstick_testcases/03-list-of-tcs.rst @@ -29,6 +29,8 @@ Generic NFVI Test Case Descriptions opnfv_yardstick_tc010.rst opnfv_yardstick_tc012.rst opnfv_yardstick_tc014.rst + opnfv_yardstick_tc019.rst + opnfv_yardstick_tc025.rst opnfv_yardstick_tc028.rst opnfv_yardstick_tc037.rst opnfv_yardstick_tc038.rst diff --git a/docs/configguide/yardstick_testcases/opnfv_yardstick_tc005.rst b/docs/configguide/yardstick_testcases/opnfv_yardstick_tc005.rst index 8b7474696..83b6aedd6 100644 --- a/docs/configguide/yardstick_testcases/opnfv_yardstick_tc005.rst +++ b/docs/configguide/yardstick_testcases/opnfv_yardstick_tc005.rst @@ -60,8 +60,8 @@ Yardstick Test Case Description TC005 |test sequence | description and expected result | | | | +--------------+--------------------------------------------------------------+ -|step 1 | The hosts are installed, as server and client. fio is | -| | invoked and logs are produced and stored. | +|step 1 | The host is installed and fio is invoked and logs are | +| | produced and stored. | | | | | | Result: Logs are stored. | | | | diff --git a/docs/configguide/yardstick_testcases/opnfv_yardstick_tc019.rst b/docs/configguide/yardstick_testcases/opnfv_yardstick_tc019.rst new file mode 100644 index 000000000..482260b48 --- /dev/null +++ b/docs/configguide/yardstick_testcases/opnfv_yardstick_tc019.rst @@ -0,0 +1,129 @@ +************************************* +Yardstick Test Case Description TC019 +************************************* + ++-----------------------------------------------------------------------------+ +|Control Node Openstack Service High Availability | +| | ++--------------+--------------------------------------------------------------+ +|test case id | OPNFV_YARDSTICK_TC019_HA: Control node Openstack service down| +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | This test case will verify the high availability of the | +| | service provided by OpenStack (like nova-api, neutro-server) | +| | on control node. | +| | | ++--------------+--------------------------------------------------------------+ +|test method | This test case kills the processes of a specific Openstack | +| | service on a selected control node, then checks whether the | +| | request of the related Openstack command is OK and the killed| +| | processes are recovered. | +| | | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, an attacker called "kill-process" is | +| | needed. This attacker includes three parameters: | +| | 1) fault_type: which is used for finding the attacker's | +| | scripts. It should be always set to "kill-process" in this | +| | test case. | +| | 2) process_name: which is the process name of the specified | +| | OpenStack service. If there are multiple processes use the | +| | same name on the host, all of them are killed by this | +| | attacker. | +| | 3) host: which is the name of a control node being attacked. | +| | | +| | e.g. | +| | -fault_type: "kill-process" | +| | -process_name: "nova-api" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|monitors | In this test case, two kinds of monitor are needed: | +| | 1. the "openstack-cmd" monitor constantly request a specific | +| | Openstack command, which needs two parameters: | +| | 1) monitor_type: which is used for finding the monitor class | +| | and related scritps. It should be always set to | +| | "openstack-cmd" for this monitor. | +| | 2) command_name: which is the command name used for request | +| | | +| | 2. the "process" monitor check whether a process is running | +| | on a specific node, which needs three parameters: | +| | 1) monitor_type: which used for finding the monitor class and| +| | related scritps. It should be always set to "process" | +| | for this monitor. | +| | 2) process_name: which is the process name for monitor | +| | 3) host: which is the name of the node runing the process | +| | | +| | e.g. | +| | monitor1: | +| | -monitor_type: "openstack-cmd" | +| | -command_name: "nova image-list" | +| | monitor2: | +| | -monitor_type: "process" | +| | -process_name: "nova-api" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there are two metrics: | +| | 1)service_outage_time: which indicates the maximum outage | +| | time (seconds) of the specified Openstack command request. | +| | 2)process_recover_time: which indicates the maximun time | +| | (seconds) from the process being killed to recovered | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Developed by the project. Please see folder: | +| | "yardstick/benchmark/scenarios/availability/ha_tools" | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI NFV REL001 | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1) test case file: opnfv_yardstick_tc019.yaml | +| | -Attackers: see above "attackers" discription | +| | -waiting_time: which is the time (seconds) from the process | +| | being killed to stoping monitors the monitors | +| | -Monitors: see above "monitors" discription | +| | -SLA: see above "metrics" discription | +| | | +| | 2)POD file: pod.yaml | +| | The POD configuration should record on pod.yaml first. | +| | the "host" item in this test case will use the node name in | +| | the pod.yaml. | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | start monitors: | +| | each monitor will run with independently process | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | do attacker: connect the host through SSH, and then execute | +| | the kill process script with param value specified by | +| | "process_name" | +| | | +| | Result: Process will be killed. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | stop monitors after a period of time specified by | +| | "waiting_time" | +| | | +| | Result: The monitor info will be aggregated. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | verify the SLA | +| | | +| | Result: The test case is passed or not. | +| | | ++--------------+--------------------------------------------------------------+ +|post-action | It is the action when the test cases exist. It will check the| +| | status of the specified process on the host, and restart the | +| | process if it is not running for next test cases | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | Fails only if SLA is not passed, or if there is a test case | +| | execution problem. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/configguide/yardstick_testcases/opnfv_yardstick_tc025.rst b/docs/configguide/yardstick_testcases/opnfv_yardstick_tc025.rst new file mode 100644 index 000000000..f01bd3bb7 --- /dev/null +++ b/docs/configguide/yardstick_testcases/opnfv_yardstick_tc025.rst @@ -0,0 +1,99 @@ +.. image:: ../../etc/opnfv-logo.png + :height: 40 + :width: 200 + :alt: OPNFV + :align: left + +************************************* +Yardstick Test Case Description TC025 +************************************* + ++-----------------------------------------------------------------------------+ +|OpenStack Controll Node abnormally shutdown High Availability | ++==============+==============================================================+ +|test case id | OPNFV_YARDSTICK_TC025_HA: OpenStack Controll Node abnormally | +| | shutdown | ++--------------+--------------------------------------------------------------+ +|test purpose | This test case will verify the high availability of control | +| | nodethe. When one of the control node abnormally shutdown, | +| | the service provided by it should be OK. | ++--------------+--------------------------------------------------------------+ +|test method | This test case shutdowns a specified control node with some | +| | fault injection tools, then checks whether all services | +| | provided by the control node are OK with some monitor tools. | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, an attacker called "host-shutdown" is | +| | needed. This attacker includes two parameters: | +| | 1) fault_type: which is used for finding the attacker's | +| | scripts. It should be always set to "host-shutdown" in | +| | this test case. | +| | 2) host: which is the name of a control node being attacked. | +| | | +| | e.g. | +| | -fault_type: "host-shutdown" | +| | -host: node1 | ++--------------+--------------------------------------------------------------+ +|monitors | In this test case, one kind of monitor are needed: | +| | 1. the "openstack-cmd" monitor constantly request a specific | +| | Openstack command, which needs two parameters | +| | 1) monitor_type: which is used for finding the monitor class | +| | and related scritps. It should be always set to | +| | "openstack-cmd" for this monitor. | +| | 2) command_name: which is the command name used for request | +| | | +| | There are four instance of the "openstack-cmd" monitor: | +| | monitor1: | +| | -monitor_type: "openstack-cmd" | +| | -api_name: "nova image-list" | +| | monitor2: | +| | -monitor_type: "openstack-cmd" | +| | -api_name: "neutron router-list" | +| | monitor3: | +| | -monitor_type: "openstack-cmd" | +| | -api_name: "heat stack-list" | +| | monitor4: | +| | -monitor_type: "openstack-cmd" | +| | -api_name: "cinder list" | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there is one metric: | +| | 1)service_outage_time: which indicates the maximum outage | +| | time (seconds) of the specified Openstack command request. | ++--------------+--------------------------------------------------------------+ +|test tool | None. Self-developed. | ++--------------+--------------------------------------------------------------+ +|references | ETSI NFV REL001 | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1) test case file: opnfv_yardstick_tc019.yaml | +| | -Attackers: see above "attackers" discription | +| | -waiting_time: which is the time (seconds) from the process | +| | being killed to stoping monitors the monitors | +| | -Monitors: see above "monitors" discription | +| | -SLA: see above "metrics" discription | +| | | +| | 2)POD file: pod.yaml | +| | The POD configuration should record on pod.yaml first. | +| | the "host" item in this test case will use the node name in | +| | the pod.yaml. | ++--------------+------+----------------------------------+--------------------+ +|test sequence | step | description | result | +| +------+----------------------------------+--------------------+ +| | 1 | start monitors: each monitor will| The monitor info | +| | | run with independently process | will be collected. | +| +------+----------------------------------+--------------------+ +| | 2 | do attacker: connect the host | the host will be | +| | | through SSH, and then execute | shutdown. | +| | | shutdown script on the host | | +| +------+----------------------------------+--------------------+ +| | 3 | stop monitors after a period of | All monitor result | +| | | time specified by "waiting_time" | will be aggregated.| +| +------+----------------------------------+--------------------+ +| | 4 | verify the SLA | The test case is | +| | | | passed or not. | ++--------------+------+----------------------------------+--------------------+ +|post-action | It is the action when the test cases exist. It restarts the | +| | specified control node if it is not restarted. | ++--------------+------+----------------------------------+--------------------+ +|test verdict | Fails only if SLA is not passed, or if there is a test case | +| | execution problem. | ++--------------+--------------------------------------------------------------+ diff --git a/docs/yardstick/opnfv_yardstick_tc019.rst b/docs/yardstick/opnfv_yardstick_tc019.rst deleted file mode 100644 index 482260b48..000000000 --- a/docs/yardstick/opnfv_yardstick_tc019.rst +++ /dev/null @@ -1,129 +0,0 @@ -************************************* -Yardstick Test Case Description TC019 -************************************* - -+-----------------------------------------------------------------------------+ -|Control Node Openstack Service High Availability | -| | -+--------------+--------------------------------------------------------------+ -|test case id | OPNFV_YARDSTICK_TC019_HA: Control node Openstack service down| -| | | -+--------------+--------------------------------------------------------------+ -|test purpose | This test case will verify the high availability of the | -| | service provided by OpenStack (like nova-api, neutro-server) | -| | on control node. | -| | | -+--------------+--------------------------------------------------------------+ -|test method | This test case kills the processes of a specific Openstack | -| | service on a selected control node, then checks whether the | -| | request of the related Openstack command is OK and the killed| -| | processes are recovered. | -| | | -+--------------+--------------------------------------------------------------+ -|attackers | In this test case, an attacker called "kill-process" is | -| | needed. This attacker includes three parameters: | -| | 1) fault_type: which is used for finding the attacker's | -| | scripts. It should be always set to "kill-process" in this | -| | test case. | -| | 2) process_name: which is the process name of the specified | -| | OpenStack service. If there are multiple processes use the | -| | same name on the host, all of them are killed by this | -| | attacker. | -| | 3) host: which is the name of a control node being attacked. | -| | | -| | e.g. | -| | -fault_type: "kill-process" | -| | -process_name: "nova-api" | -| | -host: node1 | -| | | -+--------------+--------------------------------------------------------------+ -|monitors | In this test case, two kinds of monitor are needed: | -| | 1. the "openstack-cmd" monitor constantly request a specific | -| | Openstack command, which needs two parameters: | -| | 1) monitor_type: which is used for finding the monitor class | -| | and related scritps. It should be always set to | -| | "openstack-cmd" for this monitor. | -| | 2) command_name: which is the command name used for request | -| | | -| | 2. the "process" monitor check whether a process is running | -| | on a specific node, which needs three parameters: | -| | 1) monitor_type: which used for finding the monitor class and| -| | related scritps. It should be always set to "process" | -| | for this monitor. | -| | 2) process_name: which is the process name for monitor | -| | 3) host: which is the name of the node runing the process | -| | | -| | e.g. | -| | monitor1: | -| | -monitor_type: "openstack-cmd" | -| | -command_name: "nova image-list" | -| | monitor2: | -| | -monitor_type: "process" | -| | -process_name: "nova-api" | -| | -host: node1 | -| | | -+--------------+--------------------------------------------------------------+ -|metrics | In this test case, there are two metrics: | -| | 1)service_outage_time: which indicates the maximum outage | -| | time (seconds) of the specified Openstack command request. | -| | 2)process_recover_time: which indicates the maximun time | -| | (seconds) from the process being killed to recovered | -| | | -+--------------+--------------------------------------------------------------+ -|test tool | Developed by the project. Please see folder: | -| | "yardstick/benchmark/scenarios/availability/ha_tools" | -| | | -+--------------+--------------------------------------------------------------+ -|references | ETSI NFV REL001 | -| | | -+--------------+--------------------------------------------------------------+ -|configuration | This test case needs two configuration files: | -| | 1) test case file: opnfv_yardstick_tc019.yaml | -| | -Attackers: see above "attackers" discription | -| | -waiting_time: which is the time (seconds) from the process | -| | being killed to stoping monitors the monitors | -| | -Monitors: see above "monitors" discription | -| | -SLA: see above "metrics" discription | -| | | -| | 2)POD file: pod.yaml | -| | The POD configuration should record on pod.yaml first. | -| | the "host" item in this test case will use the node name in | -| | the pod.yaml. | -| | | -+--------------+--------------------------------------------------------------+ -|test sequence | description and expected result | -| | | -+--------------+--------------------------------------------------------------+ -|step 1 | start monitors: | -| | each monitor will run with independently process | -| | | -| | Result: The monitor info will be collected. | -| | | -+--------------+--------------------------------------------------------------+ -|step 2 | do attacker: connect the host through SSH, and then execute | -| | the kill process script with param value specified by | -| | "process_name" | -| | | -| | Result: Process will be killed. | -| | | -+--------------+--------------------------------------------------------------+ -|step 3 | stop monitors after a period of time specified by | -| | "waiting_time" | -| | | -| | Result: The monitor info will be aggregated. | -| | | -+--------------+--------------------------------------------------------------+ -|step 4 | verify the SLA | -| | | -| | Result: The test case is passed or not. | -| | | -+--------------+--------------------------------------------------------------+ -|post-action | It is the action when the test cases exist. It will check the| -| | status of the specified process on the host, and restart the | -| | process if it is not running for next test cases | -| | | -+--------------+--------------------------------------------------------------+ -|test verdict | Fails only if SLA is not passed, or if there is a test case | -| | execution problem. | -| | | -+--------------+--------------------------------------------------------------+ diff --git a/docs/yardstick/opnfv_yardstick_tc025.rst b/docs/yardstick/opnfv_yardstick_tc025.rst deleted file mode 100644 index f01bd3bb7..000000000 --- a/docs/yardstick/opnfv_yardstick_tc025.rst +++ /dev/null @@ -1,99 +0,0 @@ -.. image:: ../../etc/opnfv-logo.png - :height: 40 - :width: 200 - :alt: OPNFV - :align: left - -************************************* -Yardstick Test Case Description TC025 -************************************* - -+-----------------------------------------------------------------------------+ -|OpenStack Controll Node abnormally shutdown High Availability | -+==============+==============================================================+ -|test case id | OPNFV_YARDSTICK_TC025_HA: OpenStack Controll Node abnormally | -| | shutdown | -+--------------+--------------------------------------------------------------+ -|test purpose | This test case will verify the high availability of control | -| | nodethe. When one of the control node abnormally shutdown, | -| | the service provided by it should be OK. | -+--------------+--------------------------------------------------------------+ -|test method | This test case shutdowns a specified control node with some | -| | fault injection tools, then checks whether all services | -| | provided by the control node are OK with some monitor tools. | -+--------------+--------------------------------------------------------------+ -|attackers | In this test case, an attacker called "host-shutdown" is | -| | needed. This attacker includes two parameters: | -| | 1) fault_type: which is used for finding the attacker's | -| | scripts. It should be always set to "host-shutdown" in | -| | this test case. | -| | 2) host: which is the name of a control node being attacked. | -| | | -| | e.g. | -| | -fault_type: "host-shutdown" | -| | -host: node1 | -+--------------+--------------------------------------------------------------+ -|monitors | In this test case, one kind of monitor are needed: | -| | 1. the "openstack-cmd" monitor constantly request a specific | -| | Openstack command, which needs two parameters | -| | 1) monitor_type: which is used for finding the monitor class | -| | and related scritps. It should be always set to | -| | "openstack-cmd" for this monitor. | -| | 2) command_name: which is the command name used for request | -| | | -| | There are four instance of the "openstack-cmd" monitor: | -| | monitor1: | -| | -monitor_type: "openstack-cmd" | -| | -api_name: "nova image-list" | -| | monitor2: | -| | -monitor_type: "openstack-cmd" | -| | -api_name: "neutron router-list" | -| | monitor3: | -| | -monitor_type: "openstack-cmd" | -| | -api_name: "heat stack-list" | -| | monitor4: | -| | -monitor_type: "openstack-cmd" | -| | -api_name: "cinder list" | -+--------------+--------------------------------------------------------------+ -|metrics | In this test case, there is one metric: | -| | 1)service_outage_time: which indicates the maximum outage | -| | time (seconds) of the specified Openstack command request. | -+--------------+--------------------------------------------------------------+ -|test tool | None. Self-developed. | -+--------------+--------------------------------------------------------------+ -|references | ETSI NFV REL001 | -+--------------+--------------------------------------------------------------+ -|configuration | This test case needs two configuration files: | -| | 1) test case file: opnfv_yardstick_tc019.yaml | -| | -Attackers: see above "attackers" discription | -| | -waiting_time: which is the time (seconds) from the process | -| | being killed to stoping monitors the monitors | -| | -Monitors: see above "monitors" discription | -| | -SLA: see above "metrics" discription | -| | | -| | 2)POD file: pod.yaml | -| | The POD configuration should record on pod.yaml first. | -| | the "host" item in this test case will use the node name in | -| | the pod.yaml. | -+--------------+------+----------------------------------+--------------------+ -|test sequence | step | description | result | -| +------+----------------------------------+--------------------+ -| | 1 | start monitors: each monitor will| The monitor info | -| | | run with independently process | will be collected. | -| +------+----------------------------------+--------------------+ -| | 2 | do attacker: connect the host | the host will be | -| | | through SSH, and then execute | shutdown. | -| | | shutdown script on the host | | -| +------+----------------------------------+--------------------+ -| | 3 | stop monitors after a period of | All monitor result | -| | | time specified by "waiting_time" | will be aggregated.| -| +------+----------------------------------+--------------------+ -| | 4 | verify the SLA | The test case is | -| | | | passed or not. | -+--------------+------+----------------------------------+--------------------+ -|post-action | It is the action when the test cases exist. It restarts the | -| | specified control node if it is not restarted. | -+--------------+------+----------------------------------+--------------------+ -|test verdict | Fails only if SLA is not passed, or if there is a test case | -| | execution problem. | -+--------------+--------------------------------------------------------------+ -- cgit 1.2.3-korg