summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorhongbotian <hongbo.tianhongbo@huawei.com>2017-07-24 07:52:52 +0000
committerhongbotian <hongbo.tianhongbo@huawei.com>2017-08-10 08:40:09 +0000
commit159a024d34d8a07ab702326f5a1c3f7d2bf35a55 (patch)
tree5ad0b88c4fe8e94430a6852e051e7c3e24b37999
parent7ed1cf189676693d84c9d48bb964c178b16bae76 (diff)
Test case specification for VIM Image
1. Adding the alias VIM image test cases into testscope. 2. Adding test case specification for VIM Image. Change-Id: Ifa7aa5ddea22d2862ee5d8ad465523d450a78fde Signed-off-by: hongbotian <hongbo.tianhongbo@huawei.com>
-rw-r--r--docs/testing/developer/testscope/index.rst11
-rw-r--r--docs/testing/user/testspecification/vimoperationsimage/index.rst392
2 files changed, 386 insertions, 17 deletions
diff --git a/docs/testing/developer/testscope/index.rst b/docs/testing/developer/testscope/index.rst
index ab1577da..b2dceec4 100644
--- a/docs/testing/developer/testscope/index.rst
+++ b/docs/testing/developer/testscope/index.rst
@@ -167,12 +167,16 @@ Image get tests using the Glance v2 API
| tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_delete_deleted_image
| tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_image_null_id
| tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_non_existent_image
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_get_image_schema
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_get_images_schema
CRUD image operations in Images API v2
--------------------------------------
| tempest.api.image.v2.test_images.ListImagesTest.test_list_no_params
+| tempest.api.image.v2.test_images.ListImagesTest.test_index_no_params
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_list_no_params
Image list tests using the Glance v2 API
@@ -185,6 +189,13 @@ Image list tests using the Glance v2 API
| tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_size
| tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_status
| tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_visibility
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_container_format
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_disk_format
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_limit
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_min_max_size
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_size
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_status
+| tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_visibility
Image update tests using the Glance v2 API
diff --git a/docs/testing/user/testspecification/vimoperationsimage/index.rst b/docs/testing/user/testspecification/vimoperationsimage/index.rst
index 932c7382..8e5f08ce 100644
--- a/docs/testing/user/testspecification/vimoperationsimage/index.rst
+++ b/docs/testing/user/testspecification/vimoperationsimage/index.rst
@@ -1,6 +1,6 @@
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
-.. (c) Ericsson AB
+.. (c) Ericsson AB, Huawei Technologies Co.,Ltd
=======================================
VIM image operations test specification
@@ -9,17 +9,191 @@ VIM image operations test specification
.. toctree::
:maxdepth: 2
-Each test case requires documentation according to:
-* Use case specification
-* Test preconditions
-* Basic test flow execution descriptor
-* Post conditions and pass fail criteria
+Scope
+=====
+
+The VIM image test area evaluates the ability of the system under test to support
+VIM image operations. The test cases documented here are the Image API test cases
+in the Openstack Interop guideline 2016.8 as implemented by the Refstack client.
+These test cases will evaluate basic Openstack (as a VIM) image operations including
+image creation, image list, image update and image deletion capabilities using Glance v2 API.
+
+References
+==========
+
+- Defcore test cases
+
+ - https://github.com/openstack/interop/blob/master/2016.08.json
+
+- Refstack client
+
+ - https://github.com/openstack/refstack-client
+
+Definitions and abbreviations
+=============================
+
+The following terms and abbreviations are used in conjunction with this test area
+
+- API - Application Programming Interface
+- CRUD - Create, Read, Update, and Delete
+- NFVi - Network Functions Virtualization infrastructure
+- VIM - Virtual Infrastructure Manager
+
+System Under Test (SUT)
+=======================
+
+The system under test is assumed to be the NFVi and VIM in operation on a
+Pharos compliant infrastructure.
+
+Test Area Structure
+====================
+
+The test area is structured based on VIM image operations. Each test case is able
+to run independently, i.e. irrelevant of the state created by a previous test.
+
+For brevity, the test cases in this test area are summarized together based on
+the operations they are testing.
+
+Test Descriptions
+=================
+
+API Used and Reference
+----------------------
+
+Images: https://developer.openstack.org/api-ref/image/v2/
+
+- create image
+- delete image
+- show image details
+- show images
+- show image schema
+- show images schema
+- upload binary image data
+- add image tag
+- delete image tag
+
+---------------------------------------
+Image get tests using the Glance v2 API
+---------------------------------------
+
+Test case specification
+-----------------------
-tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_delete_image
-tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image
tempest.api.image.v2.test_images.ListImagesTest.test_get_image_schema
tempest.api.image.v2.test_images.ListImagesTest.test_get_images_schema
+tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_delete_deleted_image
+tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_image_null_id
+tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_non_existent_image
+
+tempest.api.image.v2.test_images.ListUserImagesTest.test_get_image_schema
+tempest.api.image.v2.test_images.ListUserImagesTest.test_get_images_schema
+
+Note: the latter two test cases are the alias of the former first two, respectively.
+Alias should always be included so that the test run will be tempest version agnostic,
+which can be used to test different version of Openstack.
+
+Test preconditions
+------------------
+
+Glance is available.
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Test execution
+''''''''''''''
+
+* Test action 1: Create 6 images and store their ids in a created images list.
+* Test action 2: Use image v2 API to show image schema and check the body of the response.
+* **Test assertion 1:** In the body of the response, the value of the key 'name' is 'image'.
+* Test action 3: Use image v2 API to show images schema and check the body of the response.
+* **Test assertion 2:** In the body of the response, the value of the key 'name' is 'images'.
+* Test action 4: Create an image with name 'test', container_formats 'bare' and
+ disk_formats 'raw'. Delete this image with its id and then try to show it with
+ its id. Delete this deleted image again with its id and check the API's response code.
+* **Test assertion 3:** The operations of showing and deleting a deleted image with its id
+ both get 404 response code.
+* Test action 5: Use a null image id to show a image and check the API's response code.
+* **Test assertion 4:** The API's response code is 404.
+* Test action 6: Generate a random uuid and use it as the image id to show the image.
+* **Test assertion 5:** The API's response code is 404.
+* Test action 7: Delete the 6 images with the stored ids. Show all images and check
+ whether the 6 images' ids are not in the show list.
+* **Test assertion 6:** The 6 images' ids are not found in the show list.
+
+Pass / fail criteria
+''''''''''''''''''''
+
+The first two test cases evaluate the ability to use Glance v2 API to show image
+and images schema. The latter three test cases evaluate the ability to use Glance
+v2 API to show images with a deleted image id, a null image id and a non-existing
+image id. Specifically it verifies that:
+
+* Glance image get API can show the image and images schema.
+* Glance image get API can't show an image with a deleted image id.
+* Glance image get API can't show an image with a null image id.
+* Glance image get API can't show an image with a non-existing image id.
+
+In order to pass this test, all test assertions listed in the test execution above need to pass.
+
+Post conditions
+---------------
+
+None
+
+--------------------------------------
+CRUD image operations in Images API v2
+--------------------------------------
+
+Test case specification
+-----------------------
+
+tempest.api.image.v2.test_images.ListImagesTest.test_list_no_params
+
tempest.api.image.v2.test_images.ListImagesTest.test_index_no_params
+tempest.api.image.v2.test_images.ListUserImagesTest.test_list_no_params
+
+Note: the latter two test cases are the alias of the former one. Alias should
+always be included so that the test run will be tempest version agnostic,
+which can be used to test different version of Openstack.
+
+Test preconditions
+------------------
+
+Glance is available.
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Test execution
+''''''''''''''
+
+* Test action 1: Create 6 images and store their ids in a created images list.
+* Test action 2: List all images and check whether the ids listed are in the created images list.
+* **Test assertion 1:** The ids get from the list images API are in the created images list.
+
+Pass / fail criteria
+''''''''''''''''''''
+
+This test case evaluates the ability to use Glance v2 API to list images.
+Specifically it verifies that:
+
+* Glance image API can show the images.
+
+In order to pass this test, all test assertions listed in the test execution above need to pass.
+
+Post conditions
+---------------
+
+None
+
+----------------------------------------
+Image list tests using the Glance v2 API
+----------------------------------------
+
+Test case specification
+-----------------------
+
tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_container_format
tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_disk_format
tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_limit
@@ -27,9 +201,7 @@ tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_min_max_s
tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_size
tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_status
tempest.api.image.v2.test_images.ListImagesTest.test_list_images_param_visibility
-tempest.api.image.v2.test_images.ListImagesTest.test_list_no_params
-tempest.api.image.v2.test_images.ListUserImagesTest.test_get_image_schema
-tempest.api.image.v2.test_images.ListUserImagesTest.test_get_images_schema
+
tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_container_format
tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_disk_format
tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_limit
@@ -37,12 +209,198 @@ tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_min_m
tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_size
tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_status
tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_visibility
-tempest.api.image.v2.test_images.ListUserImagesTest.test_list_no_params
+
+Note: the latter 7 test cases are the alias of the former 7, respectively.
+Alias should always be included so that the test run will be tempest version agnostic,
+which can be used to test different version of Openstack.
+
+Test preconditions
+------------------
+
+Glance is available.
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Test execution
+''''''''''''''
+
+* Test action 1: Create 6 images with a random size ranging from 1024 to 4096 and
+ visibility 'private'; set their (container_format, disk_format) pair to be
+ (ami, ami), (ami, ari), (ami, aki), (ami, vhd), (ami, vmdk) and (ami, raw);
+ store their ids in a list and upload the binary images data.
+* Test action 2: Use Glance v2 API to list all images whose container_format is 'ami'
+ and store the response details in a list.
+* **Test assertion 1:** The list is not empty and all the values of container_format
+ in the list are 'ami'.
+* Test action 3: Use Glance v2 API to list all images whose disk_format is 'raw'
+ and store the response details in a list.
+* **Test assertion 2:** The list is not empty and all the values of disk_format
+ in the list are 'raw'.
+* Test action 4: Use Glance v2 API to list one image by setting limit to be 1 and
+ store the response details in a list.
+* **Test assertion 3:** The length of the list is one.
+* Test action 5: Use Glance v2 API to list images by setting size_min and size_max,
+ and store the response images' sizes in a list. Choose the first image's size as
+ the median, size_min is median-500 and size_max is median+500.
+* **Test assertion 4:** All sizes in the list are no less than size_min and no more
+ than size_max.
+* Test action 6: Use Glance v2 API to show the first created image with its id and
+ get its size from the response. Use Glance v2 API to list images whose size is equal
+ to this size and store the response details in a list.
+* **Test assertion 5:** All sizes of the images in the list are equal to the size
+ used to list the images.
+* Test action 7: Use Glance v2 API to list the images whose status is active and
+ store the response details in a list.
+* **Test assertion 6:** All status of images in the list are active.
+* Test action 8: Use Glance v2 API to list the images whose visibility is private and
+ store the response details in a list.
+* **Test assertion 7:** All images' values of visibility in the list are private.
+* Test action 9: Delete the 6 images with the stored ids. Show images and check whether
+ the 6 ids are not in the show list.
+* **Test assertion 8:** The stored 6 ids are not found in the show list.
+
+Pass / fail criteria
+''''''''''''''''''''
+
+This test case evaluates the ability to use Glance v2 API to list images with
+different parameters. Specifically it verifies that:
+
+* Glance image API can show the images with the container_format.
+* Glance image API can show the images with the disk_format.
+* Glance image API can show the images by setting a limit number.
+* Glance image API can show the images with the size_min and size_max.
+* Glance image API can show the images with the size.
+* Glance image API can show the images with the status.
+* Glance image API can show the images with the visibility type.
+
+In order to pass this test, all test assertions listed in the test execution above need to pass.
+
+Post conditions
+---------------
+
+None
+
+------------------------------------------
+Image update tests using the Glance v2 API
+------------------------------------------
+
+Test case specification
+-----------------------
+
+tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image
+tempest.api.image.v2.test_images_tags.ImagesTagsTest.test_update_delete_tags_for_image
+tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_update_tags_for_non_existing_image
+
+Test preconditions
+------------------
+
+Glance is available.
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Test execution
+''''''''''''''
+
+* Test action 1: Create an image with container_formats 'ami', disk_formats 'ami'
+ and visibility 'private' and store its id returned in the response. Check whether
+ the status of the created image is 'queued'.
+* **Test assertion 1:** The status of the created image is 'queued'.
+* Test action 2: Use the stored image id to upload the binary image data and update
+ this image's name. Show this image with the stored id. Check if the stored id and
+ name used to update the image are equal to the id and name in the show list.
+* **Test assertion 2:** The id and name returned in the show list are equal to
+ the stored id and name used to update the image.
+* Test action 3: Create an image with container_formats 'bare', disk_formats 'raw'
+ and visibility 'private' and store its id returned in the response.
+* Test action 4: Use the stored id to add a tag. Show the image with the stored id
+ and check if the tag used to add is in the image's tags returned in the show list.
+* **Test assertion 3:** The tag used to add into the image is in the show list.
+* Test action 5: Use the stored id to delete this tag. Show the image with the
+ stored id and check if the tag used to delete is not in the show list.
+* **Test assertion 4:** The tag used to delete from the image is not in the show list.
+* Test action 6: Generate a random uuid as the image id. Use the image id to add a tag
+ into the image's tags.
+* **Test assertion 5:** The API's response code is 404.
+* Test action 7: Delete the images created in test action 1 and 3. Show the images
+ and check whether the ids are not in the show list.
+* **Test assertion 6:** The two ids are not found in the show list.
+
+Pass / fail criteria
+''''''''''''''''''''
+
+This test case evaluates the ability to use Glance v2 API to update images with
+different parameters. Specifically it verifies that:
+
+* Glance image API can update image's name with the existing image id.
+* Glance image API can update image's tags with the existing image id.
+* Glance image API can't update image's tags with a non-existing image id.
+
+In order to pass this test, all test assertions listed in the test execution above need to pass.
+
+Post conditions
+---------------
+
+None
+
+--------------------------------------------
+Image deletion tests using the Glance v2 API
+--------------------------------------------
+
+Test case specification
+-----------------------
+
+tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_delete_image
tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_delete_image_null_id
tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_delete_non_existing_image
-tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_delete_deleted_image
-tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_image_null_id
-tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_non_existent_image
-tempest.api.image.v2.test_images_tags.ImagesTagsTest.test_update_delete_tags_for_image
tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_delete_non_existing_tag
-tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_update_tags_for_non_existing_image
+
+Test preconditions
+------------------
+
+Glance is available.
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Test execution
+''''''''''''''
+
+* Test action 1: Create an image with container_formats 'ami', disk_formats 'ami'
+ and visibility 'private'. Use the id of the created image to delete the image.
+ List all images and check whether this id is in the list.
+* **Test assertion 1:** The id of the created image is not found in the list
+ of all images after the deletion operation.
+* Test action 2: Delete images with a null id and check the API's response code.
+* **Test assertion 2:** The API's response code is 404.
+* Test action 3: Generate a random uuid and delete images with this uuid as image id.
+ Check the API's response code.
+* **Test assertion 3:** The API's response code is 404.
+* Test action 4: Create an image with container_formats 'bare', disk_formats 'raw'
+ and visibility 'private'. Delete this image's tag with the image id and a random tag
+ Check the API's response code.
+* **Test assertion 4:** The API's response code is 404.
+* Test action 5: Delete the images created in test action 1 and 4. List all images
+ and check whether the ids are in the list.
+* **Test assertion 5:** The two ids are not found in the list.
+
+Pass / fail criteria
+''''''''''''''''''''
+
+The first three test cases evaluate the ability to use Glance v2 API to delete images
+with an existing image id, a null image id and a non-existing image id. The last one
+evaluates the ability to use the API to delete a non-existing image tag.
+Specifically it verifies that:
+
+* Glance image deletion API can delete the image with an existing id.
+* Glance image deletion API can't delete an image with a null image id.
+* Glance image deletion API can't delete an image with a non-existing image id.
+* Glance image deletion API can't delete an image tag with a non-existing image tag.
+
+In order to pass this test, all test assertions listed in the test execution above need to pass.
+
+Post conditions
+---------------
+
+None