--- tiers: - name: smoke_cntt description: >- Set of basic Functional tests to validate the OPNFV scenarios. testcases: - case_name: tempest_neutron_cntt project_name: functest criteria: 100 blocking: false deny_skipping: true tests_count: 546 description: >- This test case runs the Tempest suite proposed by the Neutron project. The list of test cases is generated by Tempest automatically and depends on the parameters of the OpenStack deployment. run: name: tempest_common args: mode: "(?!.*admin.test_agent_availability_zone)\ (?!.*admin.test_dhcp_agent_scheduler)\ (?!.*admin.test_l3_agent_scheduler)\ (?!.*admin.test_logging)\ (?!.*admin.test_logging_negative)\ (?!.*admin.test_network_segment_range)\ (?!.*admin.test_ports.PortTestCasesAdmin.test_regenerate_mac_address)\ (?!.*admin.test_ports.PortTestCasesResourceRequest)\ (?!.*admin.test_routers_dvr)\ (?!.*admin.test_routers_flavors)\ (?!.*admin.test_routers_ha)\ (?!.*test_conntrack_helper)\ (?!.*test_floating_ips.FloatingIPPoolTestJSON)\ (?!.*test_floating_ips.FloatingIPTestJSON.test_create_update_floatingip_port_details)\ (?!.*test_metering_extensions)\ (?!.*test_metering_negative)\ (?!.*test_networks.NetworksSearchCriteriaTest.test_list_validation_filters)\ (?!.*test_networks.NetworksTestAdmin.test_create_tenant_network_vxlan)\ (?!.*test_networks.NetworksTestJSON.test_create_update_network_dns_domain)\ (?!.*test_port_forwarding_negative)\ (?!.*test_port_forwardings)\ (?!.*test_ports.PortsTaggingOnCreation)\ (?!.*test_ports.PortsTestJSON.test_create_port_with_propagate_uplink_status)\ (?!.*test_ports.PortsTestJSON.test_create_port_without_propagate_uplink_status)\ (?!.*test_ports.PortsTestJSON.test_create_update_port_with_dns_domain)\ (?!.*test_ports.PortsTestJSON.test_create_update_port_with_dns_name)\ (?!.*test_ports.PortsTestJSON.test_create_update_port_with_no_dns_name)\ (?!.*test_qos.QosMinimumBandwidthRuleTestJSON)\ (?!.*test_revisions.TestRevisions.test_update_dns_domain_bumps_revision)\ (?!.*test_revisions.TestRevisions.test_update_router_extra_attributes_bumps_revision)\ (?!.*test_router_interface_fip)\ (?!.*test_routers.DvrRoutersTest)\ (?!.*test_routers.HaRoutersTest)\ (?!.*test_routers.RoutersIpV6Test.test_extra_routes_atomic)\ (?!.*test_routers.RoutersTest.test_extra_routes_atomic)\ (?!.*test_routers_negative.DvrRoutersNegativeTest)\ (?!.*test_routers_negative.DvrRoutersNegativeTestExtended)\ (?!.*test_routers_negative.HaRoutersNegativeTest)\ (?!.*test_security_groups.RbacSharedSecurityGroupTest)\ (?!.*test_subnetpool_prefix_ops)\ (?!.*test_subnetpools.RbacSubnetPoolTest)\ (?!.*test_subnetpools.SubnetPoolsSearchCriteriaTest.test_list_validation_filters)\ (?!.*test_subnetpools_negative.SubnetPoolsNegativeTestJSON.test_tenant_create_subnetpool_associate_shared_address_scope)\ (?!.*test_subnets.SubnetsSearchCriteriaTest.test_list_validation_filters)\ (?!.*test_timestamp.TestTimeStamp.test_segment_with_timestamp)\ (?!.*test_trunk.TrunkTestInheritJSONBase.test_add_subport)\ (?!.*test_trunk.TrunkTestMtusJSON)\ (?!.*test_trunk_negative.TrunkTestJSON.test_create_subport_invalid_inherit_network_segmentation_type)\ (?!.*test_trunk_negative.TrunkTestMtusJSON)\ (^neutron_tempest_plugin.api)" option: - '--concurrency=4' - case_name: tempest_cinder_cntt project_name: functest criteria: 100 blocking: false deny_skipping: true tests_count: 10 description: >- This test case runs the Tempest suite proposed by the Cinder project. run: name: tempest_common args: mode: "(?!.*test_incremental_backup)\ (?!.*test_consistencygroups)\ (?!.*test_backup_crossproject_admin_negative)\ (?!.*test_backup_crossproject_user_negative)\ (?!.*test_volume_encrypted.TestEncryptedCinderVolumes)\ (?!.*rbac)\ (^cinder_tempest_plugin.)" option: - '--concurrency=4' - case_name: tempest_keystone_cntt project_name: functest criteria: 100 blocking: false deny_skipping: true tests_count: 27 description: >- This test case runs the Tempest suite proposed by the Keystone project. run: name: tempest_common args: mode: "(?!.*api.identity.v3.test_oauth1_tokens)\ (?!.*rbac)\ (?!.*scenario.test_federated_authentication)\ keystone_tempest_plugin." option: - '--concurrency=4' - case_name: tempest_heat_cntt project_name: functest criteria: 100 blocking: false deny_skipping: true tests_count: 125 description: >- This test case runs the Tempest suite proposed by the Heat project. run: name: tempest_heat args: mode: "(?!.*functional.test_lbaasv2)\ (?!.*functional.test_encryption_vol_type)\ (?!.*RemoteStackTest.test_stack_create_with_cloud_credential)\ (?!.*scenario.test_aodh_alarm)\ (?!.*tests.scenario.test_autoscaling_lb)\ (?!.*scenario.test_autoscaling_lbv2)\ (?!.*scenario.test_server_software_config)\ (?!.*test_volumes.VolumeBackupRestoreIntegrationTest)\ (?!.*scenario.test_octavia_lbaas)\ (?!.*scenario.test_server_cfn_init)\ ^heat_tempest_plugin.tests" option: - '--concurrency=1' - case_name: rally_sanity_cntt project_name: functest criteria: 100 blocking: false description: >- This test case runs a sub group of tests of the OpenStack Rally suite in smoke mode. run: name: rally_sanity args: tests: - 'authenticate' - 'glance' - 'cinder' - 'heat' - 'keystone' - 'neutron' - 'nova' - 'quotas' - 'swift' - case_name: tempest_full_cntt project_name: functest criteria: 100 blocking: false deny_skipping: true tests_count: 1269 description: >- The list of test cases is generated by Tempest automatically and depends on the parameters of the OpenStack deployment. https://github.com/openstack/tempest/blob/18.0.0/tox.ini#L83 run: name: tempest_common args: mode: "(?!.*admin.test_agents)(?!.*test_fixed_ips)\ (?!.*test_fixed_ips_negative)\ (?!.*test_auto_allocate_network)(?!.*test_floating_ips_bulk)\ (?!.*test_flavors_microversions.FlavorsV255TestJSON)\ (?!.*test_flavors_microversions.FlavorsV261TestJSON)\ (?!.*test_live_migration.LiveAutoBlockMigrationV225Test.test_iscsi_volume)\ (?!.*test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration)\ (?!.*test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration_paused)\ (?!.*test_live_migration.LiveAutoBlockMigrationV225Test.test_volume_backed_live_migration)\ (?!.*test_live_migration.LiveMigrationTest.test_iscsi_volume)\ (?!.*test_live_migration.LiveMigrationTest.test_live_block_migration)\ (?!.*test_live_migration.LiveMigrationTest.test_live_block_migration_paused)\ (?!.*test_live_migration.LiveMigrationTest.test_volume_backed_live_migration)\ (?!.*test_live_migration.LiveMigrationRemoteConsolesV26Test)\ (?!.*test_quotas.QuotasAdminTestV257)\ (?!.*test_servers.ServersAdminTestJSON.test_reset_network_inject_network_info)\ (?!.*certificates.test_certificates)\ (?!.*test_quotas_negative.QuotasSecurityGroupAdminNegativeTest)\ (?!.*test_novnc)(?!.*test_server_personality)\ (?!.*test_servers.ServerShowV263Test.test_show_update_rebuild_list_server)\ (?!.*test_servers_microversions.ServerShowV254Test)\ (?!.*test_servers_microversions.ServerShowV257Test)\ (?!.*test_servers_negative.ServersNegativeTestJSON.test_personality_file_contents_not_encoded)\ (?!.*servers.test_virtual_interfaces)\ (?!.*test_server_actions.ServerActionsTestJSON.test_change_server_password)\ (?!.*test_server_actions.ServerActionsTestJSON.test_get_vnc_console)\ (?!.*test_server_actions.ServerActionsTestJSON.test_reboot_server_soft)\ (?!.*test_server_rescue.ServerBootFromVolumeStableRescueTest)\ (?!.*test_server_rescue.ServerStableDeviceRescueTest)\ (?!.*test_security_group_default_rules)\ (?!.*test_security_groups_negative.SecurityGroupsNegativeTestJSON.test_security_group_create_with_duplicate_name)\ (?!.*test_security_groups_negative.SecurityGroupsNegativeTestJSON.test_security_group_create_with_invalid_group_description)\ (?!.*test_security_groups_negative.SecurityGroupsNegativeTestJSON.test_security_group_create_with_invalid_group_name)\ (?!.*test_security_groups_negative.SecurityGroupsNegativeTestJSON.test_update_security_group_with_invalid_sg_des)\ (?!.*test_security_groups_negative.SecurityGroupsNegativeTestJSON.test_update_security_group_with_invalid_sg_id)\ (?!.*test_security_groups_negative.SecurityGroupsNegativeTestJSON.test_update_security_group_with_invalid_sg_name)\ (?!.*test_create_server.ServersTestFqdnHostnames.test_create_server_with_fqdn_name)\ (?!.*test_server_metadata.ServerMetadataTestJSON)\ (?!.*test_server_metadata_negative.ServerMetadataNegativeTestJSON.test_delete_metadata_non_existent_server)\ (?!.*test_server_metadata_negative.ServerMetadataNegativeTestJSON.test_metadata_items_limit)\ (?!.*test_server_metadata_negative.ServerMetadataNegativeTestJSON.test_set_metadata_invalid_key)\ (?!.*test_server_metadata_negative.ServerMetadataNegativeTestJSON.test_set_metadata_non_existent_server)\ (?!.*test_server_metadata_negative.ServerMetadataNegativeTestJSON.test_set_server_metadata_blank_key)\ (?!.*test_server_metadata_negative.ServerMetadataNegativeTestJSON.test_set_server_metadata_missing_metadata)\ (?!.*test_server_metadata_negative.ServerMetadataNegativeTestJSON.test_update_metadata_non_existent_server)\ (?!.*test_server_metadata_negative.ServerMetadataNegativeTestJSON.test_update_metadata_with_blank_key)\ (?!.*test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filtered_by_ip_regex)\ (?!.*compute.test_virtual_interfaces)(?!.*compute.test_virtual_interfaces_negative)\ (?!.*compute.test_networks)\ (?!.*test_attach_volume.AttachVolumeMultiAttach)\ (?!.*identity.admin.v2)(?!.*identity.v2)\ (?!.*identity.v3.test_access_rules)\ (?!.*identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_create_application_credential_access_rules)\ (?!.*image.v1)\ (?!.*image.v2.admin.test_images.ImportCopyImagesTest)\ (?!.*image.v2.test_images_negative.ImagesNegativeTest.test_create_image_reserved_property)\ (?!.*image.v2.test_images_negative.ImagesNegativeTest.test_update_image_reserved_property)\ (?!.*image.v2.test_images_negative.ImportImagesNegativeTest.test_image_web_download_import_with_bad_url)\ (?!.*image.v2.test_images.ImportImagesTest)\ (?!.*image.v2.test_images.MultiStoresImportImages)\ (?!.*admin.test_dhcp_agent_scheduler)\ (?!.*admin.test_routers_dvr)\ (?!.*test_metering_extensions)(?!.*network.test_tags)\ (?!.*test_routers_negative.DvrRoutersNegativeTest)\ (?!.*test_routers.RoutersIpV6Test.test_create_router_set_gateway_with_fixed_ip)\ (?!.*test_routers.RoutersTest.test_create_router_set_gateway_with_fixed_ip)\ (?!.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding)\ (?!.*test_encrypted_volumes_extend)\ (?!.*test_group_snapshots.GroupSnapshotsV319Test.test_reset_group_snapshot_status)\ (?!.*test_multi_backend)\ (?!.*test_volume_retype.VolumeRetypeWithMigrationTest)\ (?!.*test_volume_delete_cascade.VolumesDeleteCascade.test_volume_from_snapshot_cascade_delete)\ (?!.*test_volumes_backup.VolumesBackupsTest.test_volume_backup_create_get_detailed_list_restore_delete)\ (?!.*test_volumes_negative.UpdateMultiattachVolumeNegativeTest.test_multiattach_rw_volume_update_failure)\ (?!.*test_volumes_extend.VolumesExtendAttachedTest.test_extend_attached_volume)\ (?!.*\\[.*\\bslow\\b.*\\])(^tempest.api)" option: - '--concurrency=4' - case_name: tempest_scenario_cntt project_name: functest criteria: 100 blocking: false deny_skipping: true tests_count: 10 description: >- The list of test cases is generated by Tempest automatically and depends on the parameters of the OpenStack deployment. https://github.com/openstack/tempest/blob/18.0.0/tox.ini#L84 run: name: tempest_common args: mode: "\ (?!.*test_minbw_allocation_placement)\ (?!.*test_unified_limits.ImageQuotaTest.test_image_count_uploading_quota)\ (?!.*test_unified_limits.ImageQuotaTest.test_image_stage_quota)\ (?!.*test_volume_boot_pattern.TestVolumeBootPattern.test_boot_server_from_encrypted_volume_luks)\ (?!.*\\[.*\\bslow\\b.*\\])(^tempest.scenario)" option: - '--concurrency=1' - case_name: tempest_slow_cntt project_name: functest criteria: 100 blocking: false deny_skipping: true tests_count: 43 description: >- The list of test cases is generated by Tempest automatically and depends on the parameters of the OpenStack deployment. https://github.com/openstack/tempest/blob/18.0.0/tox.ini#L84 run: name: tempest_common args: mode: "(?!.*test_volume_swap)\ (?!.*test_server_personality)\ (?!.*test_server_rescue.ServerBootFromVolumeStableRescueTest)\ (?!.*test_container_sync.ContainerSyncTest.test_container_synchronization)\ (?!.*test_container_sync_middleware.ContainerSyncMiddlewareTest.test_container_synchronization)\ (?!.*test_encrypted_cinder_volumes)\ (?!.*test_minbw_allocation_placement)\ (?!.*test_network_basic_ops.TestNetworkBasicOps.test_router_rescheduling)\ (?!.*test_shelve_instance.TestShelveInstance.test_cold_migrate_unshelved_instance)\ (?!.*test_volume_migrate_attached)\ (?!.*test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_cold_migration_revert)\ (?=.*\\[.*\\bslow\\b.*\\])(^tempest.)" option: - '--concurrency=1'