diff options
Diffstat (limited to 'docs')
30 files changed, 962 insertions, 516 deletions
diff --git a/docs/conf.py b/docs/conf.py index eb12e74b..3c9978bb 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -1 +1,2 @@ from docs_conf.conf import * # noqa: F401,F403 +master_doc = 'index' diff --git a/docs/development/index.rst b/docs/development/index.rst index 2dc16a82..a7d2817b 100644 --- a/docs/development/index.rst +++ b/docs/development/index.rst @@ -2,18 +2,18 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) 2016 OPNFV. +.. _development: -====== -Doctor -====== +=========== +Development +=========== .. toctree:: :maxdepth: 2 - ./design/index.rst - ./requirements/index.rst - ./manuals/index.rst - ./overview/functest_scenario/index.rst + ./design/index + ./overview/index + ./requirements/index Indices ======= diff --git a/docs/development/overview/functest_scenario/doctor-scenario-in-functest.rst b/docs/development/overview/functest_scenario/doctor-scenario-in-functest.rst deleted file mode 100644 index 9f92b5bf..00000000 --- a/docs/development/overview/functest_scenario/doctor-scenario-in-functest.rst +++ /dev/null @@ -1,253 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - - - -Platform overview -""""""""""""""""" - -Doctor platform provides these features since `Danube Release <https://wiki.opnfv.org/display/SWREL/Danube>`_: - -* Immediate Notification -* Consistent resource state awareness for compute host down -* Valid compute host status given to VM owner - -These features enable high availability of Network Services on top of -the virtualized infrastructure. Immediate notification allows VNF managers -(VNFM) to process recovery actions promptly once a failure has occurred. -Same framework can also be utilized to have VNFM awareness about -infrastructure maintenance. - -Consistency of resource state is necessary to execute recovery actions -properly in the VIM. - -Ability to query host status gives VM owner the possibility to get -consistent state information through an API in case of a compute host -fault. - -The Doctor platform consists of the following components: - -* OpenStack Compute (Nova) -* OpenStack Networking (Neutron) -* OpenStack Telemetry (Ceilometer) -* OpenStack Alarming (AODH) -* Doctor Sample Inspector, OpenStack Congress or OpenStack Vitrage -* Doctor Sample Monitor or any monitor supported by Congress or Vitrage - -.. note:: - Doctor Sample Monitor is used in Doctor testing. However in real - implementation like Vitrage, there are several other monitors supported. - -You can see an overview of the Doctor platform and how components interact in -:numref:`figure-p1`. - -.. figure:: ./images/Fault-management-design.png - :name: figure-p1 - :width: 100% - - Doctor platform and typical sequence - -Detailed information on the Doctor architecture can be found in the Doctor -requirements documentation: -http://artifacts.opnfv.org/doctor/docs/requirements/05-implementation.html - -Running test cases -"""""""""""""""""" - -Functest will call the "doctor_tests/main.py" in Doctor to run the test job. -Doctor testing can also be triggered by tox on OPNFV installer jumphost. Tox -is normally used for functional, module and coding style testing in Python -project. - -Currently, 'Apex', 'Daisy', 'Fuel' and 'local' installer are supported. - - -Fault management use case -""""""""""""""""""""""""" - -* A consumer of the NFVI wants to receive immediate notifications about faults - in the NFVI affecting the proper functioning of the virtual resources. - Therefore, such faults have to be detected as quickly as possible, and, when - a critical error is observed, the affected consumer is immediately informed - about the fault and can switch over to the STBY configuration. - -The faults to be monitored (and at which detection rate) will be configured by -the consumer. Once a fault is detected, the Inspector in the Doctor -architecture will check the resource map maintained by the Controller, to find -out which virtual resources are affected and then update the resources state. -The Notifier will receive the failure event requests sent from the Controller, -and notify the consumer(s) of the affected resources according to the alarm -configuration. - -Detailed workflow information is as follows: - -* Consumer(VNFM): (step 0) creates resources (network, server/instance) and an - event alarm on state down notification of that server/instance or Neutron - port. - -* Monitor: (step 1) periodically checks nodes, such as ping from/to each - dplane nic to/from gw of node, (step 2) once it fails to send out event - with "raw" fault event information to Inspector - -* Inspector: when it receives an event, it will (step 3) mark the host down - ("mark-host-down"), (step 4) map the PM to VM, and change the VM status to - down. In network failure case, also Neutron port is changed to down. - -* Controller: (step 5) sends out instance update event to Ceilometer. In network - failure case, also Neutron port is changed to down and corresponding event is - sent to Ceilometer. - -* Notifier: (step 6) Ceilometer transforms and passes the events to AODH, - (step 7) AODH will evaluate events with the registered alarm definitions, - then (step 8) it will fire the alarm to the "consumer" who owns the - instance - -* Consumer(VNFM): (step 9) receives the event and (step 10) recreates a new - instance - -Fault management test case -"""""""""""""""""""""""""" - -Functest will call the 'doctor-test' command in Doctor to run the test job. - -The following steps are executed: - -Firstly, get the installer ip according to the installer type. Then ssh to -the installer node to get the private key for accessing to the cloud. As -'fuel' installer, ssh to the controller node to modify nova and ceilometer -configurations. - -Secondly, prepare image for booting VM, then create a test project and test -user (both default to doctor) for the Doctor tests. - -Thirdly, boot a VM under the doctor project and check the VM status to verify -that the VM is launched completely. Then get the compute host info where the VM -is launched to verify connectivity to the target compute host. Get the consumer -ip according to the route to compute ip and create an alarm event in Ceilometer -using the consumer ip. - -Fourthly, the Doctor components are started, and, based on the above preparation, -a failure is injected to the system, i.e. the network of compute host is -disabled for 3 minutes. To ensure the host is down, the status of the host -will be checked. - -Finally, the notification time, i.e. the time between the execution of step 2 -(Monitor detects failure) and step 9 (Consumer receives failure notification) -is calculated. - -According to the Doctor requirements, the Doctor test is successful if the -notification time is below 1 second. - -Maintenance use case -"""""""""""""""""""" - -* A consumer of the NFVI wants to interact with NFVI maintenance, upgrade, - scaling and to have graceful retirement. Receiving notifications over these - NFVI events and responding to those within given time window, consumer can - guarantee zero downtime to his service. - -The maintenance use case adds the Doctor platform an `admin tool` and an -`app manager` component. Overview of maintenance components can be seen in -:numref:`figure-p2`. - -.. figure:: ./images/Maintenance-design.png - :name: figure-p2 - :width: 100% - - Doctor platform components in maintenance use case - -In maintenance use case, `app manager` (VNFM) will subscribe to maintenance -notifications triggered by project specific alarms through AODH. This is the way -it gets to know different NFVI maintenance, upgrade and scaling operations that -effect to its instances. The `app manager` can do actions depicted in `green -color` or tell `admin tool` to do admin actions depicted in `orange color` - -Any infrastructure component like `Inspector` can subscribe to maintenance -notifications triggered by host specific alarms through AODH. Subscribing to the -notifications needs admin privileges and can tell when a host is out of use as -in maintenance and when it is taken back to production. - -Maintenance test case -""""""""""""""""""""" - -Maintenance test case is currently running in our Apex CI and executed by tox. -This is because the special limitation mentioned below and also the fact we -currently have only sample implementation as a proof of concept. Environmental -variable TEST_CASE='maintenance' needs to be used when executing -"doctor_tests/main.py". Test case workflow can be seen in :numref:`figure-p3`. - -.. figure:: ./images/Maintenance-workflow.png - :name: figure-p3 - :width: 100% - - Maintenance test case workflow - -In test case all compute capacity will be consumed with project (VNF) instances. -For redundant services on instances and an empty compute needed for maintenance, -test case will need at least 3 compute nodes in system. There will be 2 -instances on each compute, so minimum number of VCPUs is also 2. Depending on -how many compute nodes there is application will always have 2 redundant -instances (ACT-STDBY) on different compute nodes and rest of the compute -capacity will be filled with non-redundant instances. - -For each project specific maintenance message there is a time window for -`app manager` to make any needed action. This will guarantee zero -down time for his service. All replies back are done by calling `admin tool` API -given in the message. - -The following steps are executed: - -Infrastructure admin will call `admin tool` API to trigger maintenance for -compute hosts having instances belonging to a VNF. - -Project specific `MAINTENANCE` notification is triggered to tell `app manager` -that his instances are going to hit by infrastructure maintenance at a specific -point in time. `app manager` will call `admin tool` API to answer back -`ACK_MAINTENANCE`. - -When the time comes to start the actual maintenance workflow in `admin tool`, -a `DOWN_SCALE` notification is triggered as there is no empty compute node for -maintenance (or compute upgrade). Project receives corresponding alarm and scales -down instances and call `admin tool` API to answer back `ACK_DOWN_SCALE`. - -As it might happen instances are not scaled down (removed) from a single -compute node, `admin tool` might need to figure out what compute node should be -made empty first and send `PREPARE_MAINTENANCE` to project telling which instance -needs to be migrated to have the needed empty compute. `app manager` makes sure -he is ready to migrate instance and call `admin tool` API to answer back -`ACK_PREPARE_MAINTENANCE`. `admin tool` will make the migration and answer -`ADMIN_ACTION_DONE`, so `app manager` knows instance can be again used. - -:numref:`figure-p3` has next a light blue section of actions to be done for each -compute. However as we now have one empty compute, we will maintain/upgrade that -first. So on first round, we can straight put compute in maintenance and send -admin level host specific `IN_MAINTENANCE` message. This is caught by `Inspector` -to know host is down for maintenance. `Inspector` can now disable any automatic -fault management actions for the host as it can be down for a purpose. After -`admin tool` has completed maintenance/upgrade `MAINTENANCE_COMPLETE` message -is sent to tell host is back in production. - -Next rounds we always have instances on compute, so we need to have -`PLANNED_MAINTANANCE` message to tell that those instances are now going to hit -by maintenance. When `app manager` now receives this message, he knows instances -to be moved away from compute will now move to already maintained/upgraded host. -In test case no upgrade is done on application side to upgrade instances -according to new infrastructure capabilities, but this could be done here as -this information is also passed in the message. This might be just upgrading -some RPMs, but also totally re-instantiating instance with a new flavor. Now if -application runs an active side of a redundant instance on this compute, -a switch over will be done. After `app manager` is ready he will call -`admin tool` API to answer back `ACK_PLANNED_MAINTENANCE`. In test case the -answer is `migrate`, so `admin tool` will migrate instances and reply -`ADMIN_ACTION_DONE` and then `app manager` knows instances can be again used. -Then we are ready to make the actual maintenance as previously trough -`IN_MAINTENANCE` and `MAINTENANCE_COMPLETE` steps. - -After all computes are maintained, `admin tool` can send `MAINTENANCE_COMPLETE` -to tell maintenance/upgrade is now complete. For `app manager` this means he -can scale back to full capacity. - -This is the current sample implementation and test case. Real life -implementation is started in OpenStack Fenix project and there we should -eventually address requirements more deeply and update the test case with Fenix -implementation. diff --git a/docs/development/overview/index.rst b/docs/development/overview/index.rst index 956e73e3..f6d78d57 100644 --- a/docs/development/overview/index.rst +++ b/docs/development/overview/index.rst @@ -3,11 +3,12 @@ .. _doctor-overview: -************************ -Doctor Development Guide -************************ +******** +Overview +******** .. toctree:: :maxdepth: 2 + overview.rst testing.rst diff --git a/docs/development/overview/overview.rst b/docs/development/overview/overview.rst new file mode 100644 index 00000000..21f5439e --- /dev/null +++ b/docs/development/overview/overview.rst @@ -0,0 +1,52 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Platform overview +""""""""""""""""" + +Doctor platform provides these features since `Danube Release <https://wiki.opnfv.org/display/SWREL/Danube>`_: + +* Immediate Notification +* Consistent resource state awareness for compute host down +* Valid compute host status given to VM owner + +These features enable high availability of Network Services on top of +the virtualized infrastructure. Immediate notification allows VNF managers +(VNFM) to process recovery actions promptly once a failure has occurred. +Same framework can also be utilized to have VNFM awareness about +infrastructure maintenance. + +Consistency of resource state is necessary to execute recovery actions +properly in the VIM. + +Ability to query host status gives VM owner the possibility to get +consistent state information through an API in case of a compute host +fault. + +The Doctor platform consists of the following components: + +* OpenStack Compute (Nova) +* OpenStack Networking (Neutron) +* OpenStack Telemetry (Ceilometer) +* OpenStack Alarming (AODH) +* Doctor Sample Inspector, OpenStack Congress or OpenStack Vitrage +* Doctor Sample Monitor or any monitor supported by Congress or Vitrage + +.. note:: + Doctor Sample Monitor is used in Doctor testing. However in real + implementation like Vitrage, there are several other monitors supported. + +You can see an overview of the Doctor platform and how components interact in +:numref:`figure-p1`. + + +Maintenance use case provides these features since `Iruya Release <https://wiki.opnfv.org/display/SWREL/Iruya>`_: + +* Infrastructure maintenance and upgrade workflow +* Interaction between VNFM and infrastructe workflow + +Since `Jerma Release <https://wiki.opnfv.org/display/SWREL/Jerma>`_ maintenance +use case also supports 'ETSI FEAT03' implementation to have the infrastructure +maintenance and upgrade fully optimized while keeping zero impact on VNF +service. + diff --git a/docs/development/requirements/index.rst b/docs/development/requirements/index.rst index fceaebf0..ccc35cb8 100644 --- a/docs/development/requirements/index.rst +++ b/docs/development/requirements/index.rst @@ -3,9 +3,9 @@ .. _doctor-requirements: -**************************************** -Doctor: Fault Management and Maintenance -**************************************** +********************************************** +Requirements: Fault Management and Maintenance +********************************************** :Project: Doctor, https://wiki.opnfv.org/doctor :Editors: Ashiq Khan (NTT DOCOMO), Gerald Kunzmann (NTT DOCOMO) diff --git a/docs/index.rst b/docs/index.rst index 4dedb98d..b8e8bfd0 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -12,6 +12,6 @@ Fault Management and Maintenance (Doctor) :numbered: :maxdepth: 2 - release/index development/index - + release/index + testing/index diff --git a/docs/release/configguide/feature.configuration.rst b/docs/release/configguide/feature.configuration.rst index 64928eea..8fbff50e 100644 --- a/docs/release/configguide/feature.configuration.rst +++ b/docs/release/configguide/feature.configuration.rst @@ -159,3 +159,57 @@ You can configure the Sample Monitor as follows (Example for Apex deployment): "http://127.0.0.1:$INSPECTOR_PORT/events" > monitor.log 2>&1 & **Collectd Monitor** + +OpenStack components +==================== + +In OPNFV and with Doctor testing you can have all OpenStack components configured +as needed. Here is sample of the needed configuration modifications. + +Ceilometer +---------- + +/etc/ceilometer/event_definitions.yaml: +# Maintenance use case needs new alarm definitions to be added +- event_type: maintenance.scheduled + traits: + actions_at: + fields: payload.maintenance_at + type: datetime + allowed_actions: + fields: payload.allowed_actions + host_id: + fields: payload.host_id + instances: + fields: payload.instances + metadata: + fields: payload.metadata + project_id: + fields: payload.project_id + reply_url: + fields: payload.reply_url + session_id: + fields: payload.session_id + state: + fields: payload.state +- event_type: maintenance.host + traits: + host: + fields: payload.host + project_id: + fields: payload.project_id + session_id: + fields: payload.session_id + state: + fields: payload.state + +/etc/ceilometer/event_pipeline.yaml: +# Maintenance and Fault management both needs these to be added + - notifier:// + - notifier://?topic=alarm.all + +Nova +---- + +/etc/nova/nova.conf +cpu_allocation_ratio=1.0 diff --git a/docs/release/configguide/index.rst b/docs/release/configguide/index.rst index b1e7c33d..c2331115 100644 --- a/docs/release/configguide/index.rst +++ b/docs/release/configguide/index.rst @@ -3,9 +3,9 @@ .. _doctor-configguide: -************************* -Doctor Installation Guide -************************* +************************** +Doctor Configuration Guide +************************** .. toctree:: :maxdepth: 2 diff --git a/docs/release/index.rst b/docs/release/index.rst index 8a1bf405..67eb4c5f 100644 --- a/docs/release/index.rst +++ b/docs/release/index.rst @@ -2,14 +2,18 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) 2017 OPNFV. +.. _release: -====== -Doctor -====== +======= +Release +======= .. toctree:: :maxdepth: 2 + ./configguide/index.rst ./installation/index.rst + ./release-notes/index.rst + ./scenarios/fault_management/fault_management.rst + ./scenarios/maintenance/maintenance.rst ./userguide/index.rst - diff --git a/docs/development/manuals/index.rst b/docs/release/installation/index.rst index f705f94a..f6527e5d 100644 --- a/docs/development/manuals/index.rst +++ b/docs/release/installation/index.rst @@ -1,13 +1,13 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 -.. _doctor-manuals: +.. _doctor-configguide: -******* -Manuals -******* +************************* +Doctor Installation Guide +************************* .. toctree:: + :maxdepth: 2 -.. include:: mark-host-down_manual.rst -.. include:: get-valid-server-state.rst + installation.rst diff --git a/docs/release/installation/installation.rst b/docs/release/installation/installation.rst new file mode 100644 index 00000000..564f19fd --- /dev/null +++ b/docs/release/installation/installation.rst @@ -0,0 +1,44 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Doctor Installation +==================== + +You can clone doctor project in OPNFV installer jumphost or if you are not +in OPNFV environment you can clone Doctor to DevStack controller node + +git clone https://gerrit.opnfv.org/gerrit/doctor + +In DevStack controller here is a sample of including what Doctor testing +will require for sample fault management testing and for maintenance +testing using Fenix + +.. code-block:: bash + + git clone https://github.com/openstack/devstack -b stable/train + +.. code-block:: bash + + cd devstack vi local.conf + +.. code-block:: bash + + [[local|localrc]] + GIT_BASE=https://git.openstack.org + HOST_IP=<host_ip> + ADMIN_PASSWORD=admin + DATABASE_PASSWORD=admin + RABBIT_PASSWORD=admin + SERVICE_PASSWORD=admin + LOGFILE=/opt/stack/stack.sh.log + + PUBLIC_INTERFACE=eth0 + + CEILOMETER_EVENT_ALARM=True + + ENABLED_SERVICES=key,rabbit,mysql,fenix-engine,fenix-api,aodh-evaluator,aodh-notifier,aodh-api + + enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer stable/train + enable_plugin aodh https://git.openstack.org/openstack/aodh stable/train + enable_plugin gnocchi https://github.com/openstack/gnocchi + enable_plugin fenix https://opendev.org/x/fenix master diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst index 142bfacf..b525335e 100644 --- a/docs/release/release-notes/release-notes.rst +++ b/docs/release/release-notes/release-notes.rst @@ -2,140 +2,46 @@ .. http://creativecommons.org/licenses/by/4.0 -This document provides the release notes for Gambia of Doctor. +This document provides the release notes for Iruya version of Doctor. Important notes =============== -In Gambia release, Doctor has been working with our second use case over -maintenance. Design guideline is now done and test case exists with sample -maintenance workflow code implemented in Doctor. Work has also started to have -the real implementation done in the OpenStack Fenix project -https://wiki.openstack.org/wiki/Fenix. - -Doctor CI testing has now moved to use tox on jumphots instead of running test -through features container. Also in Apex we use OpenStack services running in -containers. Functest daily testing supports Doctor fault management test case -for Apex, Daisy and Fuel installers. This testing is done through features -container. - -In this release, Doctor has not been working with the fault management use case as -the basic framework has been already done. However, we might need to get back to -it later to better meet the tough industry requirements as well as requirements -from edge, containers and 5G. +Jerma release has mainly been for finalizing maintenance use case testing +supporting the ETSI FEAT03 defined interactino between VNFM and infrastructure. +This is mainly to have infrastructure maintenance and upgrade operations +opttimized as fast as they can while keeping VNFs on top with zero impact +on their service. +Further more this is the final release of Doctor and the more deep testing is +moving more to upstream projects like Fenix for the maintenance. Also in +this release we have made sure that all Doctor testing and any deeper testing +with ehe upstream projects can be done in DevStack. This also makes DevStack +the most important installer. Summary ======= -Gambia Doctor framework uses OpenStack Queens integrated into its test cases. -Compared to the previous release, the Heat project is also being used in the -maintenance test case. +Jerma Doctor framework uses OpenStack Train integrated into its test cases. Release Data ============ Doctor changes -+------------------------------------------+----------------------------------------------------------+ -| **commit-ID** | **Subject** | -+------------------------------------------+----------------------------------------------------------+ -| 5b3f5937e7b861fca46b2a6b2d6708866b800f95 | fix building docs | -+------------------------------------------+----------------------------------------------------------+ -| 2ca5924081ce4784f599437707bd32807aa155ce | Fix SSH client connection reset | -+------------------------------------------+----------------------------------------------------------+ -| baac6579556f8216b36db0d0f87f9c2d4f8b4ef5 | Support Apex with services in containers | -+------------------------------------------+----------------------------------------------------------+ -| 23bf63c4616040cb0d69cd26238af2a4a7c00a90 | fix the username to login undercloud in Apex | -+------------------------------------------+----------------------------------------------------------+ -| 61eb3927ada784cc3dffb5ddd17f66e47871f708 | Local Documentation Builds | -+------------------------------------------+----------------------------------------------------------+ -| 0f1dd4314b9e0247d9af7af6df2410462423aeca | Updated from global requirements | -+------------------------------------------+----------------------------------------------------------+ -| 2d4a9f0c0a93797da6534583f6e74553a4b634be | Fix links to remove references to submodules | -+------------------------------------------+----------------------------------------------------------+ -| 3ddc2392b0ed364eede49ff006d64df3ea456350 | Gambia release notes | -+------------------------------------------+----------------------------------------------------------+ -| 825a0a0dd5e8028129b782ed21c549586257b1c5 | delete doctor datasource in congress when cleanup | -+------------------------------------------+----------------------------------------------------------+ -| fcf53129ab2b18b84571faff13d7cb118b3a41b3 | run profile even the notification time is larger than 1S | -+------------------------------------------+----------------------------------------------------------+ -| 495965d0336d42fc36494c81fd15cee2f34c96e9 | Update and add test case | -+------------------------------------------+----------------------------------------------------------+ -| da25598a6a31abe0579ffed12d1719e5ff75f9a7 | bugfix: add doctor datasource in congress | -+------------------------------------------+----------------------------------------------------------+ -| f9e1e3b1ae4be80bc2dc61d9c4213c81c091ea72 | Update the maintenance design document | -+------------------------------------------+----------------------------------------------------------+ -| 4639f15e6db2f1480b41f6fbfd11d70312d4e421 | Add maintenance test code | -+------------------------------------------+----------------------------------------------------------+ -| b54cbc5dd2d32fcb27238680b4657ed384d021c5 | Add setup and cleanup for maintenance test | -+------------------------------------------+----------------------------------------------------------+ -| b2bb504032ac81a2ed3f404113b097d9ce3d7f14 | bugfix: kill the stunnel when cleanup | -+------------------------------------------+----------------------------------------------------------+ -| eaeb3c0f9dc9e6645a159d0a78b9fc181fce53d4 | add ssh_keyfile for connect to installer in Apex | -+------------------------------------------+----------------------------------------------------------+ -| dcbe7bf1c26052b0e95d209254e7273aa1eaace1 | Add tox and test case to testing document | -+------------------------------------------+----------------------------------------------------------+ -| 0f607cb5efd91ee497346b7f792dfa844d15595c | enlarge the time of link down | -+------------------------------------------+----------------------------------------------------------+ -| 1351038a65739b8d799820de515178326ad05f7b | bugfix: fix the filename of ssh tunnel | -+------------------------------------------+----------------------------------------------------------+ -| e70bf248daac03eee6b449cd1654d2ee6265dd8c | Use py34 instead of py35 | -+------------------------------------------+----------------------------------------------------------+ -| 2a60d460eaf018951456451077b7118b60219b32 | add INSPECTOR_TYPE and TEST_CASE to tox env | -+------------------------------------------+----------------------------------------------------------+ -| 2043ceeb08c1eca849daeb2b3696d385425ba061 | [consumer] fix default value for port number | -+------------------------------------------+----------------------------------------------------------+ - -Releng changes - -+------------------------------------------+-----------------------------------------------------------------------+ -| **commit-ID** | **Subject** | -+------------------------------------------+-----------------------------------------------------------------------+ -| c87309f5a75ccc5d595f708817b97793c24c4387 | Add Doctor maintenance job | -+------------------------------------------+-----------------------------------------------------------------------+ -| bd16a9756ffd0743e143f0f2f966da8dd666c7a3 | remove congress test in Daisy | -+------------------------------------------+-----------------------------------------------------------------------+ -| c47aaaa53c91aae93877f2532c72374beaa4eabe | remove fuel job in Doctor | -+------------------------------------------+-----------------------------------------------------------------------+ -| ab2fed2522eaf82ea7c63dd05008a37c56e825d0 | use 'workspace-cleanup' plugin in publisher | -+------------------------------------------+-----------------------------------------------------------------------+ -| 3aaed5cf40092744f1b87680b9205a2901baecf3 | clean the workspace in the publisher | -+------------------------------------------+-----------------------------------------------------------------------+ -| 50151eb3717edd4ddd996f3705fbe1732de7f3b7 | run tox with 'sudo' | -+------------------------------------------+-----------------------------------------------------------------------+ -| a3adc85ecb52f5d19ec4e9c49ca1ac35aa429ff9 | remove inspector variable form job template | -+------------------------------------------+-----------------------------------------------------------------------+ -| adfbaf2a3e8487e4c9152bf864a653a0425b8582 | run doctor tests with different inspectors in sequence | -+------------------------------------------+-----------------------------------------------------------------------+ -| 2e98e56224cd550cb3bf9798e420eece28139bd9 | add the ssh_key info if the key_file is exist | -+------------------------------------------+-----------------------------------------------------------------------+ -| c109c271018e9a85d94be1b9b468338d64589684 | prepare installer info for doctor test | -+------------------------------------------+-----------------------------------------------------------------------+ -| 57cbefc7160958eae1d49e4753779180a25864af | use py34 for tox | -+------------------------------------------+-----------------------------------------------------------------------+ -| 3547754e808a581b09c9d22e013a7d986d9f6cd1 | specify the cacert file when it exits | -+------------------------------------------+-----------------------------------------------------------------------+ -| ef4f36aa1c2ff0819d73cde44f84b99a42e15c7e | bugfix: wrong usage of '!include-raw' | -+------------------------------------------+-----------------------------------------------------------------------+ -| 0e0e0d4cb71fb27b1789a2bef2d3c4ff313e67ff | use tox instead of functest for doctor CI jobs | -+------------------------------------------+-----------------------------------------------------------------------+ -| 5b22f1b95feacaec0380f6a7543cbf510b628451 | pass value to parameters | -+------------------------------------------+-----------------------------------------------------------------------+ -| 44ab0cea07fa2a734c4f6b80776ad48fd006d1b8 | Doctor job bugfix: fix the scenario | -+------------------------------------------+-----------------------------------------------------------------------+ -| 17617f1c0a78c7bdad0d11d329a6c7e119cbbddd | bugfix: run doctor tests parallelly | -+------------------------------------------+-----------------------------------------------------------------------+ -| 811e4ef7f4c37b7bc246afc34ff880c014ecc05d | delete 'opnfv-build-ubuntu-defaults' parameters for doctor verify job | -+------------------------------------------+-----------------------------------------------------------------------+ -| 0705f31ab5bc54c073df120cbe0fe62cf10f9a81 | delete the 'node' parameter in 'doctor-slave-parameter' macro | -+------------------------------------------+-----------------------------------------------------------------------+ -| 304151b15f9d7241db8c5fea067cafe048287d84 | fix the default node label for doctor test | -+------------------------------------------+-----------------------------------------------------------------------+ -| a6963f92f015a33b44b27199886952205499b44c | Fix project name | -+------------------------------------------+-----------------------------------------------------------------------+ -| f122bfed998b3b0e0178106a7538377c609c6512 | add a default value for SSH_KEY | -+------------------------------------------+-----------------------------------------------------------------------+ +- Maintenance use case updated to support latest version of Fenix. +- Maintenance use case now supports ETSI FEAT03 optimization with Fenix. +- Doctor testing is now preferred to be done in DevStack environment + where one can easily select OpenStack release from Rocky to Ussuri to + test Doctor functionality. Latest OPNFV Fuel can also be used for the + OpenStack version it supports. + +Doctor CI + +- Doctor tested with fuel installer. +- Fault management use case is tested with sample inspector. +- Maintenance use case is tested with sample implementation and towards + the latest Fenix version. The includes the new ETSI FEAT03 optimization. Version change ^^^^^^^^^^^^^^ @@ -143,49 +49,34 @@ Version change Module version changes ~~~~~~~~~~~~~~~~~~~~~~ -- OpenStack has changed from Pike-1 to Queens-1 +- OpenStack has changed Train Document version changes ~~~~~~~~~~~~~~~~~~~~~~~~ -These documents have been updated in Gambia release - -- Testing document - docs/development/overview/testing.rst -- Doctor scenario in functest - docs/development/overview/functest_scenario/doctor-scenario-in-functest.rst -- Maintenance design guideline - docs/development/design/maintenance-design-guideline.rst +All documentation is updated to OPNFV unified format according to +documentation guidelines. Small updates in many documents. Reason for version ^^^^^^^^^^^^^^^^^^ -Documentation is updated due to tox usage in testing and adding maintenance -use case related documentation. +N/A Feature additions ~~~~~~~~~~~~~~~~~ -+--------------------+--------------------------------------------------------+ -| **JIRA REFERENCE** | **SLOGAN** | -+--------------------+--------------------------------------------------------+ -| DOCTOR-106 | Maintenance scenario | -+--------------------+--------------------------------------------------------+ -| DOCTOR-125 | Maintenance design document according to our test case | -+--------------------+--------------------------------------------------------+ -| DOCTOR-126 | Use Tox instead of Functest for doctor CI jobs | -+--------------------+--------------------------------------------------------+ -| DOCTOR-127 | Maintenance test POD | -+--------------------+--------------------------------------------------------+ -| DOCTOR-130 | Apex with containers | -+--------------------+--------------------------------------------------------+ - ++--------------------+--------------------------------------------+ +| **JIRA REFERENCE** | **SLOGAN** | ++--------------------+--------------------------------------------+ +| DOCTOR-137 | VNFM maintenance with ETSI changes | ++--------------------+--------------------------------------------+ +| DOCTOR-136 | DevStack support | ++--------------------+--------------------------------------------+ Deliverables ------------ - Software deliverables ===================== @@ -226,74 +117,21 @@ Doctor CI results with TEST_CASE='fault_management' and INSPECTOR_TYPE=sample +--------------------------------------+--------------+ | **TEST-SUITE** | **Results:** | +--------------------------------------+--------------+ -| INSTALLER_TYPE='Apex' | SUCCESS | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Compass' | N/A | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Daisy' | SUCCESS | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Fuel' | No POD | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Joid' | N/A | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Local' | N/A | +| INSTALLER_TYPE='fuel' | SUCCESS | +--------------------------------------+--------------+ -Doctor CI results with TEST_CASE='fault_management' and INSPECTOR_TYPE=congress -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Doctor CI results with TEST_CASE='maintenance' and INSPECTOR_TYPE=sample +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +--------------------------------------+--------------+ | **TEST-SUITE** | **Results:** | +--------------------------------------+--------------+ -| INSTALLER_TYPE='Apex' | FAILED | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Compass' | N/A | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Daisy' | N/A | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Fuel' | No POD | +| INSTALLER_TYPE='fuel' | SUCCESS | +| ADMIN_TOOL_TYPE='fenix' *) | | +--------------------------------------+--------------+ -| INSTALLER_TYPE='Joid' | N/A | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Local' | N/A | -+--------------------------------------+--------------+ - -Doctor Functest results with TEST_CASE='fault_management' -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------------------------------+--------------+ -| **TEST-SUITE** | **Results:** | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Apex' | skipped | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Compass' | N/A | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Daisy' | skipped | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Fuel' | skipped | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Joid' | N/A | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Local' | N/A | -+--------------------------------------+--------------+ - -Note: Installer Functest does not currently test features or skips running the -project test cases - -Doctor CI results with TEST_CASE='maintenance' -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------------------------------+--------------+ -| **TEST-SUITE** | **Results:** | -+--------------------------------------+--------------+ -| INSTALLER_TYPE='Apex' | SUCCESS | -+--------------------------------------+--------------+ - -Doctor Functest results with TEST_CASE='maintenance' -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -N/A - Needs special target and currently there is only sample implementation +*) Sample implementation not updated according to latest upstream Fenix + and is currently not being tested. References ========== @@ -301,3 +139,8 @@ References For more information about the OPNFV Doctor latest work, please see: https://wiki.opnfv.org/display/doctor/Doctor+Home + +Further information about ETSI FEAT03 optimization can be found from Fenix +Documentation: + +https://fenix.readthedocs.io/en/latest diff --git a/docs/release/release-notes/releasenotes_gambia.rst b/docs/release/release-notes/releasenotes_gambia.rst new file mode 100644 index 00000000..142bfacf --- /dev/null +++ b/docs/release/release-notes/releasenotes_gambia.rst @@ -0,0 +1,303 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +This document provides the release notes for Gambia of Doctor. + +Important notes +=============== + +In Gambia release, Doctor has been working with our second use case over +maintenance. Design guideline is now done and test case exists with sample +maintenance workflow code implemented in Doctor. Work has also started to have +the real implementation done in the OpenStack Fenix project +https://wiki.openstack.org/wiki/Fenix. + +Doctor CI testing has now moved to use tox on jumphots instead of running test +through features container. Also in Apex we use OpenStack services running in +containers. Functest daily testing supports Doctor fault management test case +for Apex, Daisy and Fuel installers. This testing is done through features +container. + +In this release, Doctor has not been working with the fault management use case as +the basic framework has been already done. However, we might need to get back to +it later to better meet the tough industry requirements as well as requirements +from edge, containers and 5G. + + +Summary +======= + +Gambia Doctor framework uses OpenStack Queens integrated into its test cases. +Compared to the previous release, the Heat project is also being used in the +maintenance test case. + +Release Data +============ + +Doctor changes + ++------------------------------------------+----------------------------------------------------------+ +| **commit-ID** | **Subject** | ++------------------------------------------+----------------------------------------------------------+ +| 5b3f5937e7b861fca46b2a6b2d6708866b800f95 | fix building docs | ++------------------------------------------+----------------------------------------------------------+ +| 2ca5924081ce4784f599437707bd32807aa155ce | Fix SSH client connection reset | ++------------------------------------------+----------------------------------------------------------+ +| baac6579556f8216b36db0d0f87f9c2d4f8b4ef5 | Support Apex with services in containers | ++------------------------------------------+----------------------------------------------------------+ +| 23bf63c4616040cb0d69cd26238af2a4a7c00a90 | fix the username to login undercloud in Apex | ++------------------------------------------+----------------------------------------------------------+ +| 61eb3927ada784cc3dffb5ddd17f66e47871f708 | Local Documentation Builds | ++------------------------------------------+----------------------------------------------------------+ +| 0f1dd4314b9e0247d9af7af6df2410462423aeca | Updated from global requirements | ++------------------------------------------+----------------------------------------------------------+ +| 2d4a9f0c0a93797da6534583f6e74553a4b634be | Fix links to remove references to submodules | ++------------------------------------------+----------------------------------------------------------+ +| 3ddc2392b0ed364eede49ff006d64df3ea456350 | Gambia release notes | ++------------------------------------------+----------------------------------------------------------+ +| 825a0a0dd5e8028129b782ed21c549586257b1c5 | delete doctor datasource in congress when cleanup | ++------------------------------------------+----------------------------------------------------------+ +| fcf53129ab2b18b84571faff13d7cb118b3a41b3 | run profile even the notification time is larger than 1S | ++------------------------------------------+----------------------------------------------------------+ +| 495965d0336d42fc36494c81fd15cee2f34c96e9 | Update and add test case | ++------------------------------------------+----------------------------------------------------------+ +| da25598a6a31abe0579ffed12d1719e5ff75f9a7 | bugfix: add doctor datasource in congress | ++------------------------------------------+----------------------------------------------------------+ +| f9e1e3b1ae4be80bc2dc61d9c4213c81c091ea72 | Update the maintenance design document | ++------------------------------------------+----------------------------------------------------------+ +| 4639f15e6db2f1480b41f6fbfd11d70312d4e421 | Add maintenance test code | ++------------------------------------------+----------------------------------------------------------+ +| b54cbc5dd2d32fcb27238680b4657ed384d021c5 | Add setup and cleanup for maintenance test | ++------------------------------------------+----------------------------------------------------------+ +| b2bb504032ac81a2ed3f404113b097d9ce3d7f14 | bugfix: kill the stunnel when cleanup | ++------------------------------------------+----------------------------------------------------------+ +| eaeb3c0f9dc9e6645a159d0a78b9fc181fce53d4 | add ssh_keyfile for connect to installer in Apex | ++------------------------------------------+----------------------------------------------------------+ +| dcbe7bf1c26052b0e95d209254e7273aa1eaace1 | Add tox and test case to testing document | ++------------------------------------------+----------------------------------------------------------+ +| 0f607cb5efd91ee497346b7f792dfa844d15595c | enlarge the time of link down | ++------------------------------------------+----------------------------------------------------------+ +| 1351038a65739b8d799820de515178326ad05f7b | bugfix: fix the filename of ssh tunnel | ++------------------------------------------+----------------------------------------------------------+ +| e70bf248daac03eee6b449cd1654d2ee6265dd8c | Use py34 instead of py35 | ++------------------------------------------+----------------------------------------------------------+ +| 2a60d460eaf018951456451077b7118b60219b32 | add INSPECTOR_TYPE and TEST_CASE to tox env | ++------------------------------------------+----------------------------------------------------------+ +| 2043ceeb08c1eca849daeb2b3696d385425ba061 | [consumer] fix default value for port number | ++------------------------------------------+----------------------------------------------------------+ + +Releng changes + ++------------------------------------------+-----------------------------------------------------------------------+ +| **commit-ID** | **Subject** | ++------------------------------------------+-----------------------------------------------------------------------+ +| c87309f5a75ccc5d595f708817b97793c24c4387 | Add Doctor maintenance job | ++------------------------------------------+-----------------------------------------------------------------------+ +| bd16a9756ffd0743e143f0f2f966da8dd666c7a3 | remove congress test in Daisy | ++------------------------------------------+-----------------------------------------------------------------------+ +| c47aaaa53c91aae93877f2532c72374beaa4eabe | remove fuel job in Doctor | ++------------------------------------------+-----------------------------------------------------------------------+ +| ab2fed2522eaf82ea7c63dd05008a37c56e825d0 | use 'workspace-cleanup' plugin in publisher | ++------------------------------------------+-----------------------------------------------------------------------+ +| 3aaed5cf40092744f1b87680b9205a2901baecf3 | clean the workspace in the publisher | ++------------------------------------------+-----------------------------------------------------------------------+ +| 50151eb3717edd4ddd996f3705fbe1732de7f3b7 | run tox with 'sudo' | ++------------------------------------------+-----------------------------------------------------------------------+ +| a3adc85ecb52f5d19ec4e9c49ca1ac35aa429ff9 | remove inspector variable form job template | ++------------------------------------------+-----------------------------------------------------------------------+ +| adfbaf2a3e8487e4c9152bf864a653a0425b8582 | run doctor tests with different inspectors in sequence | ++------------------------------------------+-----------------------------------------------------------------------+ +| 2e98e56224cd550cb3bf9798e420eece28139bd9 | add the ssh_key info if the key_file is exist | ++------------------------------------------+-----------------------------------------------------------------------+ +| c109c271018e9a85d94be1b9b468338d64589684 | prepare installer info for doctor test | ++------------------------------------------+-----------------------------------------------------------------------+ +| 57cbefc7160958eae1d49e4753779180a25864af | use py34 for tox | ++------------------------------------------+-----------------------------------------------------------------------+ +| 3547754e808a581b09c9d22e013a7d986d9f6cd1 | specify the cacert file when it exits | ++------------------------------------------+-----------------------------------------------------------------------+ +| ef4f36aa1c2ff0819d73cde44f84b99a42e15c7e | bugfix: wrong usage of '!include-raw' | ++------------------------------------------+-----------------------------------------------------------------------+ +| 0e0e0d4cb71fb27b1789a2bef2d3c4ff313e67ff | use tox instead of functest for doctor CI jobs | ++------------------------------------------+-----------------------------------------------------------------------+ +| 5b22f1b95feacaec0380f6a7543cbf510b628451 | pass value to parameters | ++------------------------------------------+-----------------------------------------------------------------------+ +| 44ab0cea07fa2a734c4f6b80776ad48fd006d1b8 | Doctor job bugfix: fix the scenario | ++------------------------------------------+-----------------------------------------------------------------------+ +| 17617f1c0a78c7bdad0d11d329a6c7e119cbbddd | bugfix: run doctor tests parallelly | ++------------------------------------------+-----------------------------------------------------------------------+ +| 811e4ef7f4c37b7bc246afc34ff880c014ecc05d | delete 'opnfv-build-ubuntu-defaults' parameters for doctor verify job | ++------------------------------------------+-----------------------------------------------------------------------+ +| 0705f31ab5bc54c073df120cbe0fe62cf10f9a81 | delete the 'node' parameter in 'doctor-slave-parameter' macro | ++------------------------------------------+-----------------------------------------------------------------------+ +| 304151b15f9d7241db8c5fea067cafe048287d84 | fix the default node label for doctor test | ++------------------------------------------+-----------------------------------------------------------------------+ +| a6963f92f015a33b44b27199886952205499b44c | Fix project name | ++------------------------------------------+-----------------------------------------------------------------------+ +| f122bfed998b3b0e0178106a7538377c609c6512 | add a default value for SSH_KEY | ++------------------------------------------+-----------------------------------------------------------------------+ + +Version change +^^^^^^^^^^^^^^ + +Module version changes +~~~~~~~~~~~~~~~~~~~~~~ + +- OpenStack has changed from Pike-1 to Queens-1 + +Document version changes +~~~~~~~~~~~~~~~~~~~~~~~~ + +These documents have been updated in Gambia release + +- Testing document + docs/development/overview/testing.rst +- Doctor scenario in functest + docs/development/overview/functest_scenario/doctor-scenario-in-functest.rst +- Maintenance design guideline + docs/development/design/maintenance-design-guideline.rst + +Reason for version +^^^^^^^^^^^^^^^^^^ + +Documentation is updated due to tox usage in testing and adding maintenance +use case related documentation. + +Feature additions +~~~~~~~~~~~~~~~~~ + ++--------------------+--------------------------------------------------------+ +| **JIRA REFERENCE** | **SLOGAN** | ++--------------------+--------------------------------------------------------+ +| DOCTOR-106 | Maintenance scenario | ++--------------------+--------------------------------------------------------+ +| DOCTOR-125 | Maintenance design document according to our test case | ++--------------------+--------------------------------------------------------+ +| DOCTOR-126 | Use Tox instead of Functest for doctor CI jobs | ++--------------------+--------------------------------------------------------+ +| DOCTOR-127 | Maintenance test POD | ++--------------------+--------------------------------------------------------+ +| DOCTOR-130 | Apex with containers | ++--------------------+--------------------------------------------------------+ + + + +Deliverables +------------ + + +Software deliverables +===================== + +None + +Documentation deliverables +========================== + +https://git.opnfv.org/doctor/tree/docs + +Known Limitations, Issues and Workarounds +========================================= + +System Limitations +^^^^^^^^^^^^^^^^^^ + +Maintenance test case requirements: + +- Minimum number of nodes: 1 Controller, 3 Computes +- Min number of VCPUs: 2 VCPUs for each compute + +Known issues +^^^^^^^^^^^^ + +None + +Workarounds +^^^^^^^^^^^ + +None + +Test Result +=========== + +Doctor CI results with TEST_CASE='fault_management' and INSPECTOR_TYPE=sample +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ++--------------------------------------+--------------+ +| **TEST-SUITE** | **Results:** | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Apex' | SUCCESS | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Compass' | N/A | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Daisy' | SUCCESS | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Fuel' | No POD | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Joid' | N/A | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Local' | N/A | ++--------------------------------------+--------------+ + +Doctor CI results with TEST_CASE='fault_management' and INSPECTOR_TYPE=congress +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ++--------------------------------------+--------------+ +| **TEST-SUITE** | **Results:** | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Apex' | FAILED | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Compass' | N/A | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Daisy' | N/A | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Fuel' | No POD | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Joid' | N/A | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Local' | N/A | ++--------------------------------------+--------------+ + + +Doctor Functest results with TEST_CASE='fault_management' +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ++--------------------------------------+--------------+ +| **TEST-SUITE** | **Results:** | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Apex' | skipped | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Compass' | N/A | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Daisy' | skipped | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Fuel' | skipped | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Joid' | N/A | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Local' | N/A | ++--------------------------------------+--------------+ + +Note: Installer Functest does not currently test features or skips running the +project test cases + +Doctor CI results with TEST_CASE='maintenance' +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ++--------------------------------------+--------------+ +| **TEST-SUITE** | **Results:** | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='Apex' | SUCCESS | ++--------------------------------------+--------------+ + +Doctor Functest results with TEST_CASE='maintenance' +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +N/A - Needs special target and currently there is only sample implementation + +References +========== + +For more information about the OPNFV Doctor latest work, please see: + +https://wiki.opnfv.org/display/doctor/Doctor+Home diff --git a/docs/release/release-notes/releasenotes_iruya.rst b/docs/release/release-notes/releasenotes_iruya.rst new file mode 100644 index 00000000..92775557 --- /dev/null +++ b/docs/release/release-notes/releasenotes_iruya.rst @@ -0,0 +1,129 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +This document provides the release notes for Iruya version of Doctor. + +Important notes +=============== + +In Iruya release there has not been many changes. + +All testing is now being made with Fuel installer. Maintenance use case +is now only tested against latest upstream Fenix. Only sample inspector is +tested as Fuel do not support Vitrage or Congress. + +Summary +======= + +Iruya Doctor framework uses OpenStack Stein integrated into its test cases. + +Release Data +============ + +Doctor changes + +- Maintenance use case updated to support latest version of Fenix running + in container on controller node +- Maintenance use case now support Fuel installer +- Doctor updated to use OpenStack Stein and only python 3.6 +- Testing only sample inspector as lacking installer support for + Vitrage and Congress + +Releng changes + +- Doctor testing running with python 3.6 and with sample inspector +- Doctor is only tested with Fuel installer + +Version change +^^^^^^^^^^^^^^ + +Module version changes +~~~~~~~~~~~~~~~~~~~~~~ + +- OpenStack has changed from Rocky to Stein since previous Hunter release. + +Document version changes +~~~~~~~~~~~~~~~~~~~~~~~~ + +N/A + +Reason for version +^^^^^^^^^^^^^^^^^^ + +N/A + +Feature additions +~~~~~~~~~~~~~~~~~ + ++--------------------+--------------------------------------------------------------+ +| **JIRA REFERENCE** | **SLOGAN** | ++--------------------+--------------------------------------------------------------+ +| DOCTOR-134 | Update Doctor maintenance use case to work with latest Fenix | ++--------------------+--------------------------------------------------------------+ + +Deliverables +------------ + +Software deliverables +===================== + +None + +Documentation deliverables +========================== + +https://git.opnfv.org/doctor/tree/docs + +Known Limitations, Issues and Workarounds +========================================= + +System Limitations +^^^^^^^^^^^^^^^^^^ + +Maintenance test case requirements: + +- Minimum number of nodes: 1 Controller, 3 Computes +- Min number of VCPUs: 2 VCPUs for each compute + +Known issues +^^^^^^^^^^^^ + +None + +Workarounds +^^^^^^^^^^^ + +None + +Test Result +=========== + +Doctor CI results with TEST_CASE='fault_management' and INSPECTOR_TYPE=sample +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ++--------------------------------------+--------------+ +| **TEST-SUITE** | **Results:** | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='fuel' | SUCCESS | ++--------------------------------------+--------------+ + +Doctor CI results with TEST_CASE='maintenance' and INSPECTOR_TYPE=sample +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ++--------------------------------------+--------------+ +| **TEST-SUITE** | **Results:** | ++--------------------------------------+--------------+ +| INSTALLER_TYPE='fuel' | SUCCESS | +| ADMIN_TOOL_TYPE='fenix' *) | | ++--------------------------------------+--------------+ + +*) Sample implementation not updated according to latest upstream Fenix + and is currently not being tested. + +References +========== + +For more information about the OPNFV Doctor latest work, please see: + +https://wiki.opnfv.org/display/doctor/Doctor+Home diff --git a/docs/release/scenarios/fault_management/fault_management.rst b/docs/release/scenarios/fault_management/fault_management.rst new file mode 100644 index 00000000..99371201 --- /dev/null +++ b/docs/release/scenarios/fault_management/fault_management.rst @@ -0,0 +1,90 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +Running test cases +"""""""""""""""""" + +Functest will call the "doctor_tests/main.py" in Doctor to run the test job. +Doctor testing can also be triggered by tox on OPNFV installer jumphost. Tox +is normally used for functional, module and coding style testing in Python +project. + +Currently 'MCP' and 'devstack' installer are supported. + + +Fault management use case +""""""""""""""""""""""""" + +* A consumer of the NFVI wants to receive immediate notifications about faults + in the NFVI affecting the proper functioning of the virtual resources. + Therefore, such faults have to be detected as quickly as possible, and, when + a critical error is observed, the affected consumer is immediately informed + about the fault and can switch over to the STBY configuration. + +The faults to be monitored (and at which detection rate) will be configured by +the consumer. Once a fault is detected, the Inspector in the Doctor +architecture will check the resource map maintained by the Controller, to find +out which virtual resources are affected and then update the resources state. +The Notifier will receive the failure event requests sent from the Controller, +and notify the consumer(s) of the affected resources according to the alarm +configuration. + +Detailed workflow information is as follows: + +* Consumer(VNFM): (step 0) creates resources (network, server/instance) and an + event alarm on state down notification of that server/instance or Neutron + port. + +* Monitor: (step 1) periodically checks nodes, such as ping from/to each + dplane nic to/from gw of node, (step 2) once it fails to send out event + with "raw" fault event information to Inspector + +* Inspector: when it receives an event, it will (step 3) mark the host down + ("mark-host-down"), (step 4) map the PM to VM, and change the VM status to + down. In network failure case, also Neutron port is changed to down. + +* Controller: (step 5) sends out instance update event to Ceilometer. In network + failure case, also Neutron port is changed to down and corresponding event is + sent to Ceilometer. + +* Notifier: (step 6) Ceilometer transforms and passes the events to AODH, + (step 7) AODH will evaluate events with the registered alarm definitions, + then (step 8) it will fire the alarm to the "consumer" who owns the + instance + +* Consumer(VNFM): (step 9) receives the event and (step 10) recreates a new + instance + +Fault management test case +"""""""""""""""""""""""""" + +Functest will call the 'doctor-test' command in Doctor to run the test job. + +The following steps are executed: + +Firstly, get the installer ip according to the installer type. Then ssh to +the installer node to get the private key for accessing to the cloud. As +'fuel' installer, ssh to the controller node to modify nova and ceilometer +configurations. + +Secondly, prepare image for booting VM, then create a test project and test +user (both default to doctor) for the Doctor tests. + +Thirdly, boot a VM under the doctor project and check the VM status to verify +that the VM is launched completely. Then get the compute host info where the VM +is launched to verify connectivity to the target compute host. Get the consumer +ip according to the route to compute ip and create an alarm event in Ceilometer +using the consumer ip. + +Fourthly, the Doctor components are started, and, based on the above preparation, +a failure is injected to the system, i.e. the network of compute host is +disabled for 3 minutes. To ensure the host is down, the status of the host +will be checked. + +Finally, the notification time, i.e. the time between the execution of step 2 +(Monitor detects failure) and step 9 (Consumer receives failure notification) +is calculated. + +According to the Doctor requirements, the Doctor test is successful if the +notification time is below 1 second. diff --git a/docs/development/overview/functest_scenario/images/Fault-management-design.png b/docs/release/scenarios/maintenance/images/Fault-management-design.png Binary files differindex 6d98cdec..6d98cdec 100644 --- a/docs/development/overview/functest_scenario/images/Fault-management-design.png +++ b/docs/release/scenarios/maintenance/images/Fault-management-design.png diff --git a/docs/development/overview/functest_scenario/images/LICENSE b/docs/release/scenarios/maintenance/images/LICENSE index 21a2d03d..21a2d03d 100644 --- a/docs/development/overview/functest_scenario/images/LICENSE +++ b/docs/release/scenarios/maintenance/images/LICENSE diff --git a/docs/development/overview/functest_scenario/images/Maintenance-design.png b/docs/release/scenarios/maintenance/images/Maintenance-design.png Binary files differindex 8f21db6a..8f21db6a 100644 --- a/docs/development/overview/functest_scenario/images/Maintenance-design.png +++ b/docs/release/scenarios/maintenance/images/Maintenance-design.png diff --git a/docs/development/overview/functest_scenario/images/Maintenance-workflow.png b/docs/release/scenarios/maintenance/images/Maintenance-workflow.png Binary files differindex 9b65fd59..9b65fd59 100644 --- a/docs/development/overview/functest_scenario/images/Maintenance-workflow.png +++ b/docs/release/scenarios/maintenance/images/Maintenance-workflow.png diff --git a/docs/release/scenarios/maintenance/maintenance.rst b/docs/release/scenarios/maintenance/maintenance.rst new file mode 100644 index 00000000..ecfe76b1 --- /dev/null +++ b/docs/release/scenarios/maintenance/maintenance.rst @@ -0,0 +1,120 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +Maintenance use case +"""""""""""""""""""" + +* A consumer of the NFVI wants to interact with NFVI maintenance, upgrade, + scaling and to have graceful retirement. Receiving notifications over these + NFVI events and responding to those within given time window, consumer can + guarantee zero downtime to his service. + +The maintenance use case adds the Doctor platform an `admin tool` and an +`app manager` component. Overview of maintenance components can be seen in +:numref:`figure-p2`. + +.. figure:: ./images/Maintenance-design.png + :name: figure-p2 + :width: 100% + + Doctor platform components in maintenance use case + +In maintenance use case, `app manager` (VNFM) will subscribe to maintenance +notifications triggered by project specific alarms through AODH. This is the way +it gets to know different NFVI maintenance, upgrade and scaling operations that +effect to its instances. The `app manager` can do actions depicted in `green +color` or tell `admin tool` to do admin actions depicted in `orange color` + +Any infrastructure component like `Inspector` can subscribe to maintenance +notifications triggered by host specific alarms through AODH. Subscribing to the +notifications needs admin privileges and can tell when a host is out of use as +in maintenance and when it is taken back to production. + +Maintenance test case +""""""""""""""""""""" + +Maintenance test case is currently running in our Apex CI and executed by tox. +This is because the special limitation mentioned below and also the fact we +currently have only sample implementation as a proof of concept and we also +support unofficial OpenStack project Fenix. Environment variable +TEST_CASE='maintenance' needs to be used when executing "doctor_tests/main.py" +and ADMIN_TOOL_TYPE='fenix' if want to test with Fenix instead of sample +implementation. Test case workflow can be seen in :numref:`figure-p3`. + +.. figure:: ./images/Maintenance-workflow.png + :name: figure-p3 + :width: 100% + + Maintenance test case workflow + +In test case all compute capacity will be consumed with project (VNF) instances. +For redundant services on instances and an empty compute needed for maintenance, +test case will need at least 3 compute nodes in system. There will be 2 +instances on each compute, so minimum number of VCPUs is also 2. Depending on +how many compute nodes there is application will always have 2 redundant +instances (ACT-STDBY) on different compute nodes and rest of the compute +capacity will be filled with non-redundant instances. + +For each project specific maintenance message there is a time window for +`app manager` to make any needed action. This will guarantee zero +down time for his service. All replies back are done by calling `admin tool` API +given in the message. + +The following steps are executed: + +Infrastructure admin will call `admin tool` API to trigger maintenance for +compute hosts having instances belonging to a VNF. + +Project specific `MAINTENANCE` notification is triggered to tell `app manager` +that his instances are going to hit by infrastructure maintenance at a specific +point in time. `app manager` will call `admin tool` API to answer back +`ACK_MAINTENANCE`. + +When the time comes to start the actual maintenance workflow in `admin tool`, +a `DOWN_SCALE` notification is triggered as there is no empty compute node for +maintenance (or compute upgrade). Project receives corresponding alarm and scales +down instances and call `admin tool` API to answer back `ACK_DOWN_SCALE`. + +As it might happen instances are not scaled down (removed) from a single +compute node, `admin tool` might need to figure out what compute node should be +made empty first and send `PREPARE_MAINTENANCE` to project telling which instance +needs to be migrated to have the needed empty compute. `app manager` makes sure +he is ready to migrate instance and call `admin tool` API to answer back +`ACK_PREPARE_MAINTENANCE`. `admin tool` will make the migration and answer +`ADMIN_ACTION_DONE`, so `app manager` knows instance can be again used. + +:numref:`figure-p3` has next a light blue section of actions to be done for each +compute. However as we now have one empty compute, we will maintain/upgrade that +first. So on first round, we can straight put compute in maintenance and send +admin level host specific `IN_MAINTENANCE` message. This is caught by `Inspector` +to know host is down for maintenance. `Inspector` can now disable any automatic +fault management actions for the host as it can be down for a purpose. After +`admin tool` has completed maintenance/upgrade `MAINTENANCE_COMPLETE` message +is sent to tell host is back in production. + +Next rounds we always have instances on compute, so we need to have +`PLANNED_MAINTANANCE` message to tell that those instances are now going to hit +by maintenance. When `app manager` now receives this message, he knows instances +to be moved away from compute will now move to already maintained/upgraded host. +In test case no upgrade is done on application side to upgrade instances +according to new infrastructure capabilities, but this could be done here as +this information is also passed in the message. This might be just upgrading +some RPMs, but also totally re-instantiating instance with a new flavor. Now if +application runs an active side of a redundant instance on this compute, +a switch over will be done. After `app manager` is ready he will call +`admin tool` API to answer back `ACK_PLANNED_MAINTENANCE`. In test case the +answer is `migrate`, so `admin tool` will migrate instances and reply +`ADMIN_ACTION_DONE` and then `app manager` knows instances can be again used. +Then we are ready to make the actual maintenance as previously trough +`IN_MAINTENANCE` and `MAINTENANCE_COMPLETE` steps. + +After all computes are maintained, `admin tool` can send `MAINTENANCE_COMPLETE` +to tell maintenance/upgrade is now complete. For `app manager` this means he +can scale back to full capacity. + +There is currently sample implementation on VNFM and test case. In +infrastructure side there is sample implementation of 'admin_tool' and +there is also support for the OpenStack Fenix that extends the use case to +support 'ETSI FEAT03' for VNFM interaction and to optimize the whole +infrastructure mainteannce and upgrade. diff --git a/docs/development/manuals/get-valid-server-state.rst b/docs/release/userguide/get-valid-server-state.rst index 824ea3c2..824ea3c2 100644 --- a/docs/development/manuals/get-valid-server-state.rst +++ b/docs/release/userguide/get-valid-server-state.rst diff --git a/docs/release/userguide/index.rst b/docs/release/userguide/index.rst index eee855dc..577072c7 100644 --- a/docs/release/userguide/index.rst +++ b/docs/release/userguide/index.rst @@ -11,3 +11,6 @@ Doctor User Guide :maxdepth: 2 feature.userguide.rst + get-valid-server-state.rst + mark-host-down_manual.rst + monitors.rst diff --git a/docs/development/manuals/mark-host-down_manual.rst b/docs/release/userguide/mark-host-down_manual.rst index 3815205d..3815205d 100644 --- a/docs/development/manuals/mark-host-down_manual.rst +++ b/docs/release/userguide/mark-host-down_manual.rst diff --git a/docs/development/manuals/monitors.rst b/docs/release/userguide/monitors.rst index eeb5e226..eeb5e226 100644 --- a/docs/development/manuals/monitors.rst +++ b/docs/release/userguide/monitors.rst diff --git a/docs/testing/developer/index.rst b/docs/testing/developer/index.rst new file mode 100644 index 00000000..dfbcfa74 --- /dev/null +++ b/docs/testing/developer/index.rst @@ -0,0 +1,13 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. SPDX-License-Identifier: CC-BY-4.0 +.. (c) Open Platform for NFV Project, Inc. and its contributors + +********* +Developer +********* + +.. toctree:: + :numbered: + :maxdepth: 2 + + testing.rst diff --git a/docs/development/overview/testing.rst b/docs/testing/developer/testing.rst index a4b9ffa6..6a929130 100644 --- a/docs/development/overview/testing.rst +++ b/docs/testing/developer/testing.rst @@ -41,6 +41,16 @@ export TEST_CASE with different values: #Run both tests cases export TEST_CASE='all' + #Use Fenix in maintenance testing instead of sample admin_tool + #This is only for 'mainteanance' test case + export ADMIN_TOOL_TYPE='fenix' + export APP_MANAGER_TYPE='vnfm' + + #Run in different installer jumphost 'fuel' or 'apex' + #In multinode DevStack you run Doctor in controller node + #with value export APP_MANAGER_TYPE=vnfm + export INSTALLER_TYPE='fuel' + Run Python Test Script ~~~~~~~~~~~~~~~~~~~~~~ @@ -57,42 +67,16 @@ environment and then run the test. .. _doctor.sample.conf: https://git.opnfv.org/doctor/tree/etc/doctor.sample.conf -In OPNFV Apex jumphost you can run Doctor testing as follows using tox: +In OPNFV testing environment jumphost you can run Doctor testing as follows +using tox: .. code-block:: bash - #Before Gambia: overcloudrc.v3 source overcloudrc export INSTALLER_IP=${INSTALLER_IP} export INSTALLER_TYPE=${INSTALLER_TYPE} git clone https://gerrit.opnfv.org/gerrit/doctor cd doctor sudo -E tox - -Run Functest Suite -================== - -Functest supports Doctor testing by triggering the test script above in a -Functest container. You can run the Doctor test with the following steps: - -.. code-block:: bash - - DOCKER_TAG=latest - docker pull docker.io/opnfv/functest-features:${DOCKER_TAG} - docker run --privileged=true -id \ - -e INSTALLER_TYPE=${INSTALLER_TYPE} \ - -e INSTALLER_IP=${INSTALLER_IP} \ - -e INSPECTOR_TYPE=sample \ - docker.io/opnfv/functest-features:${DOCKER_TAG} /bin/bash - docker exec <container_id> functest testcase run doctor-notification - -See `Functest Userguide`_ for more information. - -.. _Functest Userguide: :doc:`<functest:testing/user/userguide>` - - -For testing with stable version, change DOCKER_TAG to 'stable' or other release -tag identifier. - -Tips -==== + +Note! In DevStack you run Doctor in controller node. diff --git a/docs/testing/index.rst b/docs/testing/index.rst new file mode 100644 index 00000000..3fae9568 --- /dev/null +++ b/docs/testing/index.rst @@ -0,0 +1,15 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. SPDX-License-Identifier: CC-BY-4.0 +.. (c) Open Platform for NFV Project, Inc. and its contributors + +.. _testing: + +======= +Testing +======= + +.. toctree:: + :maxdepth: 2 + + ./developer/index.rst + ./user/index.rst diff --git a/docs/testing/user/index.rst b/docs/testing/user/index.rst new file mode 100644 index 00000000..1be9c7eb --- /dev/null +++ b/docs/testing/user/index.rst @@ -0,0 +1,13 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. SPDX-License-Identifier: CC-BY-4.0 +.. (c) Open Platform for NFV Project, Inc. and its contributors + +**** +User +**** + +.. toctree:: + :numbered: + :maxdepth: 2 + + testing.rst diff --git a/docs/testing/user/testing.rst b/docs/testing/user/testing.rst new file mode 100644 index 00000000..6172d26a --- /dev/null +++ b/docs/testing/user/testing.rst @@ -0,0 +1,30 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Run Functest Suite (obsolete) +============================= + +Functest supports Doctor testing by triggering the test script above in a +Functest container. You can run the Doctor test with the following steps: + +.. code-block:: bash + + DOCKER_TAG=latest + docker pull docker.io/opnfv/functest-features:${DOCKER_TAG} + docker run --privileged=true -id \ + -e INSTALLER_TYPE=${INSTALLER_TYPE} \ + -e INSTALLER_IP=${INSTALLER_IP} \ + -e INSPECTOR_TYPE=sample \ + docker.io/opnfv/functest-features:${DOCKER_TAG} /bin/bash + docker exec <container_id> functest testcase run doctor-notification + +See `Functest Userguide`_ for more information. + +.. _Functest Userguide: :doc:`<functest:testing/user/userguide>` + + +For testing with stable version, change DOCKER_TAG to 'stable' or other release +tag identifier. + +Tips +==== |
