diff options
Diffstat (limited to 'docs/release/configguide/postinstall.rst')
| -rw-r--r-- | docs/release/configguide/postinstall.rst | 169 |
1 files changed, 49 insertions, 120 deletions
diff --git a/docs/release/configguide/postinstall.rst b/docs/release/configguide/postinstall.rst index 602a8d5c..dad56b99 100644 --- a/docs/release/configguide/postinstall.rst +++ b/docs/release/configguide/postinstall.rst @@ -1,3 +1,5 @@ +.. _barometer-postinstall: + .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 @@ -6,8 +8,18 @@ Barometer post installation procedures ====================================== This document describes briefly the methods of validating the Barometer installation. +.. TODO: Update this to include reference to containers rather than an Openstack deployment. + Automated post installation activities -------------------------------------- +.. This section will include how to run plugin validation tests, when they are created/merged. +.. This section will also include some troubleshooting and debugging information. + +.. note:: This section is outdated and needs to be updated. + +.. TODO: Update this section; post-installation/verification shouldn't be in + the config guide. It should be in testing. + The Barometer test-suite in Functest is called ``barometercollectd`` and is part of the ``Features`` tier. Running these tests is done automatically by the OPNFV deployment pipeline on the supported scenarios. The testing consists of basic verifications that each plugin is functional per their @@ -17,148 +29,65 @@ default configurations. Inside the Functest container, the detailed results can Barometer post configuration procedures --------------------------------------- The functionality for each plugin (such as enabling/disabling and configuring its capabilities) -is controlled as described in the User Guide through their individual ``.conf`` file located in -the ``/etc/collectd/collectd.conf.d/`` folder on the compute node(s). In order for any changes to -take effect, the collectd service must be stopped and then started again. +is controlled as described in the :ref:`User Guide <barometer-userguide>` through their individual +``.conf`` file located in the ``/etc/collectd/collectd.conf.d/`` on the host(s). In order for any +changes to take effect, the collectd service must be stopped and then started again. -Platform components validation - Apex -------------------------------------- -The following steps describe how to perform a simple "manual" testing of the Barometer components: +Plugin verification +~~~~~~~~~~~~~~~~~~~ +Once collectd has been installed and deployed, you will see metrics from most plugins immediately. However, in some cases, you may want to verify that the configuration is correct and that the plugion is functioning as intended (particularly during development, or when testing an experimental version). The following sections provide some verification steps to make sure the plugins are working as expected. -On the controller: - -You will need update the archive policy rule for gnocchi via the command line. -The default is low, which means that you only get a metric every -5 minutes. To do this: +MCElog +^^^^^^ +On the collectd host, you can induce an event monitored by the plugins; e.g. a corrected memory error: .. code:: bash - $ openstack metric archive-policy rule delete default - $ openstack metric archive-policy-rule create -a bool -m '*' default + $ git clone https://git.kernel.org/pub/scm/utils/cpu/mce/mce-inject.git + $ cd mce-inject + $ make + $ modprobe mce-inject -Using the ``bool`` archive policy reduces the number of aggregation methods run -to one (last), and it keeps all metrics at a 1 second interval. -In order to query this you need to explicitly choose an aggregation method to -display (by default, ``measures show`` uses mean). You may have to update the -command for checking the metrics, this is the CLI command: +Modify the test/corrected script to include the following: .. code:: bash - $ watch –n2 –d openstack metric measures show --aggregation last <metric_id> - -More on testing and displaying metrics is shown below. - -On the compute: - -1. Connect to any compute node and ensure that the collectd service is running. The log file - ``collectd.log`` should contain no errors and should indicate that each plugin was successfully - loaded. For example, from the Jump Host: - - .. code:: bash - - $ opnfv-util overcloud compute0 - $ ls /etc/collectd/collectd.conf.d/ - $ systemctl status collectd - $ vi /opt/stack/collectd.log - - The following plugings should be found loaded: - aodh, gnocchi, hugepages, intel_rdt, mcelog, ovs_events, ovs_stats, snmp, virt - -2. On the compute node, induce an event monitored by the plugins; e.g. a corrected memory error: - - .. code:: bash - - $ git clone https://git.kernel.org/pub/scm/utils/cpu/mce/mce-inject.git - $ cd mce-inject - $ make - $ modprobe mce-inject - - Modify the test/corrected script to include the following: + CPU 0 BANK 0 + STATUS 0xcc00008000010090 + ADDR 0x0010FFFFFFF - .. code:: bash - - CPU 0 BANK 0 - STATUS 0xcc00008000010090 - ADDR 0x0010FFFFFFF - - Inject the error: - - .. code:: bash - - $ ./mce-inject < test/corrected - -3. Connect to the controller and query the monitoring services. Make sure the overcloudrc.v3 - file has been copied to the controller (from the undercloud VM or from the Jump Host) in order - to be able to authenticate for OpenStack services. - - .. code:: bash +Inject the error: - $ opnfv-util overcloud controller0 - $ su - $ source overcloudrc.v3 - $ gnocchi metric list - $ aodh alarm list - - The output for the gnocchi and aodh queries should be similar to the excerpts below: - - .. code:: bash - - +--------------------------------------+---------------------+------------------------------------------------------------------------------------------------------------+-----------+-------------+ - | id | archive_policy/name | name | unit | resource_id | - +--------------------------------------+---------------------+------------------------------------------------------------------------------------------------------------+-----------+-------------+ - [...] - | 0550d7c1-384f-4129-83bc-03321b6ba157 | high | overcloud-novacompute-0.jf.intel.com-hugepages-mm-2048Kb@vmpage_number.free | Pages | None | - | 0cf9f871-0473-4059-9497-1fea96e5d83a | high | overcloud-novacompute-0.jf.intel.com-hugepages-node0-2048Kb@vmpage_number.free | Pages | None | - | 0d56472e-99d2-4a64-8652-81b990cd177a | high | overcloud-novacompute-0.jf.intel.com-hugepages-node1-1048576Kb@vmpage_number.used | Pages | None | - | 0ed71a49-6913-4e57-a475-d30ca2e8c3d2 | high | overcloud-novacompute-0.jf.intel.com-hugepages-mm-1048576Kb@vmpage_number.used | Pages | None | - | 11c7be53-b2c1-4c0e-bad7-3152d82c6503 | high | overcloud-novacompute-0.jf.intel.com-mcelog- | None | None | - | | | SOCKET_0_CHANNEL_any_DIMM_any@errors.uncorrected_memory_errors_in_24h | | | - | 120752d4-385e-4153-aed8-458598a2a0e0 | high | overcloud-novacompute-0.jf.intel.com-cpu-24@cpu.interrupt | jiffies | None | - | 1213161e-472e-4e1b-9e56-5c6ad1647c69 | high | overcloud-novacompute-0.jf.intel.com-cpu-6@cpu.softirq | jiffies | None | - [...] - - +--------------------------------------+-------+------------------------------------------------------------------+-------+----------+---------+ - | alarm_id | type | name | state | severity | enabled | - +--------------------------------------+-------+------------------------------------------------------------------+-------+----------+---------+ - | fbd06539-45dd-42c5-a991-5c5dbf679730 | event | gauge.memory_erros(overcloud-novacompute-0.jf.intel.com-mcelog) | ok | moderate | True | - | d73251a5-1c4e-4f16-bd3d-377dd1e8cdbe | event | gauge.mcelog_status(overcloud-novacompute-0.jf.intel.com-mcelog) | ok | moderate | True | - [...] +.. code:: bash + $ ./mce-inject < test/corrected -Platform components validation - Compass4nfv --------------------------------------------- +.. TODO: How to check that the event was propogated to collectd -The procedure is similar to the above. +.. _barometer-docker-verification: -The following steps describe how to perform a simple "manual" testing of the Barometer components: +Barometer post installation verification on barometer-collectd container +------------------------------------------------------------------------ -On the compute: +The following steps describe how to perform simple "manual" testing of the Barometer components +after :ref:`successfully deploying the barometer-collectd container<barometer-docker-userguide>`: -1. Connect to any compute node and ensure that the collectd service is running. The log file - ``collectd.log`` should contain no errors and should indicate that each plugin was successfully - loaded. For example, ssh into a compute node and test: +1. Connect to any compute node and ensure that the collectd container is running. .. code:: bash - $ ls /etc/collectd/collectd.conf.d/ - $ systemctl status collectd - $ vi /var/log/collectd.log - - The following plugings should be found loaded: - aodh, gnocchi, hugepages, mcelog, ovs_events, ovs_stats, cpu, interface, memory, disk, numa, virt, rrdtool + root@host2:~# docker ps | grep collectd -2. Testing using mce-inject is similar to #2 shown above. + You should see the container ``opnfv/barometer-collectd`` running. -On the controller: +2. Use a web browser to connect to Grafana at ``http://<serverip>:3000/``, using the hostname or + IP of your server and port 3000. Log in with admin/admin. You will see ``collectd`` + InfluxDB database in the ``Data Sources``. Also, you will notice metrics coming in the several + dashboards such as ``CPU Usage`` and ``Host Overview``. -3. Connect to the controller and query the monitoring services. Make sure to log in to the lxc-utility -container before using the OpenStack CLI. Please refer to this wiki for details: -https://wiki.opnfv.org/display/compass4nfv/Containerized+Compass#ContainerizedCompass-HowtouseOpenStackCLI - - .. code:: bash +For more details on the Barometer containers, Grafana and InfluxDB, please refer to +the following documentation links: - $ source ~/openrc - $ gnocchi metric list - $ aodh alarm list +`Barometer Containers wiki page <https://wiki.opnfv.org/display/fastpath/Barometer+Containers#BarometerContainers-barometer-collectdcontainer>`_ - The output for the gnocchi and aodh queries should be similar to the excerpts shown in #3 above. +:ref:`Barometer Docker install guide<barometer-docker-userguide>` |