From aa2982090189856bcf082d71c0d15d80367fa0d1 Mon Sep 17 00:00:00 2001 From: Emma Foley Date: Thu, 5 Aug 2021 16:32:48 +0100 Subject: [docs][userguide] Use literalinclude instead of duplicating the sample configs This should reduce the inaccuracies, since the sample configs are easier to test, and changes don't have to be sychronised between documents. Change-Id: I92a1937ff0308df25ea3a345f2772027033f0302 Signed-off-by: Emma Foley --- docs/release/userguide/feature.userguide.rst | 189 +++++++-------------------- 1 file changed, 48 insertions(+), 141 deletions(-) (limited to 'docs') diff --git a/docs/release/userguide/feature.userguide.rst b/docs/release/userguide/feature.userguide.rst index c76e79a5..2f4639c7 100644 --- a/docs/release/userguide/feature.userguide.rst +++ b/docs/release/userguide/feature.userguide.rst @@ -212,48 +212,18 @@ Example of specifying custom paths to DPDK headers and libraries: This will install collectd to default folder ``/opt/collectd``. The collectd configuration file (``collectd.conf``) can be found at ``/opt/collectd/etc``. To configure the dpdkstats plugin you need to modify the configuration file to -include: - -.. code:: bash - - LoadPlugin dpdkstat - - Coremask "0xf" - ProcessType "secondary" - FilePrefix "rte" - EnabledPortMask 0xffff - PortName "interface1" - PortName "interface2" - +include (uncomment lines as appropriate): +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/dpdkstat.conf + :start-at: LoadPlugin + :language: bash To configure the dpdkevents plugin you need to modify the configuration file to -include: +include (uncomment lines as appropriate): -.. code:: bash - - - Interval 1 - - - - - Coremask "0x1" - MemoryChannels "4" - FilePrefix "rte" - - - SendEventsOnUpdate false - EnabledPortMask 0xffff - SendNotification true - - - SendEventsOnUpdate false - LCoreMask "0xf" - KeepAliveShmName "/dpdk_keepalive_shm_name" - SendNotification true - - +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/dpdkevents.conf + :start-at: LoadPlugin + :language: bash .. note:: Currently, the DPDK library doesn’t support API to de-initialize the DPDK resources allocated on the initialization. It means, the collectd @@ -352,16 +322,9 @@ configuration file (``collectd.conf``) can be found at ``/opt/collectd/etc``. To configure the hugepages plugin you need to modify the configuration file to include: -.. code:: bash - - LoadPlugin hugepages - - ReportPerNodeHP true - ReportRootHP true - ValuesPages true - ValuesBytes false - ValuesPercentage false - +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/hugepages.conf + :start-at: LoadPlugin + :language: bash For more information on the plugin parameters, please see: https://github.com/collectd/collectd/blob/main/src/collectd.conf.pod @@ -421,31 +384,12 @@ configuration file (``collectd.conf``) can be found at ``/opt/collectd/etc``. To configure the PMU plugin you need to modify the configuration file to include: -.. code:: bash - - - Interval 1 - - - ReportHardwareCacheEvents true - ReportKernelPMUEvents true - ReportSoftwareEvents true - Cores "" - - -If you want to monitor Intel CPU specific CPU events, make sure to enable the -additional two options shown below: +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/intel_pmu.conf + :start-at: LoadPlugin + :language: bash -.. code:: bash - - - ReportHardwareCacheEvents true - ReportKernelPMUEvents true - ReportSoftwareEvents true - EventList "$HOME/.cache/pmu-events/GenuineIntel-6-2D-core.json" - HardwareEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS" "L2_RQSTS.ALL_CODE_RD" - Cores "" - +If you want to monitor Intel CPU specific CPU events, make sure to uncomment the +``EventList`` and ``HardwareEvents`` options above. .. note:: If you set XDG_CACHE_HOME to anything other than the variable above - you will need to modify @@ -513,14 +457,9 @@ configuration file (``collectd.conf``) can be found at ``/opt/collectd/etc``. To configure the RDT plugin you need to modify the configuration file to include: -.. code:: bash - - - Interval 1 - - - Cores "" - +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/rdt.conf + :start-at: LoadPlugin + :language: bash For more information on the plugin parameters, please see: https://github.com/collectd/collectd/blob/main/src/collectd.conf.pod @@ -726,18 +665,9 @@ configuration file (``collectd.conf``) can be found at ``/opt/collectd/etc``. To configure the mcelog plugin you need to modify the configuration file to include: -.. code:: bash - - - Interval 1 - - - - McelogClientSocket "/var/run/mcelog-client" - PersistentNotification false - - #McelogLogfile "/var/log/mcelog" - +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/mcelog.conf + :start-at: LoadPlugin + :language: bash For more information on the plugin parameters, please see: https://github.com/collectd/collectd/blob/main/src/collectd.conf.pod @@ -881,35 +811,19 @@ Clone and install the collectd ovs plugin: This will install collectd to default folder ``/opt/collectd``. The collectd configuration file (``collectd.conf``) can be found at ``/opt/collectd/etc``. -To configure the OVS events plugin you need to modify the configuration file to include: +To configure the OVS events plugin you need to modify the configuration file +(uncommenting and updating values as appropriate) to include: -.. code:: bash - - - Interval 1 - - - Port "6640" - Address "127.0.0.1" - Socket "/var/run/openvswitch/db.sock" - Interfaces "br0" "veth0" - SendNotification true - +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/ovs_events.conf + :start-at: LoadPlugin + :language: bash To configure the OVS stats plugin you need to modify the configuration file -to include: +(uncommenting and updating values as appropriate) to include: -.. code:: bash - - - Interval 1 - - - Port "6640" - Address "127.0.0.1" - Socket "/var/run/openvswitch/db.sock" - Bridges "br0" - +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/ovs_stats.conf + :start-at: LoadPlugin + :language: bash For more information on the plugin parameters, please see: https://github.com/collectd/collectd/blob/main/src/collectd.conf.pod @@ -942,12 +856,11 @@ to include: .. note:: Exec plugin configuration has to be changed to use appropriate user before starting collectd service. -ovs_pmd_stat.sh calls the script for OVS PMD stats application with its argument: - -.. code:: bash +``ovs_pmd_stat.sh`` calls the script for OVS PMD stats application with its argument: - sudo python /usr/local/src/ovs_pmd_stats.py" "--socket-pid-file" - "/var/run/openvswitch/ovs-vswitchd.pid" +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/ovs_pmd_stats.sh + :start-at: python + :language: bash SNMP Agent Plugin ^^^^^^^^^^^^^^^^^ @@ -1275,29 +1188,23 @@ Additional useful plugins **Exec Plugin** : Can be used to show you when notifications are being generated by calling a bash script that dumps notifications to file. (handy -for debug). Modify /opt/collectd/etc/collectd.conf: - -.. code:: bash +for debug). Modify ``/opt/collectd/etc/collectd.conf`` to include the +``NotificationExec`` config option, taking care to add the right directory path +to the ``write_notification.sh`` script: - LoadPlugin exec - - # Exec "user:group" "/path/to/exec" - NotificationExec "user" "/barometer/src/collectd/collectd_sample_configs/write_notification.sh" - +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/exec.conf + :start-at: LoadPlugin + :emphasize-lines: 6 + :language: bash -write_notification.sh (just writes the notification passed from exec through -STDIN to a file (/tmp/notifications)): - -.. code:: bash +``write_notification.sh`` writes the notification passed from exec through +STDIN to a file (``/tmp/notifications``): - #!/bin/bash - rm -f /tmp/notifications - while read x y - do - echo $x$y >> /tmp/notifications - done +.. literalinclude:: ../../../src/collectd/collectd_sample_configs/write_notification.sh + :start-at: rm -f + :language: bash -output to /tmp/notifications should look like: +output to ``/tmp/notifications`` should look like: .. code:: bash -- cgit 1.2.3-korg From 18e27d1c59cf34373b8079e066a296a5922a5b04 Mon Sep 17 00:00:00 2001 From: Emma Foley Date: Fri, 6 Aug 2021 12:18:21 +0100 Subject: [docs] Update config and post-installation guides The configguide was removed previously because it only contained information about using the OPNFV installers, however, now there appears to be no config information about the plugins. The config information for each plugin is included in the user guide in docs/release/userguide/feature.userguide.rst. This change adds in a note to the the config guide on where to look. Signed-off-by: Emma Foley Change-Id: I7f71d51846c9116bed3882b96e762ab5d70fdfa4 --- docs/release/configguide/featureconfig.rst | 8 +++ docs/release/configguide/postinstall.rst | 74 ++++++++++++++++++++++++-- docs/release/userguide/feature.userguide.rst | 2 + docs/release/userguide/installguide.docker.rst | 8 +-- 4 files changed, 86 insertions(+), 6 deletions(-) create mode 100644 docs/release/configguide/featureconfig.rst (limited to 'docs') diff --git a/docs/release/configguide/featureconfig.rst b/docs/release/configguide/featureconfig.rst new file mode 100644 index 00000000..f9e197b7 --- /dev/null +++ b/docs/release/configguide/featureconfig.rst @@ -0,0 +1,8 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +============================= +Barometer Configuration Guide +============================= +Information on configuring Barometer are included in the :ref:`feature userguide `. + diff --git a/docs/release/configguide/postinstall.rst b/docs/release/configguide/postinstall.rst index 7424cec7..dad56b99 100644 --- a/docs/release/configguide/postinstall.rst +++ b/docs/release/configguide/postinstall.rst @@ -1,3 +1,5 @@ +.. _barometer-postinstall: + .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 @@ -10,6 +12,14 @@ This document describes briefly the methods of validating the Barometer installa Automated post installation activities -------------------------------------- +.. This section will include how to run plugin validation tests, when they are created/merged. +.. This section will also include some troubleshooting and debugging information. + +.. note:: This section is outdated and needs to be updated. + +.. TODO: Update this section; post-installation/verification shouldn't be in + the config guide. It should be in testing. + The Barometer test-suite in Functest is called ``barometercollectd`` and is part of the ``Features`` tier. Running these tests is done automatically by the OPNFV deployment pipeline on the supported scenarios. The testing consists of basic verifications that each plugin is functional per their @@ -19,7 +29,65 @@ default configurations. Inside the Functest container, the detailed results can Barometer post configuration procedures --------------------------------------- The functionality for each plugin (such as enabling/disabling and configuring its capabilities) -is controlled as described in the User Guide through their individual ``.conf`` file located in -the ``/etc/collectd/collectd.conf.d/`` folder on the compute node(s). In order for any changes to -take effect, the collectd service must be stopped and then started again. +is controlled as described in the :ref:`User Guide ` through their individual +``.conf`` file located in the ``/etc/collectd/collectd.conf.d/`` on the host(s). In order for any +changes to take effect, the collectd service must be stopped and then started again. + +Plugin verification +~~~~~~~~~~~~~~~~~~~ +Once collectd has been installed and deployed, you will see metrics from most plugins immediately. However, in some cases, you may want to verify that the configuration is correct and that the plugion is functioning as intended (particularly during development, or when testing an experimental version). The following sections provide some verification steps to make sure the plugins are working as expected. + +MCElog +^^^^^^ +On the collectd host, you can induce an event monitored by the plugins; e.g. a corrected memory error: + +.. code:: bash + + $ git clone https://git.kernel.org/pub/scm/utils/cpu/mce/mce-inject.git + $ cd mce-inject + $ make + $ modprobe mce-inject + +Modify the test/corrected script to include the following: + +.. code:: bash + + CPU 0 BANK 0 + STATUS 0xcc00008000010090 + ADDR 0x0010FFFFFFF + +Inject the error: + +.. code:: bash + + $ ./mce-inject < test/corrected + +.. TODO: How to check that the event was propogated to collectd + +.. _barometer-docker-verification: + +Barometer post installation verification on barometer-collectd container +------------------------------------------------------------------------ + +The following steps describe how to perform simple "manual" testing of the Barometer components +after :ref:`successfully deploying the barometer-collectd container`: + +1. Connect to any compute node and ensure that the collectd container is running. + + .. code:: bash + + root@host2:~# docker ps | grep collectd + + You should see the container ``opnfv/barometer-collectd`` running. + +2. Use a web browser to connect to Grafana at ``http://:3000/``, using the hostname or + IP of your server and port 3000. Log in with admin/admin. You will see ``collectd`` + InfluxDB database in the ``Data Sources``. Also, you will notice metrics coming in the several + dashboards such as ``CPU Usage`` and ``Host Overview``. + +For more details on the Barometer containers, Grafana and InfluxDB, please refer to +the following documentation links: + +`Barometer Containers wiki page `_ +:ref:`Barometer Docker install guide` diff --git a/docs/release/userguide/feature.userguide.rst b/docs/release/userguide/feature.userguide.rst index 2f4639c7..02f8bdaa 100644 --- a/docs/release/userguide/feature.userguide.rst +++ b/docs/release/userguide/feature.userguide.rst @@ -1,3 +1,5 @@ +.. _feature-userguide: + .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 .. (c) Anuket and others diff --git a/docs/release/userguide/installguide.docker.rst b/docs/release/userguide/installguide.docker.rst index 20836af7..38d467a4 100644 --- a/docs/release/userguide/installguide.docker.rst +++ b/docs/release/userguide/installguide.docker.rst @@ -1,7 +1,7 @@ +.. _barometer-docker-userguide: .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 .. (c) Anuket and others -.. _barometer-docker-userguide: ===================================== Anuket Barometer Docker Install Guide @@ -31,7 +31,9 @@ For steps to build and run VES and Kafka images please see section `Build and Ru For overview of running VES application with Kafka please see the :ref:`VES Application User Guide ` -For an alternative installation method using ansible, please see the :ref:`Barometer One Click Install Guide `. +For an alternative installation method using ansible, please see the :ref:`Barometer One Click Install Guide `. + +For post-installation verification and troubleshooting, please see the :ref:`Barometer post installation guide `. Barometer Docker Images Description ----------------------------------- @@ -93,7 +95,7 @@ The Barometer project's VES application and Kafka docker images are based on a C docker image has a dependancy on `Zookeeper `_. Kafka must be able to connect and register with an instance of Zookeeper that is either running on local or remote host. Kafka recieves and stores metrics recieved from Collectd. VES application pulls latest metrics from Kafka -which it normalizes into VES format for sending to a VES collector. Please see details in +which it normalizes into VES format for sending to a VES collector. Please see details in :ref:`VES Application User Guide ` Installing Docker -- cgit 1.2.3-korg