summaryrefslogtreecommitdiffstats
path: root/docs/testing
diff options
context:
space:
mode:
Diffstat (limited to 'docs/testing')
-rw-r--r--docs/testing/ecosystem/energy-monitoring.rst260
-rw-r--r--docs/testing/ecosystem/overview.rst22
2 files changed, 273 insertions, 9 deletions
diff --git a/docs/testing/ecosystem/energy-monitoring.rst b/docs/testing/ecosystem/energy-monitoring.rst
new file mode 100644
index 000000000..fa1f28214
--- /dev/null
+++ b/docs/testing/ecosystem/energy-monitoring.rst
@@ -0,0 +1,260 @@
+.. _energy-monitoring:
+
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. SPDX-License-Identifier: CC-BY-4.0
+.. (c) Open Platform for NFV Project, Inc. and its contributors
+
+Power Consumption Monitoring Framework
+======================================
+
+Introduction
+------------
+Power consumption is a key driver for NFV.
+As an end user is interested to know which application is good or bad regarding
+power consumption and explains why he/she has to plug his/her smartphone every
+day, we would be interested to know which VNF is power consuming.
+
+Power consumption is hard to evaluate empirically. It is however possible to
+collect information and leverage Pharos federation to try to detect some
+profiles/footprints.
+In fact thanks to CI, we know that we are running a known/deterministic list of
+cases. The idea is to correlate this knowledge with the power consumption to try
+at the end to find statistical biais.
+
+
+High Level Architecture
+-----------------------
+
+The energy recorder high level architecture may be described as follows:
+
+.. figure:: ../../images/energyrecorder.png
+ :align: center
+ :alt: Energy recorder high level architecture
+
+The energy monitoring system in based on 3 software components:
+
+ * Power info collector: poll server to collect instantaneous power consumption information
+ * Energy recording API + influxdb: On one leg receive servers consumption and
+ on the other, scenarios notfication. It then able to establish te correlation
+ between consumption and scenario and stores it into a time-series database (influxdb)
+ * Python SDK: A Python SDK using decorator to send notification to Energy
+ recording API from testcases scenarios
+
+Power Info Collector
+--------------------
+It collects instantaneous power consumption information and send it to Event
+API in charge of data storing.
+The collector use different connector to read the power consumption on remote
+servers:
+
+ * IPMI: this is the basic method and is manufacturer dependent. Depending on manufacturer, refreshing delay may vary (generally for 10 to 30 sec.)
+ * RedFish: redfish is an industry RESTFUL API for hardware managment. Unfortunatly it is not yet supported by many suppliers.
+ * ILO: HP RESTFULL API: This connector support as well 2.1 as 2.4 version of HP-ILO
+
+IPMI is supported by at least:
+
+ * HP
+ * IBM
+ * Dell
+ * Nokia
+ * Advantech
+ * Lenovo
+ * Huawei
+
+Redfish API has been successfully tested on:
+
+ * HP
+ * Dell
+ * Huawei (E9000 class servers used in OPNFV Community Labs are IPMI 2.0
+ compliant and use Redfish login Interface through Browsers supporting JRE1.7/1.8)
+
+Several test campaigns done with physical Wattmeter showed that IPMI results
+were notvery accurate but RedFish were. So if Redfish is available, it is
+highly recommended to use it.
+
+Installation
+^^^^^^^^^^^^
+
+To run the server power consumption collector agent, you need to deploy a
+docker container locally on your infrastructure.
+
+This container requires:
+
+ * Connectivy on the LAN where server administration services (ILO, eDrac, IPMI,...) are configured and IP access to the POD's servers
+ * Outgoing HTTP access to the Event API (internet)
+
+Build the image by typing::
+
+ curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/docker/server-collector.dockerfile|docker build -t energyrecorder/collector -
+
+Create local folder on your host for logs and config files::
+
+ mkdir -p /etc/energyrecorder
+ mkdir -p /var/log/energyrecorder
+
+In /etc/energyrecorder create a configuration for logging in a file named
+collector-logging.conf::
+
+ curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/server-collector/conf/collector-logging.conf.sample > /etc/energyrecorder/collector-logging.conf
+
+Check configuration for this file (folders, log levels.....)
+In /etc/energyrecorder create a configuration for the collector in a file named
+collector-settings.yaml::
+
+ curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/server-collector/conf/collector-settings.yaml.sample > /etc/energyrecorder/collector-settings.yaml
+
+Define the "PODS" section and their "servers" section according to the
+environment to monitor.
+Note: The "environment" key should correspond to the pod name, as defined in
+the "NODE_NAME" environment variable by CI when running.
+
+**IMPORTANT NOTE**: To apply a new configuration, you need to kill the running
+container an start a new one (see below)
+
+Run
+^^^
+
+To run the container, you have to map folder located on the host to folders in
+the container (config, logs)::
+
+ docker run -d --name energy-collector --restart=always -v /etc/energyrecorder:/usr/local/energyrecorder/server-collector/conf -v /var/log/energyrecorder:/var/log/energyrecorder energyrecorder/collector
+
+
+Energy Recording API
+--------------------
+An event API to insert contextual information when monitoring energy (e.g.
+start Functest, start Tempest, destroy VM, ..)
+It is associated with an influxDB to store the power consumption measures
+It is hosted on a shared environment with the folling access points:
+
++------------------------------------+----------------------------------------+
+| Component | Connectivity |
++====================================+========================================+
+| Energy recording API documentation | http://energy.opnfv.fr/resources/doc/ |
++------------------------------------+----------------------------------------+
+| influxDB (data) | http://energy.opnfv.fr:8086 |
++------------------------------------+----------------------------------------+
+
+In you need, you can also host your own version of the Energy recording API
+(in such case, the Python SDK may requires a settings update)
+If you plan to use the default shared API, following steps are not required.
+
+Image creation
+^^^^^^^^^^^^^^
+First, you need to buid an image::
+
+ curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/docker/recording-api.dockerfile|docker build -t energyrecorder/api -
+
+Setup
+^^^^^
+Create local folder on your host for logs and config files::
+
+ mkdir -p /etc/energyrecorder
+ mkdir -p /var/log/energyrecorder
+ mkdir -p /var/lib/influxdb
+
+In /etc/energyrecorder create a configuration for logging in a file named
+webapp-logging.conf::
+
+ curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/recording-api/conf/webapp-logging.conf.sample > /etc/energyrecorder/webapp-logging.conf
+
+Check configuration for this file (folders, log levels.....)
+
+In /etc/energyrecorder create a configuration for the collector in a file
+named webapp-settings.yaml::
+
+ curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/recording-api/conf/webapp-settings.yaml.sample > /etc/energyrecorder/webapp-settings.yaml
+
+Normaly included configuration is ready to use except username/passwer for
+influx (see run-container.sh bellow). Use here the admin user.
+
+**IMPORTANT NOTE**: To apply a new configuration, you need to kill the running
+container an start a new one (see bellow)
+
+Run
+^^^
+To run the container, you have to map folder located on the host to folders in
+the container (config, logs)::
+
+ docker run -d --name energyrecorder-api -p 8086:8086 -p 8888:8888 -v /etc/energyrecorder:/usr/local/energyrecorder/web.py/conf -v /var/log/energyrecorder/:/var/log/energyrecorder -v /var/lib/influxdb:/var/lib/influxdb energyrecorder/webapp admin-influx-user-name admin-password readonly-influx-user-name user-password
+
+with
+
++---------------------------+--------------------------------------------+
+| Parameter name | Description |
++===========================+============================================+
+| admin-influx-user-name | Influx user with admin grants to create |
+|---------------------------+--------------------------------------------+
+| admin-password | Influx password to set to admin user |
+|---------------------------+--------------------------------------------+
+| readonly-influx-user-name | Influx user with readonly grants to create |
+|---------------------------+--------------------------------------------+
+| user-password | Influx password to set to readonly user |
++---------------------------+--------------------------------------------+
+
+**NOTE**: Local folder /var/lib/influxdb is the location web influx data are
+stored. You may used anything else at your convience. Just remember to define
+this mapping properly when running the container.
+
+Power consumption Python SDK
+----------------------------
+a Python SDK - almost not intrusive, based on python decorator to trigger call
+to the event API.
+
+It is currently hosted in Functest repo but if other projects adopt it, a
+dedicated project could be created and/or it could be hosted in Releng.
+
+How to use the SDK
+^^^^^^^^^^^^^^^^^^
+
+import the energy library::
+
+ import functest.energy.energy as energy
+
+Notify that you want power recording in your testcase::
+
+ @energy.enable_recording
+ def run(self):
+ self.do_some_stuff1()
+ self.do_some_stuff2()
+
+If you want to register additional steps during the scenarios you can to it in
+2 different ways.
+
+Notify step on method definition::
+
+ @energy.set_step("step1")
+ def do_some_stuff1(self):
+ ...
+ @energy.set_step("step2")
+ def do_some_stuff2(self):
+
+Notify directly from code::
+
+ @energy.enable_recording
+ def run(self):
+ Energy.set_step("step1")
+ self.do_some_stuff1()
+ ...
+ Energy.set_step("step2")
+ self.do_some_stuff2()
+
+SDK Setting
+^^^^^^^^^^^
+Settings delivered in the project git are ready to use and assume that you will
+use the sahre energy recording API.
+If you want to use an other instance, you have to update the key
+"energy_recorder.api_url" in <FUNCTEST>/functest/ci/config_functest.yaml" by
+setting the proper hostname/IP
+
+Results
+-------
+Here is an example of result comming from LF POD2. This sequence represents
+several CI runs in a raw. (0 power corresponds to hard reboot of the servers)
+
+You may connect http://energy.opnfv.fr:3000 for more results (ask for
+credentials to infra team).
+
+.. figure:: ../../images/energy_LF2.png
+ :align: center
+ :alt: Energy monitoring of LF POD2
diff --git a/docs/testing/ecosystem/overview.rst b/docs/testing/ecosystem/overview.rst
index e751e4008..aefcc1f1e 100644
--- a/docs/testing/ecosystem/overview.rst
+++ b/docs/testing/ecosystem/overview.rst
@@ -38,7 +38,7 @@ The major testing projects are described in the table below:
+----------------+---------------------------------------------------------+
| Project | Description |
+================+=========================================================+
-| Bottlenecks | This project aims to find system bottlenecks by testing |
+| Bottlenecks | This project aims to find system bottlenecks by testing |
| | and verifying OPNFV infrastructure in a staging |
| | environment before committing it to a production |
| | environment. Instead of debugging a deployment in |
@@ -58,7 +58,7 @@ The major testing projects are described in the table below:
| Dovetail | This project intends to define and provide a set of |
| | OPNFV related validation criteria/tests that will |
| | provide input for the OPNFV Complaince Verification |
-| | Program. The Dovetail project is executed with the |
+| | Program. The Dovetail project is executed with the |
| | guidance and oversight of the Complaince and |
| | Certification (C&C) committee and work to secure the |
| | goals of the C&C committee for each release. The |
@@ -86,10 +86,10 @@ The major testing projects are described in the table below:
| | RFC-2544. Available as a Docker container with simple |
| | command line and REST interfaces. |
| | Easy to use as it takes care of most of the guesswork |
- | generally associated to data plane benchmarking. |
+| | generally associated to data plane benchmarking. |
| | Can run in any lab or in production environments. |
+----------------+---------------------------------------------------------+
-| Qtip | QTIP as the project for "Platform Performance |
+| QTIP | QTIP as the project for "Platform Performance |
| | Benchmarking" in OPNFV aims to provide user a simple |
| | indicator for performance, supported by comprehensive |
| | testing data and transparent calculation formula. |
@@ -97,7 +97,7 @@ The major testing projects are described in the table below:
| | performance benchmarking which helps users to build |
| | indicators by themselves with ease. |
+----------------+---------------------------------------------------------+
-| Storperf | The purpose of this project is to provide a tool to |
+| StorPerf | The purpose of this project is to provide a tool to |
| | measure block and object storage performance in an NFVI.|
| | When complemented with a characterization of typical VF |
| | storage performance requirements, it can provide |
@@ -239,6 +239,7 @@ PUT/DELETE/POST operations of the TestAPI now require token based authorization.
to be added in the request using a header 'X-Auth-Token' for access to the database.
e.g::
+
headers['X-Auth-Token']
The value of the header i.e the token can be accessed in the jenkins environment variable
@@ -317,6 +318,9 @@ use Bitergia for creating highly flexible dashboards `[TST5]`_.
:alt: Testing group testcase catalog
+.. include:: ./energy-monitoring.rst
+
+
OPNFV Test Group Information
============================
@@ -341,7 +345,7 @@ Reference Documentation
+----------------+---------------------------------------------------------+
| Project | Documentation links |
+================+=========================================================+
-| Bottlenecks | https://wiki.opnfv.org/display/bottlenecks/Bottlenecks |
+| Bottlenecks | https://wiki.opnfv.org/display/bottlenecks/Bottlenecks |
+----------------+---------------------------------------------------------+
| CPerf | https://wiki.opnfv.org/display/cperf |
+----------------+---------------------------------------------------------+
@@ -351,11 +355,11 @@ Reference Documentation
+----------------+---------------------------------------------------------+
| NFVbench | https://wiki.opnfv.org/display/nfvbench/ |
+----------------+---------------------------------------------------------+
-| Qtip | https://wiki.opnfv.org/display/qtip |
+| QTIP | https://wiki.opnfv.org/display/qtip |
+----------------+---------------------------------------------------------+
-| Storperf | https://wiki.opnfv.org/display/storperf/Storperf |
+| StorPerf | https://wiki.opnfv.org/display/storperf/Storperf |
+----------------+---------------------------------------------------------+
-| VSperf | https://wiki.opnfv.org/display/vsperf |
+| VSPERF | https://wiki.opnfv.org/display/vsperf |
+----------------+---------------------------------------------------------+
| Yardstick | https://wiki.opnfv.org/display/yardstick/Yardstick |
+----------------+---------------------------------------------------------+