diff options
Diffstat (limited to 'docs/testing')
| -rw-r--r-- | docs/testing/ecosystem/energy-monitoring.rst | 260 | ||||
| -rw-r--r-- | docs/testing/ecosystem/overview.rst | 22 |
2 files changed, 273 insertions, 9 deletions
diff --git a/docs/testing/ecosystem/energy-monitoring.rst b/docs/testing/ecosystem/energy-monitoring.rst new file mode 100644 index 000000000..fa1f28214 --- /dev/null +++ b/docs/testing/ecosystem/energy-monitoring.rst @@ -0,0 +1,260 @@ +.. _energy-monitoring: + +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. SPDX-License-Identifier: CC-BY-4.0 +.. (c) Open Platform for NFV Project, Inc. and its contributors + +Power Consumption Monitoring Framework +====================================== + +Introduction +------------ +Power consumption is a key driver for NFV. +As an end user is interested to know which application is good or bad regarding +power consumption and explains why he/she has to plug his/her smartphone every +day, we would be interested to know which VNF is power consuming. + +Power consumption is hard to evaluate empirically. It is however possible to +collect information and leverage Pharos federation to try to detect some +profiles/footprints. +In fact thanks to CI, we know that we are running a known/deterministic list of +cases. The idea is to correlate this knowledge with the power consumption to try +at the end to find statistical biais. + + +High Level Architecture +----------------------- + +The energy recorder high level architecture may be described as follows: + +.. figure:: ../../images/energyrecorder.png + :align: center + :alt: Energy recorder high level architecture + +The energy monitoring system in based on 3 software components: + + * Power info collector: poll server to collect instantaneous power consumption information + * Energy recording API + influxdb: On one leg receive servers consumption and + on the other, scenarios notfication. It then able to establish te correlation + between consumption and scenario and stores it into a time-series database (influxdb) + * Python SDK: A Python SDK using decorator to send notification to Energy + recording API from testcases scenarios + +Power Info Collector +-------------------- +It collects instantaneous power consumption information and send it to Event +API in charge of data storing. +The collector use different connector to read the power consumption on remote +servers: + + * IPMI: this is the basic method and is manufacturer dependent. Depending on manufacturer, refreshing delay may vary (generally for 10 to 30 sec.) + * RedFish: redfish is an industry RESTFUL API for hardware managment. Unfortunatly it is not yet supported by many suppliers. + * ILO: HP RESTFULL API: This connector support as well 2.1 as 2.4 version of HP-ILO + +IPMI is supported by at least: + + * HP + * IBM + * Dell + * Nokia + * Advantech + * Lenovo + * Huawei + +Redfish API has been successfully tested on: + + * HP + * Dell + * Huawei (E9000 class servers used in OPNFV Community Labs are IPMI 2.0 + compliant and use Redfish login Interface through Browsers supporting JRE1.7/1.8) + +Several test campaigns done with physical Wattmeter showed that IPMI results +were notvery accurate but RedFish were. So if Redfish is available, it is +highly recommended to use it. + +Installation +^^^^^^^^^^^^ + +To run the server power consumption collector agent, you need to deploy a +docker container locally on your infrastructure. + +This container requires: + + * Connectivy on the LAN where server administration services (ILO, eDrac, IPMI,...) are configured and IP access to the POD's servers + * Outgoing HTTP access to the Event API (internet) + +Build the image by typing:: + + curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/docker/server-collector.dockerfile|docker build -t energyrecorder/collector - + +Create local folder on your host for logs and config files:: + + mkdir -p /etc/energyrecorder + mkdir -p /var/log/energyrecorder + +In /etc/energyrecorder create a configuration for logging in a file named +collector-logging.conf:: + + curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/server-collector/conf/collector-logging.conf.sample > /etc/energyrecorder/collector-logging.conf + +Check configuration for this file (folders, log levels.....) +In /etc/energyrecorder create a configuration for the collector in a file named +collector-settings.yaml:: + + curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/server-collector/conf/collector-settings.yaml.sample > /etc/energyrecorder/collector-settings.yaml + +Define the "PODS" section and their "servers" section according to the +environment to monitor. +Note: The "environment" key should correspond to the pod name, as defined in +the "NODE_NAME" environment variable by CI when running. + +**IMPORTANT NOTE**: To apply a new configuration, you need to kill the running +container an start a new one (see below) + +Run +^^^ + +To run the container, you have to map folder located on the host to folders in +the container (config, logs):: + + docker run -d --name energy-collector --restart=always -v /etc/energyrecorder:/usr/local/energyrecorder/server-collector/conf -v /var/log/energyrecorder:/var/log/energyrecorder energyrecorder/collector + + +Energy Recording API +-------------------- +An event API to insert contextual information when monitoring energy (e.g. +start Functest, start Tempest, destroy VM, ..) +It is associated with an influxDB to store the power consumption measures +It is hosted on a shared environment with the folling access points: + ++------------------------------------+----------------------------------------+ +| Component | Connectivity | ++====================================+========================================+ +| Energy recording API documentation | http://energy.opnfv.fr/resources/doc/ | ++------------------------------------+----------------------------------------+ +| influxDB (data) | http://energy.opnfv.fr:8086 | ++------------------------------------+----------------------------------------+ + +In you need, you can also host your own version of the Energy recording API +(in such case, the Python SDK may requires a settings update) +If you plan to use the default shared API, following steps are not required. + +Image creation +^^^^^^^^^^^^^^ +First, you need to buid an image:: + + curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/docker/recording-api.dockerfile|docker build -t energyrecorder/api - + +Setup +^^^^^ +Create local folder on your host for logs and config files:: + + mkdir -p /etc/energyrecorder + mkdir -p /var/log/energyrecorder + mkdir -p /var/lib/influxdb + +In /etc/energyrecorder create a configuration for logging in a file named +webapp-logging.conf:: + + curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/recording-api/conf/webapp-logging.conf.sample > /etc/energyrecorder/webapp-logging.conf + +Check configuration for this file (folders, log levels.....) + +In /etc/energyrecorder create a configuration for the collector in a file +named webapp-settings.yaml:: + + curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/recording-api/conf/webapp-settings.yaml.sample > /etc/energyrecorder/webapp-settings.yaml + +Normaly included configuration is ready to use except username/passwer for +influx (see run-container.sh bellow). Use here the admin user. + +**IMPORTANT NOTE**: To apply a new configuration, you need to kill the running +container an start a new one (see bellow) + +Run +^^^ +To run the container, you have to map folder located on the host to folders in +the container (config, logs):: + + docker run -d --name energyrecorder-api -p 8086:8086 -p 8888:8888 -v /etc/energyrecorder:/usr/local/energyrecorder/web.py/conf -v /var/log/energyrecorder/:/var/log/energyrecorder -v /var/lib/influxdb:/var/lib/influxdb energyrecorder/webapp admin-influx-user-name admin-password readonly-influx-user-name user-password + +with + ++---------------------------+--------------------------------------------+ +| Parameter name | Description | ++===========================+============================================+ +| admin-influx-user-name | Influx user with admin grants to create | +|---------------------------+--------------------------------------------+ +| admin-password | Influx password to set to admin user | +|---------------------------+--------------------------------------------+ +| readonly-influx-user-name | Influx user with readonly grants to create | +|---------------------------+--------------------------------------------+ +| user-password | Influx password to set to readonly user | ++---------------------------+--------------------------------------------+ + +**NOTE**: Local folder /var/lib/influxdb is the location web influx data are +stored. You may used anything else at your convience. Just remember to define +this mapping properly when running the container. + +Power consumption Python SDK +---------------------------- +a Python SDK - almost not intrusive, based on python decorator to trigger call +to the event API. + +It is currently hosted in Functest repo but if other projects adopt it, a +dedicated project could be created and/or it could be hosted in Releng. + +How to use the SDK +^^^^^^^^^^^^^^^^^^ + +import the energy library:: + + import functest.energy.energy as energy + +Notify that you want power recording in your testcase:: + + @energy.enable_recording + def run(self): + self.do_some_stuff1() + self.do_some_stuff2() + +If you want to register additional steps during the scenarios you can to it in +2 different ways. + +Notify step on method definition:: + + @energy.set_step("step1") + def do_some_stuff1(self): + ... + @energy.set_step("step2") + def do_some_stuff2(self): + +Notify directly from code:: + + @energy.enable_recording + def run(self): + Energy.set_step("step1") + self.do_some_stuff1() + ... + Energy.set_step("step2") + self.do_some_stuff2() + +SDK Setting +^^^^^^^^^^^ +Settings delivered in the project git are ready to use and assume that you will +use the sahre energy recording API. +If you want to use an other instance, you have to update the key +"energy_recorder.api_url" in <FUNCTEST>/functest/ci/config_functest.yaml" by +setting the proper hostname/IP + +Results +------- +Here is an example of result comming from LF POD2. This sequence represents +several CI runs in a raw. (0 power corresponds to hard reboot of the servers) + +You may connect http://energy.opnfv.fr:3000 for more results (ask for +credentials to infra team). + +.. figure:: ../../images/energy_LF2.png + :align: center + :alt: Energy monitoring of LF POD2 diff --git a/docs/testing/ecosystem/overview.rst b/docs/testing/ecosystem/overview.rst index e751e4008..aefcc1f1e 100644 --- a/docs/testing/ecosystem/overview.rst +++ b/docs/testing/ecosystem/overview.rst @@ -38,7 +38,7 @@ The major testing projects are described in the table below: +----------------+---------------------------------------------------------+ | Project | Description | +================+=========================================================+ -| Bottlenecks | This project aims to find system bottlenecks by testing | +| Bottlenecks | This project aims to find system bottlenecks by testing | | | and verifying OPNFV infrastructure in a staging | | | environment before committing it to a production | | | environment. Instead of debugging a deployment in | @@ -58,7 +58,7 @@ The major testing projects are described in the table below: | Dovetail | This project intends to define and provide a set of | | | OPNFV related validation criteria/tests that will | | | provide input for the OPNFV Complaince Verification | -| | Program. The Dovetail project is executed with the | +| | Program. The Dovetail project is executed with the | | | guidance and oversight of the Complaince and | | | Certification (C&C) committee and work to secure the | | | goals of the C&C committee for each release. The | @@ -86,10 +86,10 @@ The major testing projects are described in the table below: | | RFC-2544. Available as a Docker container with simple | | | command line and REST interfaces. | | | Easy to use as it takes care of most of the guesswork | - | generally associated to data plane benchmarking. | +| | generally associated to data plane benchmarking. | | | Can run in any lab or in production environments. | +----------------+---------------------------------------------------------+ -| Qtip | QTIP as the project for "Platform Performance | +| QTIP | QTIP as the project for "Platform Performance | | | Benchmarking" in OPNFV aims to provide user a simple | | | indicator for performance, supported by comprehensive | | | testing data and transparent calculation formula. | @@ -97,7 +97,7 @@ The major testing projects are described in the table below: | | performance benchmarking which helps users to build | | | indicators by themselves with ease. | +----------------+---------------------------------------------------------+ -| Storperf | The purpose of this project is to provide a tool to | +| StorPerf | The purpose of this project is to provide a tool to | | | measure block and object storage performance in an NFVI.| | | When complemented with a characterization of typical VF | | | storage performance requirements, it can provide | @@ -239,6 +239,7 @@ PUT/DELETE/POST operations of the TestAPI now require token based authorization. to be added in the request using a header 'X-Auth-Token' for access to the database. e.g:: + headers['X-Auth-Token'] The value of the header i.e the token can be accessed in the jenkins environment variable @@ -317,6 +318,9 @@ use Bitergia for creating highly flexible dashboards `[TST5]`_. :alt: Testing group testcase catalog +.. include:: ./energy-monitoring.rst + + OPNFV Test Group Information ============================ @@ -341,7 +345,7 @@ Reference Documentation +----------------+---------------------------------------------------------+ | Project | Documentation links | +================+=========================================================+ -| Bottlenecks | https://wiki.opnfv.org/display/bottlenecks/Bottlenecks | +| Bottlenecks | https://wiki.opnfv.org/display/bottlenecks/Bottlenecks | +----------------+---------------------------------------------------------+ | CPerf | https://wiki.opnfv.org/display/cperf | +----------------+---------------------------------------------------------+ @@ -351,11 +355,11 @@ Reference Documentation +----------------+---------------------------------------------------------+ | NFVbench | https://wiki.opnfv.org/display/nfvbench/ | +----------------+---------------------------------------------------------+ -| Qtip | https://wiki.opnfv.org/display/qtip | +| QTIP | https://wiki.opnfv.org/display/qtip | +----------------+---------------------------------------------------------+ -| Storperf | https://wiki.opnfv.org/display/storperf/Storperf | +| StorPerf | https://wiki.opnfv.org/display/storperf/Storperf | +----------------+---------------------------------------------------------+ -| VSperf | https://wiki.opnfv.org/display/vsperf | +| VSPERF | https://wiki.opnfv.org/display/vsperf | +----------------+---------------------------------------------------------+ | Yardstick | https://wiki.opnfv.org/display/yardstick/Yardstick | +----------------+---------------------------------------------------------+ |