[WIP] - Reporting support for Openstack and K8S Test Runs.

This patch adds support for generating report when tests are run for K8S
and openstack.

Added cloud-information gathering tool.
Move the tool to report folder. Call the save cloud information from
vsperf.

Fixed issues reported by Al. Additional fixes.

JIRA: VINEPERF-644

Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
Change-Id: I4aea2e52a4c6700f80624f3e1828b74a108e03ba

5 files changed, 298 insertions, 0 deletions

diff --git a/tools/os_deploy_tgen/osclients/openstack.py b/tools/os_deploy_tgen/osclients/openstack.py
index 58297e6c..4561db51 100644
--- a/tools/os_deploy_tgen/osclients/openstack.py
+++ b/tools/os_deploy_tgen/osclients/openstack.py
@@ -63,6 +63,7 @@ class OpenStackClient():
         if openstack_params['os_insecure']:
             cloud_config.config['verify'] = False
             cloud_config.config['cacert'] = None
+        self.conn = connection.Connection(config=cloud_config)
         self.keystone_session = cloud_config.get_session()
         self.nova = cloud_config.get_legacy_client('compute')
         self.neutron = cloud_config.get_legacy_client('network')
diff --git a/tools/report/cloud_report.jinja b/tools/report/cloud_report.jinja
new file mode 100644
index 00000000..9c0afecf
--- /dev/null
+++ b/tools/report/cloud_report.jinja
@@ -0,0 +1,136 @@
+{#
+This work is licensed under a Creative Commons Attribution 4.0 International License.
+http://creativecommons.org/licenses/by/4.0
+#}
+
+# CHARACTERIZE VIRTUAL NETWORK PERFORMANCE FOR TELCO NFV USE CASES LEVEL TEST REPORT
+
+## Table of Contents
+
+- [1. Introduction](#Introduction)
+  - [1.1. Document identifier](#DocId)
+  - [1.2. Scope](#Scope)
+  - [1.3. References](#References)
+- [2. Details of the Level Test Report](#DetailsoftheLevelTestReport)
+  - [2.1. Overview of test results](#OverviewOfTestResults)
+  - [2.2. Detailed test results](#DetailedTestResults)
+  - [2.3. Rationale for decisions](#RationaleForDecisions)
+  - [2.4. Conclusions and recommendations](#ConclusionsandRecommendations)
+- [3. General](#General)
+  - [3.1. Glossary](#Glossary)
+  - [3.2. Document change procedures and history](#DocChangeProceduresandHistory)
+
+---
+
+<a name="Introduction"></a>
+## 1. Introduction
+
+The objective of the Anuket project titled **"Characterise Performance of
+virtual network for Telco NFV Use Cases"**, is to evaluate virtual networking
+in baremetal and openstack or kubernetes clouds, to identify its
+suitability for a Telco Network Function Virtualization (NFV) usecases. As
+well as this, the project aims to identify any gaps or bottlenecks in order to
+drive architectural changes to improve virtual network performance and
+determinism. The purpose of this document is to summarize the results of the
+tests carried out on the openstack or kubernetes clouds, and, from these
+results, provide evaluations and recommendations for the virtual networking.
+Test results will be outlined in
+[Details of the Level Test Report](#DetailsoftheLevelTestReport),
+preceded by the [Document Identifier](#DocId), [Scope](#Scope) and
+[References](#References).
+
+This document is currently in draft form.
+
+<a name="DocId"></a>
+### 1.1. Document identifier
+
+The document id will be used to uniquely identify versions of the LTR. The
+format for the document id will be:
+`OPNFV_vswitchperf_LTR_ver_NUM_MONTH_YEAR_AUTHOR_STATUS`, where by the AUTHOR
+field should be replaced with the initials of the author and the status is one
+of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of
+the LTR is: `OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT`.
+
+<a name="Scope"></a>
+### 1.2. Scope
+
+The scope of this report is to detail the results of the tests that have been
+performed on the virtual Networking. This report will also evaluate the results of
+these tests and, based on these evaluations, provide recommendations on the
+suitability of the virtual networking for use in a Telco NFV environment.
+
+<a name="References"></a>
+### 1.3. References
+
+- `OPNFV_vswitchperf_LTD_ver_1.6_Jan_15_DRAFT`
+
+---
+
+<a name="DetailsoftheLevelTestReport"></a>
+## 2. Details of the Level Test Report
+
+This section provides an overview of the test results ([Section
+2.1.](#OveriewOfTestResults)) as well as detailed test results for each test
+([Section 2.2.](#DetailedTestResults)). Also included are the rationale used to
+evaluate each test ([Section 2.3.](#RationaleForDecisions)) and the conclusions
+and recommendations for each test ([Section
+2.4.](#ConclusionsandRecommendations)).
+
+<a name="OverviewOfTestResults"></a>
+### 2.1. Overview of test results
+
+##### Test Environment
+
+Below is the environment that the test was performed in:
+
+- Cloud: {{tests[0].env.os}}
+
+
+For each test, a summary of the key test results is provided.
+{% for test in tests %}
+#### Test ID: {{ test.ID }}
+
+Below are test details:
+
+- Test ID: {{ "%s"|format(test.id) }}
+- Description: {{ "%s"|format(test.conf['Description']) }}
+- Deployment: {{ "%s"|format(test.deployment) }}
+- Traffic type: {{ "%s"|format(test.result['type']) }}
+- Packet size: {{ "%s"|format(test.result['packet_size']) }}
+- Bidirectional : {{ "%s"|format(test.conf['bidir']) }}
+{%- if test.result['tunnel_type'] %}
+- Tunnel type: {{ "%s"|format(test.result['tunnel_type']) }}
+{%- endif %}
+{% endfor %}
+
+<a name="DetailedTestResults"></a>
+### 2.2. Detailed test results
+
+A detailed summary of the main results for each test is outlined below.
+{% for test in tests %}
+#### Test ID: {{ test.ID }}
+
+##### Results/Metrics Collected
+
+The following are the metrics obtained during this test:
+
+|             Metric             |             Result             |
+| ------------------------------ | ------------------------------ |
+{%- for item, value in test.result.items() %}
+| {{ "%-30s | %30s |"|format(item,value)}}
+{%- endfor %}
+
+
+<a name="RationaleForDecisions"></a>
+### 2.3. Rationale for decisions
+
+The tests conducted do not have pass/fail/conditional-pass criteria. The test
+is simply conducted and the results are reported.
+
+---
+
+<a name="Glossary"></a>
+### 3.1. Glossary
+
+- NFV - Network Function Virtualization
+- Mbps - 1,000,000bps
diff --git a/tools/report/cloud_report_rst.jinja b/tools/report/cloud_report_rst.jinja
new file mode 100644
index 00000000..7ecd0297
--- /dev/null
+++ b/tools/report/cloud_report_rst.jinja
@@ -0,0 +1,49 @@
+{#
+ Copyright (c) 2016-2017 Intel corporation.
+
+ All rights reserved. This program and the accompanying materials
+ are made available under the terms of the Apache License, Version 2.0
+ which accompanies this distribution, and is available at
+ http://www.apache.org/licenses/LICENSE-2.0
+#}
+
+Test ID: {{ tests[0].ID }}
+--------------------------
+
+Test Environment
+~~~~~~~~~~~~~~~~
+Below is the environment that the test was performed in:
+
+* CLOUD: {{tests[0].env.os}}
+
+Below are test details:
+
+* Test ID: {{ "%s"|format(tests[0].id) }}
+* Description: {{ "%s"|format(tests[0].conf['Description']) }}
+* Deployment: {{ "%s"|format(tests[0].deployment) }}
+* Traffic type: {{ "%s"|format(tests[0].result['type']) }}
+* Bidirectional : {{ "%s"|format(tests[0].conf['bidir']) }}
+{%- if tests[0].result['tunnel_type'] %}
+* Tunnel type: {{ "%s"|format(tests[0].result['tunnel_type']) }}
+{%- endif %}
+{% for test in tests %}
+Test results for packet size: {{ "%s"|format(test.result['packet_size']) }}
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A detailed summary of the main results is outlined below.
+
+Results/Metrics Collected
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following are the metrics obtained during this test:
+
+========================== ==================================
+        Metric                          Result
+========================== ==================================
+{%- for item, value in test.result.items() %}
+{{ "%-30s %30s"|format(item,value)}}
+{%- endfor %}
+========================== ==================================
+
+.. There must be blank lines around to ensure correct formatting.
+
+
diff --git a/tools/report/cloudinfo.py b/tools/report/cloudinfo.py
new file mode 100644
index 00000000..41faac0b
--- /dev/null
+++ b/tools/report/cloudinfo.py
@@ -0,0 +1,104 @@
+# Copyright 2021 Spirent Communications.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Get Cloud information and save it a file
+"""
+
+import os
+import json
+from kubernetes import client, config
+from kubernetes.client.rest import ApiException
+from conf import settings
+from tools.os_deploy_tgen.utilities import utils
+from tools.os_deploy_tgen.osclients import openstack
+
+def save_kubernetes_info():
+    """
+    Save Kubernetes Cluster Info
+    """
+    config.load_kube_config(settings.getValue('K8S_CONFIG_FILEPATH'))
+    with open(os.path.join(settings.getValue('RESULTS_PATH'),
+                           'cloud_info.txt'), 'a+') as outf:
+        api = client.CoreV1Api()
+        try:
+            node_info = api.list_node()
+        except ApiException as err:
+            raise Exception from err
+        for ni_item in node_info.items:
+            outf.write("\n ******************************************** \n")
+            outf.write("\n System Information \n")
+            sinfo = {'Architecture': ni_item.status.node_info.architecture,
+                     'Container Runtime Version':ni_item.status.node_info.container_runtime_version,
+                     'kernel version':ni_item.status.node_info.kernel_version,
+                     'Kube Proxy Version':ni_item.status.node_info.kube_proxy_version,
+                     'Kubelet Version':ni_item.status.node_info.kubelet_version,
+                     'Operating System':ni_item.status.node_info.operating_system,
+                     'OS Image':ni_item.status.node_info.os_image}
+            json.dump(sinfo, outf, indent=4)
+            outf.write("\n List of Addresses \n")
+            for addrs in ni_item.status.addresses:
+                entry = {'address': addrs.address, 'type': addrs.type}
+                json.dump(entry, outf, indent=4)
+            outf.write("\n Allocatable Resources \n")
+            json.dump(ni_item.status.allocatable, outf, indent=4)
+            outf.write("\n Available Resources \n")
+            json.dump(ni_item.status.capacity, outf, indent=4)
+        api = client.VersionApi()
+        try:
+            version_info = api.get_code()
+        except ApiException as err:
+            raise Exception from err
+        outf.write("\n Version Information \n")
+        vinfo = {'git_commit': version_info.git_commit, 'git_version': version_info.git_version,
+                 'platform': version_info.platform, 'go_version': version_info.go_version}
+        json.dump(vinfo, outf, indent=4)
+
+def save_openstack_info():
+    """
+    Save Openstack Info
+    """
+    osclient = openstack.OpenStackClient(utils.pack_openstack_params())
+    hypervisors = osclient.conn.list_hypervisors()
+    with open(os.path.join(settings.getValue('RESULTS_PATH'),
+                           'cloud_info.txt'), 'a+') as outf:
+        for hypervisor in hypervisors:
+            outf.write("\n ***************************************** \n")
+            outf.write(f"Hypervisor status:      {hypervisor.status} \n")
+            outf.write(f"Hypervisor type:        {hypervisor.hypervisor_type} \n")
+            outf.write(f"Hypervisor CPU-Arch:    {hypervisor.cpu_info['arch']} \n")
+            outf.write(f"Hypervisor CPU-model:    {hypervisor.cpu_info['model']} \n")
+            outf.write(f"Hypervisor CPU-vendor:    {hypervisor.cpu_info['vendor']} \n")
+            outf.write(f"Hypervisor CPU-topology:    {hypervisor.cpu_info['topology']} \n")
+            outf.write(f"Hypervisor id:          {hypervisor.id} \n")
+            outf.write(f"Hypervisor state:       {hypervisor.state} \n")
+            outf.write(f"Hypervisor host IP:     {hypervisor.host_ip} \n")
+            outf.write(f"Hypervisor running VMs: {hypervisor.running_vms} \n")
+            outf.write(f"Hyperviror hostname: {hypervisor.name} \n")
+        outf.write("\n ***************************************** \n")
+        version_data = osclient.keystone_session.get_all_version_data()
+        json.dump(version_data, outf, indent=2)
+
+def save_cloud_info():
+    """
+    Save Cloud Information
+    """
+    if settings.getValue('K8S'):
+        save_kubernetes_info()
+    elif settings.getValue('OPENSTACK'):
+        save_openstack_info()
+    else:
+        print("Unsupported Cloud")
+        return -1
+    return 0
diff --git a/vsperf b/vsperf
index 773ad759..1fb52429 100755
--- a/vsperf
+++ b/vsperf
@@ -47,6 +47,7 @@ from tools import functions
 from tools.pkt_gen import trafficgen
 from tools.opnfvdashboard import opnfvdashboard
 from tools.os_deploy_tgen import osdt
+from tools.report import cloudinfo
 sys.dont_write_bytecode = True
 
 VERBOSITY_LEVELS = {
@@ -728,6 +729,7 @@ def main():
         settings.setValue('K8S', False)
 
     if args['openstack']:
+        settings.setValue('OPENSTACK', True)
         result = osdt.deploy_testvnf()
         if result:
             _LOGGER.info('TestVNF successfully deployed on Openstack')
@@ -735,6 +737,9 @@ def main():
         else:
             _LOGGER.error('Failed to deploy TestVNF in Openstac')
             sys.exit(1)
+    else:
+        settings.setValue('OPENSTACK', False)
+
     # update paths to trafficgens if required
     if settings.getValue('mode') == 'trafficgen':
         functions.settings_update_paths()
@@ -879,6 +884,9 @@ def main():
         suite = unittest.TestSuite()
         settings_snapshot = copy.deepcopy(settings.__dict__)
 
+        if settings.getValue('OPENSTACK') or settings.getValue('K8S'):
+            cloudinfo.save_cloud_info()
+
         for i, cfg in enumerate(selected_tests):
             settings.setValue('_TEST_INDEX', i)
             test_name = cfg.get('Name', '<Name not set>')