diff options
| -rw-r--r-- | tools/report/report.jinja | 42 | ||||
| -rw-r--r-- | tools/report/report.py | 24 | ||||
| -rw-r--r-- | tools/report/report_rst.jinja | 175 | ||||
| -rw-r--r-- | tools/systeminfo.py | 4 |
4 files changed, 224 insertions, 21 deletions
diff --git a/tools/report/report.jinja b/tools/report/report.jinja index f59dba72..60fccd00 100644 --- a/tools/report/report.jinja +++ b/tools/report/report.jinja @@ -20,19 +20,38 @@ <a name="Introduction"></a> ## 1. Introduction -The objective of the OPNFV project titled **"Characterise vSwitch Performance for Telco NFV Use Cases"**, is to evaluate a virtual switch to identify its suitability for a Telco Network Function Virtualization (NFV) environment. As well as this, the project aims to identify any gaps or bottlenecks in order to drive architectural changes to improve virtual switch performance and determinism. The purpose of this document is to summarize the results of the tests carried out on the virtual switch in the Network Function Virtualization Infrastructure (NFVI) and, from these results, provide evaluations and recommendations for the virtual switch. Test results will be outlined in [Details of the Level Test Report](#DetailsoftheLevelTestReport), preceded by the [Document Identifier](#DocId), [Scope](#Scope) and [References](#References). +The objective of the OPNFV project titled **"Characterise vSwitch Performance +for Telco NFV Use Cases"**, is to evaluate a virtual switch to identify its +suitability for a Telco Network Function Virtualization (NFV) environment. As +well as this, the project aims to identify any gaps or bottlenecks in order to +drive architectural changes to improve virtual switch performance and +determinism. The purpose of this document is to summarize the results of the +tests carried out on the virtual switch in the Network Function Virtualization +Infrastructure (NFVI) and, from these results, provide evaluations and +recommendations for the virtual switch. Test results will be outlined in +[Details of the Level Test Report](#DetailsoftheLevelTestReport), preceded by +the [Document Identifier](#DocId), [Scope](#Scope) and +[References](#References). This document is currently in draft form. <a name="DocId"></a> ### 1.1. Document identifier -The document id will be used to uniquely identify versions of the LTR. The format for the document id will be: `OPNFV_vswitchperf_LTR_ver_NUM_MONTH_YEAR_AUTHOR_STATUS`, where by the AUTHOR field should be replaced with the initials of the author and the status is one of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of the LTR is: `OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT`. +The document id will be used to uniquely identify versions of the LTR. The +format for the document id will be: +`OPNFV_vswitchperf_LTR_ver_NUM_MONTH_YEAR_AUTHOR_STATUS`, where by the AUTHOR +field should be replaced with the initials of the author and the status is one +of: DRAFT, REVIEWED, CORRECTED or FINAL. The document id for this version of +the LTR is: `OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT`. <a name="Scope"></a> ### 1.2. Scope -The scope of this report is to detail the results of the tests that have been performed on the virtual switch. This report will also evaluate the results of these tests and, based on these evaluations, provide recommendations on the suitability of the virtual switch for use in a Telco NFV environment. +The scope of this report is to detail the results of the tests that have been +performed on the virtual switch. This report will also evaluate the results of +these tests and, based on these evaluations, provide recommendations on the +suitability of the virtual switch for use in a Telco NFV environment. <a name="References"></a> ### 1.3. References @@ -44,7 +63,12 @@ The scope of this report is to detail the results of the tests that have been pe <a name="DetailsoftheLevelTestReport"></a> ## 2. Details of the Level Test Report -This section provides an overview of the test results ([Section 2.1.](#OveriewOfTestResults)) as well as detailed test results for each test ([Section 2.2.](#DetailedTestResults)). Also included are the rationale used to evaluate each test ([Section 2.3.](#RationaleForDecisions)) and the conclusions and recommendations for each test ([Section 2.4.](#ConclusionsandRecommendations)). +This section provides an overview of the test results ([Section +2.1.](#OveriewOfTestResults)) as well as detailed test results for each test +([Section 2.2.](#DetailedTestResults)). Also included are the rationale used to +evaluate each test ([Section 2.3.](#RationaleForDecisions)) and the conclusions +and recommendations for each test ([Section +2.4.](#ConclusionsandRecommendations)). <a name="OverviewOfTestResults"></a> ### 2.1. Overview of test results @@ -55,7 +79,8 @@ Below is the environment that the test was performed in: - OS: {{tests[0].env.os}} - Kernel Version: {{tests[0].env.kernel}} -- NIC(s): {{tests[0].env.nic}} +- NIC(s):{% for nic in tests[0].env.nics %} + - {{nic}}{% endfor %} - Board: {{tests[0].env.platform}} - CPU: {{tests[0].env.cpu}} - CPU cores: {{tests[0].env.cpu_cores}} @@ -67,9 +92,10 @@ Below is the environment that the test was performed in: - DPDK Version: {{tests[0].env.dpdk['version']}}, GIT tag: {{tests[0].env.dpdk['git_tag']}} {%- if 'vnf' in tests[0].env %} - VNF: {{tests[0].env.vnf['name']}}, Version: {{tests[0].env.vnf['version']}}, GIT tag: {{tests[0].env.vnf['git_tag']}} -- VM images: {% for guest_image in tests[0].env.guest_image %}{{guest_image}} {% endfor %} -- VM loopback apps: {% for loopback_app in tests[0].env.loopback_app %}{{loopback_app['name']}}, Version: {{loopback_app['version']}}, GIT tag: {{loopback_app['git_tag']}} - {% endfor %} +- VM images:{% for guest_image in tests[0].env.guest_image %} + - {{guest_image}}{% endfor %} +- VM loopback apps:{% for loopback_app in tests[0].env.loopback_app %} + - {{loopback_app['name']}}, Version: {{loopback_app['version']}}, GIT tag: {{loopback_app['git_tag']}}{% endfor %} {%- endif %} For each test, a summary of the key test results is provided. diff --git a/tools/report/report.py b/tools/report/report.py index 4264c055..1e0f87bc 100644 --- a/tools/report/report.py +++ b/tools/report/report.py @@ -27,7 +27,7 @@ from core.results.results_constants import ResultsConstants from conf import settings as S from tools import systeminfo -_TEMPLATE_FILE = 'report.jinja' +_TEMPLATE_FILES = ['report.jinja', 'report_rst.jinja'] _ROOT_DIR = os.path.normpath(os.path.dirname(os.path.realpath(__file__))) @@ -48,7 +48,7 @@ def _get_env(result): env = { 'os': systeminfo.get_os(), 'kernel': systeminfo.get_kernel(), - 'nic': systeminfo.get_nic(), + 'nics': systeminfo.get_nic(), 'cpu': systeminfo.get_cpu(), 'cpu_cores': systeminfo.get_cpu_cores(), 'memory' : systeminfo.get_memory(), @@ -78,11 +78,10 @@ def generate(input_file, tc_results, tc_stats): :returns: Path to generated report """ - output_file = '.'.join([os.path.splitext(input_file)[0], 'md']) - + output_files = [('.'.join([os.path.splitext(input_file)[0], 'md'])), + ('.'.join([os.path.splitext(input_file)[0], 'rst']))] template_loader = jinja2.FileSystemLoader(searchpath=_ROOT_DIR) template_env = jinja2.Environment(loader=template_loader) - template = template_env.get_template(_TEMPLATE_FILE) tests = [] try: @@ -112,17 +111,20 @@ def generate(input_file, tc_results, tc_stats): template_vars = { 'tests': tests, } - - output_text = template.render(template_vars) #pylint: disable=no-member - with open(output_file, 'w') as file_: - file_.write(output_text) - logging.info('Test report written to "%s"', output_file) + i = 0 + for output_file in output_files: + template = template_env.get_template(_TEMPLATE_FILES[i]) + output_text = template.render(template_vars) #pylint: disable=no-member + with open(output_file, 'w') as file_: + file_.write(output_text) + logging.info('Test report written to "%s"', output_file) + i += 1 except KeyError: logging.info("Report: Ignoring file (Wrongly defined columns): %s", (input_file)) raise - return output_file + return output_files if __name__ == '__main__': diff --git a/tools/report/report_rst.jinja b/tools/report/report_rst.jinja new file mode 100644 index 00000000..b3c2f2d0 --- /dev/null +++ b/tools/report/report_rst.jinja @@ -0,0 +1,175 @@ +========================================================================== +CHARACTERIZE VSWITCH PERFORMANCE FOR TELCO NFV USE CASES LEVEL TEST REPORT +========================================================================== + +Introduction +============ +The objective of the OPNFV project titled **"Characterise vSwitch Performance +for Telco NFV Use Cases"**, is to evaluate a virtual switch to identify its +suitability for a Telco Network Function Virtualization (NFV) environment. As +well as this, the project aims to identify any gaps or bottlenecks in order to +drive architectural changes to improve virtual switch performance and +determinism. The purpose of this document is to summarize the results of the +tests carried out on the virtual switch in the Network Function Virtualization +Infrastructure (NFVI) and, from these results, provide evaluations and +recommendations for the virtual switch. Test results will be outlined in +details-of-LTR_, preceded by the document-identifier_ and the scope_ and +references_). + +This document is currently in draft form. + +.. _document-identifier: + +Document identifier +------------------- +The document id will be used to uniquely identify versions of the LTR. The +format for the document id will be: +OPNFV\_vswitchperf\_LTR\_rel\_STATUS, the status is one of: DRAFT, REVIEWED, +CORRECTED or FINAL. The document id for this version of the LTR is: +OPNFV\_vswitchperf\_LTR\_Brahmaputra\_DRAFT. + +.. _scope: + +Scope +----- +The scope of this report is to detail the results of the tests that have been +performed on the virtual switch. This report will also evaluate the results of +these tests and, based on these evaluations, provide recommendations on the +suitability of the virtual switch for use in a Telco NFV environment. + +.. _references: + +References +---------- +`OPNFV_vswitchperf_LTD_Brahmaputra_REVIEWED +<http://artifacts.opnfv.org/vswitchperf/docs/requirements/vswitchperf_ltd.html>`__ + +.. details-of-LTR: + +Details of the Level Test Report +================================ +This section provides a test-results-overview_ as well as +detailed-test-results_. Also included are the rationale_ and the conclusions_. + +.. _test-results-overview: + +Overview of test results +------------------------ +Test Environment +~~~~~~~~~~~~~~~~ +Below is the environment that the test was performed in: + +* OS: {{tests[0].env.os}} +* Kernel Version: {{tests[0].env.kernel}} +* NIC(s):{% for nic in tests[0].env.nics %} + * {{nic}}{% endfor %} +* Board: {{tests[0].env.platform}} +* CPU: {{tests[0].env.cpu}} +* CPU cores: {{tests[0].env.cpu_cores}} +* Memory: {{tests[0].env.memory}} +* Virtual Switch Set-up: {{tests[0].deployment}} +* vswitchperf: GIT tag: {{tests[0].env.vsperf['git_tag']}} +* Traffic Generator: {{tests[0].env.traffic_gen['name']}}, Version: {{tests[0].env.traffic_gen['version']}}, GIT tag: {{tests[0].env.traffic_gen['git_tag']}} +* vSwitch: {{tests[0].env.vswitch['name']}}, Version: {{tests[0].env.vswitch['version']}}, GIT tag: {{tests[0].env.vswitch['git_tag']}} +* DPDK Version: {{tests[0].env.dpdk['version']}}, GIT tag: {{tests[0].env.dpdk['git_tag']}} +{%- if 'vnf' in tests[0].env %} +* VNF: {{tests[0].env.vnf['name']}}, Version: {{tests[0].env.vnf['version']}}, GIT tag: {{tests[0].env.vnf['git_tag']}} +* VM images:{% for guest_image in tests[0].env.guest_image %} + * {{guest_image}}{% endfor %} +* VM loopback apps:{% for loopback_app in tests[0].env.loopback_app %} + * {{loopback_app['name']}}, Version: {{loopback_app['version']}}, GIT tag: {{loopback_app['git_tag']}}{% endfor %} +{%- endif %} + +For each test, a summary of the key test results is provided. +{% for test in tests %} +Test ID: {{ test.ID }} +~~~~~~~~~~~~~~~~~~~~~~~ +Below are test details: + +* Test ID: {{ "%s"|format(test.id) }} +* Description: {{ "%s"|format(test.conf['Description']) }} +* Deployment: {{ "%s"|format(test.deployment) }} +* Traffic type: {{ "%s"|format(test.result['type']) }} +* Packet size: {{ "%s"|format(test.result['packet_size']) }} +* Bidirectional : {{ "%s"|format(test.conf['biDirectional']) }} +{% endfor %} + +.. _detailed-test-results: + +Detailed test results +====================== +A detailed summary of the main results for each test is outlined below. +{% for test in tests %} +Test ID: {{ test.ID }} +---------------------- +Results/Metrics Collected +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following are the metrics obtained during this test: + +========================== ================================== + Metric Result +========================== ================================== +{%- for item, value in test.result.items() %} +{{ "%-30s %30s"|format(item,value)}} +{%- endfor %} +========================== ================================== + +Statistics collected +~~~~~~~~~~~~~~~~~~~~ + +The following system statistics were collected during testcase execution: + +{% for process in test.stats %} + +========================== ================================== +Process: {{ '_'.join(process.split('_')[:-1]) }} +------------------------------------------------------------- + Statistic Value +========================== ================================== +{%- for item, value in test.stats[process].items() %} +{{ "%-30s %30s"|format(item,value)}} +{%- endfor %} +========================== ================================== + +{%- endfor %} + + +Anomalies +~~~~~~~~~~ +No anomalies were detected during the course of this test. + +Testing Activities/Events +~~~~~~~~~~~~~~~~~~~~~~~~~ +There were no significant testing activities for this test. +{% endfor %} + +.. _rationale: + +Rationale for decisions +======================= +TODO. + +.. _conclusions: + +Conclusions and recommendations +=============================== +TODO. + +General +======== + +Glossary +-------- +- NFV - Network Function Virtualization +- Mbps - 1,000,000bps + +Document change procedures and history +-------------------------------------- +=============================================== ================= ============= + Document ID Author Date Modified +=============================================== ================= ============= +`OPNFV_vswitchperf_LTR_ver_1.0_Jan_15_CN_DRAFT` Christopher Nolan 23/01/2015 +`OPNFV_vswitchperf_LTR_ver_1.1_Jan_15_CN_DRAFT` Christopher Nolan 28/01/2015 +=============================================== ================= ============= + diff --git a/tools/systeminfo.py b/tools/systeminfo.py index 901b334e..62db852b 100644 --- a/tools/systeminfo.py +++ b/tools/systeminfo.py @@ -65,7 +65,7 @@ def get_cpu(): def get_nic(): """Get NIC(s) information. - :returns: Return NIC(s) information as a string + :returns: Return NIC(s) information as a list """ nics = [] output = subprocess.check_output('lspci', shell=True) @@ -74,7 +74,7 @@ def get_nic(): for nic_pciid in S.getValue('WHITELIST_NICS'): if line.startswith(nic_pciid): nics.append(''.join(line.split(':')[2:]).strip()) - return ', '.join(nics).strip() + return nics def get_platform(): """Get platform information. |