aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--docs/userguide/10-grafana.rst69
-rw-r--r--tests/opnfv/test_suites/opnfv_os-odl_l2-fdio-ha_daily.yaml30
-rw-r--r--tests/unit/benchmark/scenarios/networking/test_networkcapacity.py112
-rw-r--r--yardstick/benchmark/scenarios/networking/networkcapacity.py142
4 files changed, 208 insertions, 145 deletions
diff --git a/docs/userguide/10-grafana.rst b/docs/userguide/10-grafana.rst
index e871a18da..416857b71 100644
--- a/docs/userguide/10-grafana.rst
+++ b/docs/userguide/10-grafana.rst
@@ -7,6 +7,7 @@
Grafana dashboard
=================
+
Abstract
========
@@ -18,12 +19,14 @@ dashboard can be found here: http://testresults.opnfv.org/grafana/
:width: 800px
:alt: Yardstick grafana dashboard
+
Public access
=============
Yardstick provids a public account for accessing to the dashboard. The username
and password are both set to ‘opnfv’.
+
Testcase dashboard
==================
@@ -39,15 +42,35 @@ For each test case dashboard. On the top left, we have a dashboard selection,
you can switch to different test cases using this pull-down menu.
Underneath, we have a pod and scenario selection.
-All the pods and scenarios that have ever published test data to the Influx DB
+All the pods and scenarios that have ever published test data to the InfluxDB
will be shown here.
You can check multiple pods or scenarios.
-For each test case, we have a short description and a link to detailed test case information in Yardstick user guide.
+For each test case, we have a short description and a link to detailed test
+case information in Yardstick user guide.
Underneath, it is the result presentation section.
-You can use the time period selection on the top right corner to zoom in or zoom out the chart.
+You can use the time period selection on the top right corner to zoom in or
+zoom out the chart.
+
+
+Administration access
+=====================
+
+For a user with administration rights it is easy to update and save any
+dashboard configuration. Saved updates immediately take effect and become live.
+This may cause issues like:
+
+- Changes and updates made to the live configuration in Grafana can compromise
+ existing Grafana content in an unwanted, unpredicted or incompatible way.
+ Grafana as such is not version controlled, there exists one single Grafana
+ configuration per dashboard.
+- There is a risk several people can disturb each other when doing updates to
+ the same Grafana dashboard at the same time.
+
+Any change made by administrator should be careful.
+
Add a dashboard into yardstick grafana
======================================
@@ -64,23 +87,33 @@ There are 6 steps to go.
:alt: Add a dashboard into yardstick grafana
-First, You need to build a local influxdb and grafana, so you can do the work
-locally. You can refer to How to deploy InfluxDB and Grafana locally wiki page
-about how to do this.
+1. You need to build a local influxdb and grafana, so you can do the work
+ locally. You can refer to How to deploy InfluxDB and Grafana locally wiki
+ page about how to do this.
+
+2. Once step one is done, you can fetch the existing grafana dashboard
+ configuration file from the yardstick repository and import it to your local
+ grafana. After import is done, you grafana dashboard will be ready to use
+ just like the community’s dashboard.
-Once step one is done, you can fetch the existing grafana dashboard
-configuration file from the yardstick repository and import it to your local
-grafana. After import is done, you grafana dashboard will be ready to use just
-like the community’s dashboard.
+3. The third step is running some test cases to generate test results and
+ publishing it to your local influxdb.
-The third step is running some test cases to generate test results and
-publishing it to your local influxdb.
+4. Now you have some data to visualize in your dashboard. In the fourth step,
+ it is time to create your own dashboard. You can either modify an existing
+ dashboard or try to create a new one from scratch. If you choose to modify
+ an existing dashboard then in the curtain menu of the existing dashboard do
+ a "Save As..." into a new dashboard copy instance, and then continue doing
+ all updates and saves within the dashboard copy.
-Now you have some data to visualize in your dashboard. In the fourth step, it
-is time to create your own dashboard. You can either modify an existing
-dashboard or try to create a new one from scratch.
+5. When finished with all Grafana configuration changes in this temporary
+ dashboard then chose "export" of the updated dashboard copy into a JSON file
+ and put it up for review in Gerrit, in file /yardstick/dashboard/Yardstick-TCxxx-yyyyyyyyyyyyy.
+ For instance a typical default name of the file would be "Yardstick-TC001 Copy-1234567891234".
-Either way, once you finish the dashboard, the next step is exporting the
-configuration file and propose a patch into Yardstick. Yardstick team will
-review and merge it into Yardstick repository.
+6. Once you finish your dashboard, the next step is exporting the configuration
+ file and propose a patch into Yardstick. Yardstick team will review and
+ merge it into Yardstick repository. After approved review Yardstick team
+ will do an "import" of the JSON file and also a "save dashboard" as soon as
+ possible to replace the old live dashboard configuration.
diff --git a/tests/opnfv/test_suites/opnfv_os-odl_l2-fdio-ha_daily.yaml b/tests/opnfv/test_suites/opnfv_os-odl_l2-fdio-ha_daily.yaml
new file mode 100644
index 000000000..2d7397a96
--- /dev/null
+++ b/tests/opnfv/test_suites/opnfv_os-odl_l2-fdio-ha_daily.yaml
@@ -0,0 +1,30 @@
+---
+# os-odl_l2-fdio-ha daily task suite
+
+schema: "yardstick:suite:0.1"
+
+name: "os-odl_l2-fdio-ha"
+test_cases_dir: "tests/opnfv/test_cases/"
+test_cases:
+-
+ file_name: opnfv_yardstick_tc002.yaml
+-
+ file_name: opnfv_yardstick_tc005.yaml
+-
+ file_name: opnfv_yardstick_tc010.yaml
+-
+ file_name: opnfv_yardstick_tc011.yaml
+-
+ file_name: opnfv_yardstick_tc012.yaml
+-
+ file_name: opnfv_yardstick_tc014.yaml
+-
+ file_name: opnfv_yardstick_tc037.yaml
+-
+ file_name: opnfv_yardstick_tc069.yaml
+-
+ file_name: opnfv_yardstick_tc070.yaml
+-
+ file_name: opnfv_yardstick_tc071.yaml
+-
+ file_name: opnfv_yardstick_tc072.yaml
diff --git a/tests/unit/benchmark/scenarios/networking/test_networkcapacity.py b/tests/unit/benchmark/scenarios/networking/test_networkcapacity.py
index e3a096446..e42832f1b 100644
--- a/tests/unit/benchmark/scenarios/networking/test_networkcapacity.py
+++ b/tests/unit/benchmark/scenarios/networking/test_networkcapacity.py
@@ -1,56 +1,56 @@
-#!/usr/bin/env python
-
-##############################################################################
-# Copyright (c) 2016 Huawei Technologies Co.,Ltd and others.
-#
-# All rights reserved. This program and the accompanying materials
-# are made available under the terms of the Apache License, Version 2.0
-# which accompanies this distribution, and is available at
-# http://www.apache.org/licenses/LICENSE-2.0
-##############################################################################
-
-# Unittest for yardstick.benchmark.scenarios.networking.networkcapacity.NetworkCapacity
-
-import mock
-import unittest
-import os
-import json
-
-from yardstick.benchmark.scenarios.networking import networkcapacity
-
-SAMPLE_OUTPUT = '{"Number of connections":"308","Number of frames received": "166503"}'
-
-@mock.patch('yardstick.benchmark.scenarios.networking.networkcapacity.ssh')
-class NetworkCapacityTestCase(unittest.TestCase):
-
- def setUp(self):
- self.ctx = {
- 'host': {
- 'ip': '172.16.0.137',
- 'user': 'cirros',
- 'password': "root"
- },
- }
-
- self.result = {}
-
- def test_capacity_successful_setup(self, mock_ssh):
- c = networkcapacity.NetworkCapacity({}, self.ctx)
- mock_ssh.SSH().execute.return_value = (0, '', '')
- c.setup()
- self.assertIsNotNone(c.client)
- self.assertTrue(c.setup_done)
-
- def test_capacity_successful(self, mock_ssh):
- c = networkcapacity.NetworkCapacity({}, self.ctx)
-
- mock_ssh.SSH().execute.return_value = (0, SAMPLE_OUTPUT, '')
- c.run(self.result)
- expected_result = json.loads(SAMPLE_OUTPUT)
- self.assertEqual(self.result, expected_result)
-
- def test_capacity_unsuccessful_script_error(self, mock_ssh):
- c = networkcapacity.NetworkCapacity({}, self.ctx)
-
- mock_ssh.SSH().execute.return_value = (1, '', 'FOOBAR')
- self.assertRaises(RuntimeError, c.run, self.result)
+#!/usr/bin/env python
+
+##############################################################################
+# Copyright (c) 2016 Huawei Technologies Co.,Ltd and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+# Unittest for yardstick.benchmark.scenarios.networking.networkcapacity.NetworkCapacity
+
+import mock
+import unittest
+import os
+import json
+
+from yardstick.benchmark.scenarios.networking import networkcapacity
+
+SAMPLE_OUTPUT = '{"Number of connections":"308","Number of frames received": "166503"}'
+
+@mock.patch('yardstick.benchmark.scenarios.networking.networkcapacity.ssh')
+class NetworkCapacityTestCase(unittest.TestCase):
+
+ def setUp(self):
+ self.ctx = {
+ 'host': {
+ 'ip': '172.16.0.137',
+ 'user': 'cirros',
+ 'password': "root"
+ },
+ }
+
+ self.result = {}
+
+ def test_capacity_successful_setup(self, mock_ssh):
+ c = networkcapacity.NetworkCapacity({}, self.ctx)
+ mock_ssh.SSH().execute.return_value = (0, '', '')
+ c.setup()
+ self.assertIsNotNone(c.client)
+ self.assertTrue(c.setup_done)
+
+ def test_capacity_successful(self, mock_ssh):
+ c = networkcapacity.NetworkCapacity({}, self.ctx)
+
+ mock_ssh.SSH().execute.return_value = (0, SAMPLE_OUTPUT, '')
+ c.run(self.result)
+ expected_result = json.loads(SAMPLE_OUTPUT)
+ self.assertEqual(self.result, expected_result)
+
+ def test_capacity_unsuccessful_script_error(self, mock_ssh):
+ c = networkcapacity.NetworkCapacity({}, self.ctx)
+
+ mock_ssh.SSH().execute.return_value = (1, '', 'FOOBAR')
+ self.assertRaises(RuntimeError, c.run, self.result)
diff --git a/yardstick/benchmark/scenarios/networking/networkcapacity.py b/yardstick/benchmark/scenarios/networking/networkcapacity.py
index fed2fbd65..438452e40 100644
--- a/yardstick/benchmark/scenarios/networking/networkcapacity.py
+++ b/yardstick/benchmark/scenarios/networking/networkcapacity.py
@@ -1,71 +1,71 @@
-##############################################################################
-# Copyright (c) 2016 Huawei Technologies Co.,Ltd and others.
-#
-# All rights reserved. This program and the accompanying materials
-# are made available under the terms of the Apache License, Version 2.0
-# which accompanies this distribution, and is available at
-# http://www.apache.org/licenses/LICENSE-2.0
-##############################################################################
-import pkg_resources
-import logging
-import json
-
-import yardstick.ssh as ssh
-from yardstick.benchmark.scenarios import base
-
-LOG = logging.getLogger(__name__)
-
-
-class NetworkCapacity(base.Scenario):
- """Measure Network capacity and scale.
-
- This scenario reads network status including number of connections,
- number of frames sent/received.
- """
- __scenario_type__ = "NetworkCapacity"
- TARGET_SCRIPT = "networkcapacity.bash"
-
- def __init__(self, scenario_cfg, context_cfg):
- self.scenario_cfg = scenario_cfg
- self.context_cfg = context_cfg
- self.setup_done = False
-
- def setup(self):
- """scenario setup"""
- self.target_script = pkg_resources.resource_filename(
- "yardstick.benchmark.scenarios.networking",
- NetworkCapacity.TARGET_SCRIPT)
-
- host = self.context_cfg['host']
- if host is None:
- raise RuntimeError('No right node.please check the configuration')
- host_user = host.get('user', 'ubuntu')
- ssh_port = host.get("ssh_port", ssh.DEFAULT_PORT)
- host_ip = host.get('ip', None)
- host_pwd = host.get('password', None)
-
- LOG.debug("user:%s, host:%s", host_user, host_ip)
- self.client = ssh.SSH(host_user, host_ip, password=host_pwd,
- port=ssh_port)
- self.client.wait(timeout=600)
-
- # copy script to host
- self.client.run("cat > ~/networkcapacity.sh",
- stdin=open(self.target_script, 'rb'))
-
- self.setup_done = True
-
- def run(self, result):
- """execute the benchmark"""
-
- if not self.setup_done:
- self.setup()
-
- cmd = "sudo bash networkcapacity.sh"
-
- LOG.debug("Executing command: %s", cmd)
- status, stdout, stderr = self.client.execute(cmd)
- if status:
- raise RuntimeError(stderr)
-
- result.update(json.loads(stdout))
+##############################################################################
+# Copyright (c) 2016 Huawei Technologies Co.,Ltd and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+import pkg_resources
+import logging
+import json
+
+import yardstick.ssh as ssh
+from yardstick.benchmark.scenarios import base
+
+LOG = logging.getLogger(__name__)
+
+
+class NetworkCapacity(base.Scenario):
+ """Measure Network capacity and scale.
+
+ This scenario reads network status including number of connections,
+ number of frames sent/received.
+ """
+ __scenario_type__ = "NetworkCapacity"
+ TARGET_SCRIPT = "networkcapacity.bash"
+
+ def __init__(self, scenario_cfg, context_cfg):
+ self.scenario_cfg = scenario_cfg
+ self.context_cfg = context_cfg
+ self.setup_done = False
+
+ def setup(self):
+ """scenario setup"""
+ self.target_script = pkg_resources.resource_filename(
+ "yardstick.benchmark.scenarios.networking",
+ NetworkCapacity.TARGET_SCRIPT)
+
+ host = self.context_cfg['host']
+ if host is None:
+ raise RuntimeError('No right node.please check the configuration')
+ host_user = host.get('user', 'ubuntu')
+ ssh_port = host.get("ssh_port", ssh.DEFAULT_PORT)
+ host_ip = host.get('ip', None)
+ host_pwd = host.get('password', None)
+
+ LOG.debug("user:%s, host:%s", host_user, host_ip)
+ self.client = ssh.SSH(host_user, host_ip, password=host_pwd,
+ port=ssh_port)
+ self.client.wait(timeout=600)
+
+ # copy script to host
+ self.client.run("cat > ~/networkcapacity.sh",
+ stdin=open(self.target_script, 'rb'))
+
+ self.setup_done = True
+
+ def run(self, result):
+ """execute the benchmark"""
+
+ if not self.setup_done:
+ self.setup()
+
+ cmd = "sudo bash networkcapacity.sh"
+
+ LOG.debug("Executing command: %s", cmd)
+ status, stdout, stderr = self.client.execute(cmd)
+ if status:
+ raise RuntimeError(stderr)
+
+ result.update(json.loads(stdout))