From 91be1b903acf91669d98779d8c468091117d4fa5 Mon Sep 17 00:00:00 2001 From: Yujun Zhang Date: Thu, 23 Mar 2017 16:52:11 +0800 Subject: Rename docs/testing/developer/design to docs/testing/developer/devguide See https://git.opnfv.org/opnfvdocs/tree/docs/testing/testing-dev.rst Change-Id: Iac14adf4a2876fd74972dcf6279f7022e92cb9b5 Signed-off-by: Yujun Zhang --- docs/testing/developer/design/_assets/sequence.png | Bin 44132 -> 0 bytes .../developer/design/_assets/standalone.png | Bin 47515 -> 0 bytes docs/testing/developer/design/api.rst | 2 - docs/testing/developer/design/arch.rst | 24 ------ docs/testing/developer/design/cli.rst | 92 --------------------- docs/testing/developer/design/compute-qpi.rst | 69 ---------------- docs/testing/developer/design/index.rst | 16 ---- .../developer/devguide/_assets/sequence.png | Bin 0 -> 44132 bytes .../developer/devguide/_assets/standalone.png | Bin 0 -> 47515 bytes docs/testing/developer/devguide/api.rst | 2 + docs/testing/developer/devguide/arch.rst | 24 ++++++ docs/testing/developer/devguide/cli.rst | 92 +++++++++++++++++++++ docs/testing/developer/devguide/compute-qpi.rst | 69 ++++++++++++++++ docs/testing/developer/devguide/index.rst | 16 ++++ 14 files changed, 203 insertions(+), 203 deletions(-) delete mode 100644 docs/testing/developer/design/_assets/sequence.png delete mode 100644 docs/testing/developer/design/_assets/standalone.png delete mode 100644 docs/testing/developer/design/api.rst delete mode 100644 docs/testing/developer/design/arch.rst delete mode 100644 docs/testing/developer/design/cli.rst delete mode 100644 docs/testing/developer/design/compute-qpi.rst delete mode 100644 docs/testing/developer/design/index.rst create mode 100644 docs/testing/developer/devguide/_assets/sequence.png create mode 100644 docs/testing/developer/devguide/_assets/standalone.png create mode 100644 docs/testing/developer/devguide/api.rst create mode 100644 docs/testing/developer/devguide/arch.rst create mode 100644 docs/testing/developer/devguide/cli.rst create mode 100644 docs/testing/developer/devguide/compute-qpi.rst create mode 100644 docs/testing/developer/devguide/index.rst diff --git a/docs/testing/developer/design/_assets/sequence.png b/docs/testing/developer/design/_assets/sequence.png deleted file mode 100644 index b15458bf..00000000 Binary files a/docs/testing/developer/design/_assets/sequence.png and /dev/null differ diff --git a/docs/testing/developer/design/_assets/standalone.png b/docs/testing/developer/design/_assets/standalone.png deleted file mode 100644 index 50222b5b..00000000 Binary files a/docs/testing/developer/design/_assets/standalone.png and /dev/null differ diff --git a/docs/testing/developer/design/api.rst b/docs/testing/developer/design/api.rst deleted file mode 100644 index eb2b0d67..00000000 --- a/docs/testing/developer/design/api.rst +++ /dev/null @@ -1,2 +0,0 @@ -- Which framework has been used and why -- How to extend to more api diff --git a/docs/testing/developer/design/arch.rst b/docs/testing/developer/design/arch.rst deleted file mode 100644 index d95faba6..00000000 --- a/docs/testing/developer/design/arch.rst +++ /dev/null @@ -1,24 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) 2017 ZTE Corp. - - -######################## -QTIP Architecture Design -######################## - -In Danube, QTIP releases its standalone mode, which is also know as ``solo``: - -.. figure:: _assets/standalone.png - :alt: QTIP standalone mode - -The runner could be launched from CLI (command line interpreter) or API -(application programming interface) and drives the testing jobs. The generated -data including raw performance data and testing environment are fed to collector. -Performance metrics will be parsed from the raw data and used for QPI calculation. -Then the benchmark report is rendered with the benchmarking results. - -The execution can be detailed in the diagram below: - -.. figure:: _assets/sequence.png - :alt: QTIP execution sequence diff --git a/docs/testing/developer/design/cli.rst b/docs/testing/developer/design/cli.rst deleted file mode 100644 index 72d1fbaf..00000000 --- a/docs/testing/developer/design/cli.rst +++ /dev/null @@ -1,92 +0,0 @@ -*************************** -QTIP Command Line Interface -*************************** - -Abstract -######## - -QTIP consists of different tools(metrics) to benchmark the NFVI. These metrics fall under different NFVI -subsystems(QPI's) such as compute, storage and network. A plan consists of one or more QPI's, depending upon how -the end user would want to measure performance. CLI is designed to help the user, execute benchmarks and -view respective scores. - -Framework -========= - -QTIP CLI has been created using the Python package `Click`_, Command Line Interface Creation Kit. It has been -chosen for number of reasons. It presents the user with a very simple yet powerful API to build complex -applications. One of the most striking features is command nesting. - -As explained, QTIP consists of metrics, QPI's and plans. CLI is designed to provide interface to all -these components. It is responsible for execution, as well as provide listing and details of each individual -element making up these components. - -Design -====== - -CLI's entry point extends Click's built in MultiCommand class object. It provides two methods, which are -overridden to provide custom configurations. - -.. code-block:: python - - class QtipCli(click.MultiCommand): - - def list_commands(self, ctx): - rv = [] - for filename in os.listdir(cmd_folder): - if filename.endswith('.py') and \ - filename.startswith('cmd_'): - rv.append(filename[4:-3]) - rv.sort() - return rv - - def get_command(self, ctx, name): - try: - if sys.version_info[0] == 2: - name = name.encode('ascii', 'replace') - mod = __import__('qtip.cli.commands.cmd_' + name, - None, None, ['cli']) - except ImportError: - return - return mod.cli - -Commands and subcommands will then be loaded by the ``get_command`` method above. - -Extending the Framework -======================= - -Framework can be easily extended, as per the users requirements. One such example can be to override the builtin -configurations with user defined ones. These can be written in a file, loaded via a Click Context and passed -through to all the commands. - -.. code-block:: python - - class Context: - - def __init__(): - - self.config = ConfigParser.ConfigParser() - self.config.read('path/to/configuration_file') - - def get_paths(): - - paths = self.config.get('section', 'path') - return paths - -The above example loads configuration from user defined paths, which then need to be provided to the actual -command definitions. - -.. code-block:: python - - from qtip.cli.entry import Context - - pass_context = click.make_pass_decorator(Context, ensure=False) - - @cli.command('list', help='List the Plans') - @pass_context - def list(ctx): - plans = Plan.list_all(ctx.paths()) - table = utils.table('Plans', plans) - click.echo(table) - -.. _Click: http://click.pocoo.org/5/ diff --git a/docs/testing/developer/design/compute-qpi.rst b/docs/testing/developer/design/compute-qpi.rst deleted file mode 100644 index 2e5aa87c..00000000 --- a/docs/testing/developer/design/compute-qpi.rst +++ /dev/null @@ -1,69 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) 2016 ZTE Corp. - - -*********** -Compute QPI -*********** - -The compute QPI gives user an overall score for system compute performace. - -Summary -======= - -The compute QPI are calibrated a ZTE `E9000 `_ server as a baseline with score of 2500 points. -Higher scores are better, with double the score indicating double the performance. -The compute QPI provides three different kinds of scores: - -* Workload Scores -* Section Scores -* Compute QPI Scores - -Baseline -======== - -ZTE E9000 server with an 2 Deca core Intel Xeon CPU processor,128560.0MB Memory. - -Workload Scores -=============== - -Each time a workload is executed QTIP calculates a score based on the computer's performance -compared to the baseline performance. - -Section Scores -============== - -QTIP uses a number of different tests, or workloads, to measure performance. -The workloads are divided into five different sections: - -+-----------------+--------------------------------------------------------------+------------------------------------------+ -| Section | Detail | Indication | -+=================+==============================================================+==========================================+ -| Integer | Integer workloads measure the integer instruction performace | All app relies on integer | -| | of host or vm by performing Dhrystone test. | performance | -+-----------------+--------------------------------------------------------------+------------------------------------------+ -| Floating point | Floating point workloads measure the floating pointperfo | Floating point performance is especially | -| | rmance by performing Whetstone test. | important in video games,digital content | -| | | creation applications. | -+-----------------+--------------------------------------------------------------+------------------------------------------+ -| Memory | Memory workloads measure memory bandwidth by performing | Software working with cipher large | -| | RamSpeed test. | amounts data relies on SSL Performace. | -+-----------------+--------------------------------------------------------------+------------------------------------------+ -| DPI | DPI workloads measure deep-packet inspection speed by | Software working with network packet | -| | performing nDPI test. | anlysis relies on DPI performance. | -+-----------------+--------------------------------------------------------------+------------------------------------------+ -| SSL | SSL Performance workloads measure cipher speeds by | Software working with cipher large | -| | using the OpenSSL tool. | amounts data relies on SSL Performace | -+-----------------+--------------------------------------------------------------+------------------------------------------+ - -A section score is the `geometric mean `_ of all the workload scores for workloads -that are part of the section. These scores are useful for determining the performance of -the computer in a particular area. - -Compute QPI Scores -================== - -The compute QPI score is the `weighted arithmetic mean `_ of the five section scores. -The compute QPI score provides a way to quickly compare performance across different -computers and different platforms without getting bogged down in details. diff --git a/docs/testing/developer/design/index.rst b/docs/testing/developer/design/index.rst deleted file mode 100644 index 2b4bd9b0..00000000 --- a/docs/testing/developer/design/index.rst +++ /dev/null @@ -1,16 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) 2016 ZTE Corp. - - -########################## -QTIP Design Specifications -########################## - -.. toctree:: - :maxdepth: 2 - - arch.rst - cli.rst - api.rst - compute-qpi.rst diff --git a/docs/testing/developer/devguide/_assets/sequence.png b/docs/testing/developer/devguide/_assets/sequence.png new file mode 100644 index 00000000..b15458bf Binary files /dev/null and b/docs/testing/developer/devguide/_assets/sequence.png differ diff --git a/docs/testing/developer/devguide/_assets/standalone.png b/docs/testing/developer/devguide/_assets/standalone.png new file mode 100644 index 00000000..50222b5b Binary files /dev/null and b/docs/testing/developer/devguide/_assets/standalone.png differ diff --git a/docs/testing/developer/devguide/api.rst b/docs/testing/developer/devguide/api.rst new file mode 100644 index 00000000..eb2b0d67 --- /dev/null +++ b/docs/testing/developer/devguide/api.rst @@ -0,0 +1,2 @@ +- Which framework has been used and why +- How to extend to more api diff --git a/docs/testing/developer/devguide/arch.rst b/docs/testing/developer/devguide/arch.rst new file mode 100644 index 00000000..d95faba6 --- /dev/null +++ b/docs/testing/developer/devguide/arch.rst @@ -0,0 +1,24 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2017 ZTE Corp. + + +######################## +QTIP Architecture Design +######################## + +In Danube, QTIP releases its standalone mode, which is also know as ``solo``: + +.. figure:: _assets/standalone.png + :alt: QTIP standalone mode + +The runner could be launched from CLI (command line interpreter) or API +(application programming interface) and drives the testing jobs. The generated +data including raw performance data and testing environment are fed to collector. +Performance metrics will be parsed from the raw data and used for QPI calculation. +Then the benchmark report is rendered with the benchmarking results. + +The execution can be detailed in the diagram below: + +.. figure:: _assets/sequence.png + :alt: QTIP execution sequence diff --git a/docs/testing/developer/devguide/cli.rst b/docs/testing/developer/devguide/cli.rst new file mode 100644 index 00000000..72d1fbaf --- /dev/null +++ b/docs/testing/developer/devguide/cli.rst @@ -0,0 +1,92 @@ +*************************** +QTIP Command Line Interface +*************************** + +Abstract +######## + +QTIP consists of different tools(metrics) to benchmark the NFVI. These metrics fall under different NFVI +subsystems(QPI's) such as compute, storage and network. A plan consists of one or more QPI's, depending upon how +the end user would want to measure performance. CLI is designed to help the user, execute benchmarks and +view respective scores. + +Framework +========= + +QTIP CLI has been created using the Python package `Click`_, Command Line Interface Creation Kit. It has been +chosen for number of reasons. It presents the user with a very simple yet powerful API to build complex +applications. One of the most striking features is command nesting. + +As explained, QTIP consists of metrics, QPI's and plans. CLI is designed to provide interface to all +these components. It is responsible for execution, as well as provide listing and details of each individual +element making up these components. + +Design +====== + +CLI's entry point extends Click's built in MultiCommand class object. It provides two methods, which are +overridden to provide custom configurations. + +.. code-block:: python + + class QtipCli(click.MultiCommand): + + def list_commands(self, ctx): + rv = [] + for filename in os.listdir(cmd_folder): + if filename.endswith('.py') and \ + filename.startswith('cmd_'): + rv.append(filename[4:-3]) + rv.sort() + return rv + + def get_command(self, ctx, name): + try: + if sys.version_info[0] == 2: + name = name.encode('ascii', 'replace') + mod = __import__('qtip.cli.commands.cmd_' + name, + None, None, ['cli']) + except ImportError: + return + return mod.cli + +Commands and subcommands will then be loaded by the ``get_command`` method above. + +Extending the Framework +======================= + +Framework can be easily extended, as per the users requirements. One such example can be to override the builtin +configurations with user defined ones. These can be written in a file, loaded via a Click Context and passed +through to all the commands. + +.. code-block:: python + + class Context: + + def __init__(): + + self.config = ConfigParser.ConfigParser() + self.config.read('path/to/configuration_file') + + def get_paths(): + + paths = self.config.get('section', 'path') + return paths + +The above example loads configuration from user defined paths, which then need to be provided to the actual +command definitions. + +.. code-block:: python + + from qtip.cli.entry import Context + + pass_context = click.make_pass_decorator(Context, ensure=False) + + @cli.command('list', help='List the Plans') + @pass_context + def list(ctx): + plans = Plan.list_all(ctx.paths()) + table = utils.table('Plans', plans) + click.echo(table) + +.. _Click: http://click.pocoo.org/5/ diff --git a/docs/testing/developer/devguide/compute-qpi.rst b/docs/testing/developer/devguide/compute-qpi.rst new file mode 100644 index 00000000..2e5aa87c --- /dev/null +++ b/docs/testing/developer/devguide/compute-qpi.rst @@ -0,0 +1,69 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 ZTE Corp. + + +*********** +Compute QPI +*********** + +The compute QPI gives user an overall score for system compute performace. + +Summary +======= + +The compute QPI are calibrated a ZTE `E9000 `_ server as a baseline with score of 2500 points. +Higher scores are better, with double the score indicating double the performance. +The compute QPI provides three different kinds of scores: + +* Workload Scores +* Section Scores +* Compute QPI Scores + +Baseline +======== + +ZTE E9000 server with an 2 Deca core Intel Xeon CPU processor,128560.0MB Memory. + +Workload Scores +=============== + +Each time a workload is executed QTIP calculates a score based on the computer's performance +compared to the baseline performance. + +Section Scores +============== + +QTIP uses a number of different tests, or workloads, to measure performance. +The workloads are divided into five different sections: + ++-----------------+--------------------------------------------------------------+------------------------------------------+ +| Section | Detail | Indication | ++=================+==============================================================+==========================================+ +| Integer | Integer workloads measure the integer instruction performace | All app relies on integer | +| | of host or vm by performing Dhrystone test. | performance | ++-----------------+--------------------------------------------------------------+------------------------------------------+ +| Floating point | Floating point workloads measure the floating pointperfo | Floating point performance is especially | +| | rmance by performing Whetstone test. | important in video games,digital content | +| | | creation applications. | ++-----------------+--------------------------------------------------------------+------------------------------------------+ +| Memory | Memory workloads measure memory bandwidth by performing | Software working with cipher large | +| | RamSpeed test. | amounts data relies on SSL Performace. | ++-----------------+--------------------------------------------------------------+------------------------------------------+ +| DPI | DPI workloads measure deep-packet inspection speed by | Software working with network packet | +| | performing nDPI test. | anlysis relies on DPI performance. | ++-----------------+--------------------------------------------------------------+------------------------------------------+ +| SSL | SSL Performance workloads measure cipher speeds by | Software working with cipher large | +| | using the OpenSSL tool. | amounts data relies on SSL Performace | ++-----------------+--------------------------------------------------------------+------------------------------------------+ + +A section score is the `geometric mean `_ of all the workload scores for workloads +that are part of the section. These scores are useful for determining the performance of +the computer in a particular area. + +Compute QPI Scores +================== + +The compute QPI score is the `weighted arithmetic mean `_ of the five section scores. +The compute QPI score provides a way to quickly compare performance across different +computers and different platforms without getting bogged down in details. diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst new file mode 100644 index 00000000..2b4bd9b0 --- /dev/null +++ b/docs/testing/developer/devguide/index.rst @@ -0,0 +1,16 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 ZTE Corp. + + +########################## +QTIP Design Specifications +########################## + +.. toctree:: + :maxdepth: 2 + + arch.rst + cli.rst + api.rst + compute-qpi.rst -- cgit 1.2.3-korg