diff options
| -rw-r--r-- | docs/proposal/storage-qpi.rst | 70 | ||||
| -rw-r--r-- | resources/QPI/compute.yaml | 97 | ||||
| -rw-r--r-- | resources/ansible_roles/inxi/tasks/main.yml | 2 | ||||
| -rw-r--r-- | resources/ansible_roles/opnfv-testapi/tasks/report.yml | 2 |
4 files changed, 139 insertions, 32 deletions
diff --git a/docs/proposal/storage-qpi.rst b/docs/proposal/storage-qpi.rst new file mode 100644 index 00000000..af7c7954 --- /dev/null +++ b/docs/proposal/storage-qpi.rst @@ -0,0 +1,70 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2017 ZTE Corp. + + +*********** +Storage QPI +*********** + +The storage QPI gives user an overall score for storage performance. + +The measurement is done by `StorPerf`_. + +.. _StorPerf: https://wiki.opnfv.org/display/storperf + +Baseline +======== + +Baseline is established by testing with a set of work loads: + +- `Queue depth`_ (1, 2, 8) +- `Block size`_ (2KB, 8KB, 16KB) +- `Read write`_ + - sequential read + - sequential write + - random read + - random write + - random mixed read write 70/30 + +.. _Queue depth: http://fio.readthedocs.io/en/latest/fio_man.html#cmdoption-arg-iodepth +.. _Block size: http://fio.readthedocs.io/en/latest/fio_man.html#cmdoption-arg-blocksize +.. _Read write: http://fio.readthedocs.io/en/latest/fio_man.html#cmdoption-arg-readwrite + +Metrics +======= + +- Throughput: data transfer rate +- IOPS: I/O operations per second +- Latency: response time + +Workload Scores +=============== + +For each test run, if an equivalent work load in baseline is available, a score will be calculated by comparing the +result to baseline. + +Section Scores +============== + ++-----------------+--------------------------------------------------------+-----------------------------------------+ +| Section | Detail | Indication | ++=================+========================================================+=========================================+ +| IOPS | Read write I/O Operation per second under steady state | Important for frequent storage access | +| | Workloads : random read/write | such as event sinks | ++-----------------+--------------------------------------------------------+-----------------------------------------+ +| Throughput | Read write data transfer rate under steady state | Important for high throughput services | +| | Workloads: sequential read/write, block size 16KB | such as video server | ++-----------------+--------------------------------------------------------+-----------------------------------------+ +| Latency | Average response latency under steady state | Important for real time applications | +| | Workloads: all | | ++-----------------+--------------------------------------------------------+-----------------------------------------+ + +Section score is the `geometric mean <https://en.wikipedia.org/wiki/Geometric_mean>`_ of all +workload score. + +Storage QPI +=========== + +Storage QPI is the `weighted arithmetic mean <https://en.wikipedia.org/wiki/Weighted_arithmetic_mean>`_ of all section +scores. diff --git a/resources/QPI/compute.yaml b/resources/QPI/compute.yaml index 05620269..8169da4f 100644 --- a/resources/QPI/compute.yaml +++ b/resources/QPI/compute.yaml @@ -28,75 +28,112 @@ sections: # split based on different application formula: geometric mean metrics: - name: ssl_rsa + description: performance of cryptographic using RSA cipher algorithm formual: geometric mean workloads: - name: rsa_sign_512 - baseline: 14982.3 + description: rsa 512 bits sign per second - name: rsa_verify_512 - baseline: 180619.2 + description: rsa 512 bits verify per second - name: rsa_sign_1024 - baseline: 5037.7 + description: rsa 1024 bits sign per second - name: rsa_verify_1024 - baseline: 67359.9 + description: rsa 1024 bits verify per second - name: rsa_sign_2048 - baseline: 713.6 + description: rsa 2048 bits sign per second - name: rsa_verify_2048 - baseline: 23458.0 + description: rsa 2048 bits verify per second - name: rsa_sign_4096 - baseline: 102.1 + description: rsa 4096 bits sign per second - name: rsa_verify_4096 - baseline: 6402.9 + description: rsa 4096 bits verify per second - name: ssl_aes + description: > + performance of advanced encryption standard (AES) cipher algorithm in cipher block chaining (CBC) mode formual: geometric mean workloads: - name: aes_128_cbc_16_bytes - baseline: 612376.96k + description: aes 128 bits key cbc on 16 bytes blocks - name: aes_128_cbc_64_bytes - baseline: 657350.74k + description: aes 128 bits key cbc on 64 bytes blocks - name: aes_128_cbc_256_bytes - baseline: 669680.04k + description: aes 128 bits key cbc on 256 bytes blocks - name: aes_128_cbc_1024_bytes - baseline: 672675.50k + description: aes 128 bits key cbc on 1024 bytes blocks - name: aes_128_cbc_8192_bytes - baseline: 672344.75k + description: aes 128 bits key cbc on 8192 bytes blocks - name: DPI description: deep packet inspection metrics: - name: dpi_throughput + description: deep packet inspection throughput workloads: - name: dpi_pps - baseline: 2.19M + description: DPI packets per second - name: dpi_bps - baseline: 20.55G + description: DPI bits per second - name: memory description: cache and memory performance metrics: - name: floatmem + description: > + measures the maximum possible cache and memory performance while reading and writing certain blocks of data + (starting from 1Kb and further in power of 2) continuously through FPU workloads: - - name: triad - baseline: 9748.98 - - name: add - baseline: 9734.18 - name: copy - baseline: 7692.26 + description: > + transfers data from one memory location to another, + i. e. copies it (A = B) - name: scale - baseline: 7616.35 + description: > + modifies the data before writing by multiplying with a certain + constant value, i. e. scales it (A = m*B). + - name: add + descrption: > + reads data from the first memory location, then reads from the + second, adds them up and writes the result to the third place + (A = B + C). + - name: triad + description: > + reads data from the first memory location, scales it, then adds + data from the second one and writes to the third place + (A = m*B + C). - name: intmem + description: > + measures the maximum possible cache and memory performance while reading and writing certain blocks of data + (starting from 1Kb and further in power of 2) continuously through ALU workloads: - - name: triad - baseline: 12110.50 - - name: add - baseline: 12153.10 - name: copy - baseline: 11913.03 + description: > + transfers data from one memory location to another, + i. e. copies it (A = B) - name: scale - baseline: 11945.36 + description: > + modifies the data before writing by multiplying with a + certain constant value, i. e. scales it (A = m*B). + - name: add + descrption: > + reads data from the first memory location, then reads from the + second, adds them up and writes the result to the third place + (A = B + C). + - name: triad + description: > + reads data from the first memory location, scales it, then adds + data from the second one and writes to the third place + (A = m*B + C). - name: arithmetic description: arithmetic computing speed metrics: - - name: arithmetic + - name: integer + description: > + measures and compare the performance of computers, focusing on string handling without floating point + operations. workloads: - name: dhrystone_lps - baseline: 34657490.5 + description: dhrystone loops per second + - name: floating + description: > + measures the speed and efficiency of floating-point operations + workloads: - name: whetstone_MWIPS - baseline: 2495.9 + description: whetstone million instructions per second diff --git a/resources/ansible_roles/inxi/tasks/main.yml b/resources/ansible_roles/inxi/tasks/main.yml index 1050c9be..c1f0b28c 100644 --- a/resources/ansible_roles/inxi/tasks/main.yml +++ b/resources/ansible_roles/inxi/tasks/main.yml @@ -42,7 +42,7 @@ patterns: - '.+\s+Host:\s+(?P<hostname>.+)\sKernel' - '.+\sMemory:\s+(?P<memory>.+MB)\s' - - '^CPU\(s\):\s+(?P<cpu>.+)\sspeed\/max' + - '^CPU\(s\):\s+(?P<cpu>.+)\sspeed' - '.+\sDistro:\s+(?P<os>.+)' - '.+\sKernel:\s+(?P<kernel>.+)\sConsole' - '.+\s+HDD Total Size:\s+(?P<disk>.+)\s' diff --git a/resources/ansible_roles/opnfv-testapi/tasks/report.yml b/resources/ansible_roles/opnfv-testapi/tasks/report.yml index 1c3d5767..bb034d84 100644 --- a/resources/ansible_roles/opnfv-testapi/tasks/report.yml +++ b/resources/ansible_roles/opnfv-testapi/tasks/report.yml @@ -21,7 +21,7 @@ project_name: "{{ project_name }}" case_name: "{{ case_name }}" pod_name: "{{ pod_name }}" - installer: "{{ installer }}" + installer: "{{ installer_type }}" version: "{{ version }}" scenario: "{{ scenario }}" start_date: "{{ ansible_date_time.date }}" |