diff options
| -rw-r--r-- | docker/Dockerfile.aarch64.patch | 30 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc081.rst | 4 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc084.rst | 140 | ||||
| -rwxr-xr-x | tools/run_tests.sh | 7 |
4 files changed, 170 insertions, 11 deletions
diff --git a/docker/Dockerfile.aarch64.patch b/docker/Dockerfile.aarch64.patch index 33c352a1b..ca933514a 100644 --- a/docker/Dockerfile.aarch64.patch +++ b/docker/Dockerfile.aarch64.patch @@ -1,14 +1,15 @@ -From: Alexandru Nemes <alexandru.nemes@enea.com> -Date: Mon, 19 Jun 2017 14:18:24 +0300 +From: Cristina Pauna <cristina.pauna@enea.com> +Date: Thu, 11 Jan 2018 19:06:26 +0200 Subject: [PATCH] Patch for Yardstick AARCH64 Docker file +Signed-off-by: Cristina Pauna <cristina.pauna@enea.com> Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com> --- - docker/Dockerfile | 10 +++++----- - 1 file changed, 5 insertions(+), 5 deletions(-) + docker/Dockerfile | 13 +++++++------ + 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docker/Dockerfile b/docker/Dockerfile -index 96a5d77..03307a2 100644 +index 2ee5b4c..23e5ea5 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,5 +1,5 @@ @@ -22,15 +23,25 @@ index 96a5d77..03307a2 100644 # http://www.apache.org/licenses/LICENSE-2.0 ############################################################################## --FROM ubuntu:14.04 -+FROM aarch64/ubuntu:14.04 +-FROM ubuntu:16.04 ++FROM arm64v8/ubuntu:16.04 -LABEL image=opnfv/yardstick +LABEL image=opnfv/yardstick_aarch64 ARG BRANCH=master -@@ -40,8 +40,8 @@ RUN echo "daemon off;" >> /etc/nginx/nginx.conf +@@ -24,7 +24,8 @@ ENV YARDSTICK_REPO_DIR="${REPOS_DIR}/yardstick" \ + RELENG_REPO_DIR="${REPOS_DIR}/releng" \ + STORPERF_REPO_DIR="${REPOS_DIR}/storperf" + +-RUN apt-get update && apt-get install -y git python-setuptools python-pip && apt-get -y autoremove && apt-get clean ++RUN apt-get update && apt-get install -y git python-setuptools python-pip && apt-get -y autoremove && \ ++ apt-get install -y libssl-dev && apt-get -y install libffi-dev && apt-get clean + RUN easy_install -U setuptools==30.0.0 + RUN pip install appdirs==1.4.0 pyopenssl==17.5.0 + +@@ -43,8 +44,8 @@ RUN echo "daemon off;" >> /etc/nginx/nginx.conf EXPOSE 5000 @@ -39,4 +50,5 @@ index 96a5d77..03307a2 100644 +ADD http://download.cirros-cloud.net/daily/20161201/cirros-d161201-aarch64-disk.img ${IMAGE_DIR} +ADD http://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-arm64-disk1.img ${IMAGE_DIR} - COPY ./exec_tests.sh /usr/local/bin/
\ No newline at end of file + COPY ./exec_tests.sh /usr/local/bin/ + diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc081.rst b/docs/testing/user/userguide/opnfv_yardstick_tc081.rst index 90af8a382..793c3fdd5 100644 --- a/docs/testing/user/userguide/opnfv_yardstick_tc081.rst +++ b/docs/testing/user/userguide/opnfv_yardstick_tc081.rst @@ -4,7 +4,7 @@ .. (c) OPNFV, Huawei Technologies Co.,Ltd and others. ************************************* -Yardstick Test Case Description TC080 +Yardstick Test Case Description TC081 ************************************* .. _cirros-image: https://download.cirros-cloud.net @@ -21,7 +21,7 @@ Yardstick Test Case Description TC080 |metric | RTT (Round Trip Time) | | | | +--------------+--------------------------------------------------------------+ -|test purpose | The purpose of TC080 is to do a basic verification that | +|test purpose | The purpose of TC081 is to do a basic verification that | | | network latency is within acceptable boundaries when packets | | | travel between a containers and a VM. | | | | diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc084.rst b/docs/testing/user/userguide/opnfv_yardstick_tc084.rst new file mode 100644 index 000000000..2e7b28e25 --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc084.rst @@ -0,0 +1,140 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Technologies Co.,Ltd and others. + +************************************* +Yardstick Test Case Description TC084 +************************************* + +.. _spec_cpu_2006: https://www.spec.org/cpu2006/ + ++-----------------------------------------------------------------------------+ +|Compute Performance | +| | ++--------------+--------------------------------------------------------------+ +|test case id | OPNFV_YARDSTICK_TC084_SPEC CPU 2006 FOR VM | +| | | ++--------------+--------------------------------------------------------------+ +|metric | compute-intensive performance | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The purpose of TC084 is to evaluate the IaaS compute | +| | performance by using SPEC CPU 2006 benchmark. The SPEC CPU | +| | 2006 benchmark has several different ways to measure | +| | computer performance. One way is to measure how fast the | +| | computer completes a single task; this is called a speed | +| | measurement. Another way is to measure how many tasks | +| | computer can accomplish in a certain amount of time; this is | +| | called a throughput, capacity or rate measurement. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | SPEC CPU 2006 | +| | | +| | The SPEC CPU 2006 benchmark is SPEC's industry-standardized, | +| | CPU-intensive benchmark suite, stressing a system's | +| | processor, memory subsystem and compiler. This benchmark | +| | suite includes the SPECint benchmarks and the SPECfp | +| | benchmarks. The SPECint 2006 benchmark contains 12 different | +| | benchmark tests and the SPECfp 2006 benchmark contains 19 | +| | different benchmark tests. | +| | | +| | SPEC CPU 2006 is not always part of a Linux distribution. | +| | SPEC requires that users purchase a license and agree with | +| | their terms and conditions. For this test case, users must | +| | manually download cpu2006-1.2.iso from the SPEC website and | +| | save it under the yardstick/resources folder (e.g. /home/ | +| | opnfv/repos/yardstick/yardstick/resources/cpu2006-1.2.iso) | +| | SPEC CPU® 2006 benchmark is available for purchase via the | +| | SPEC order form (https://www.spec.org/order.html). | +| | | ++--------------+--------------------------------------------------------------+ +|test | This test case uses SPEC CPU 2006 benchmark to measure | +|description | compute-intensive performance of VMs. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | file: opnfv_yardstick_tc084.yaml | +| | | +| | benchmark_subset is set to int. | +| | | +| | SLA is not available in this test case. | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | Test can be configured with different: | +| | | +| | * benchmark_subset - a subset of SPEC CPU 2006 benchmarks | +| | to run; | +| | * SPECint_benchmark - a SPECint benchmark to run; | +| | * SPECint_benchmark - a SPECfp benchmark to run; | +| | * output_format - desired report format; | +| | * runspec_config - SPEC CPU 2006 config file provided to | +| | the runspec binary; | +| | * runspec_iterations - the number of benchmark iterations | +| | to execute. For a reportable run, must be 3; | +| | * runspec_tune - tuning to use (base, peak, or all). For a | +| | reportable run, must be either base or all. Reportable | +| | runs do base first, then (optionally) peak; | +| | * runspec_size - size of input data to run (test, train, or | +| | ref). Reportable runs ensure that your binaries can | +| | produce correct results with the test and train workloads | +| | | ++--------------+--------------------------------------------------------------+ +|usability | This test case is used for executing SPEC CPU 2006 benchmark | +| | on virtual machines. The SPECint 2006 benchmark takes | +| | approximately 5 hours. (The time may vary due to different | +| | VM cpu configurations) | +| | | ++--------------+--------------------------------------------------------------+ +|references | spec_cpu_2006_ | +| | | +| | ETSI-NFV-TST001 | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | To run and install SPEC CPU 2006, the following are | +|conditions | required: | +| | * For SPECint 2006: Both C99 and C++98 compilers are | +| | installed in VM images; | +| | * For SPECfp 2006: All three of C99, C++98 and Fortran-95 | +| | compilers installed in VM images; | +| | * At least 4GB of disk space availabile on VM. | +| | | +| | gcc 4.8.* and g++ 4.8.* version have been tested in Ubuntu | +| | 14.04, Ubuntu 16.04 and Redhat Enterprise Linux 7.4 image. | +| | Higher gcc and g++ version may cause compiling error. | +| | | +| | For more SPEC CPU 2006 dependencies please visit | +| | (https://www.spec.org/cpu2006/Docs/techsupport.html) | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | cpu2006-1.2.iso has been saved under the yardstick/resources | +| | folder (e.g. /home/opnfv/repos/yardstick/yardstick/resources | +| | /cpu2006-1.2.iso). Additionally, to use your custom runspec | +| | config file you can save it under the yardstick/resources/ | +| | files folder and specify the config file name in the | +| | runspec_config parameter. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Upload SPEC CPU 2006 ISO to the target VM using scp and | +| | install SPEC CPU 2006. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Connect to the target server using SSH. | +| | If custom runspec config file is used, copy this file from | +| | yardstick to the target VM via the SSH tunnel. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | SPEC CPU 2006 benchmark is invoked and SPEC CPU 2006 metrics | +| | are generated. | +| | | ++--------------+--------------------------------------------------------------+ +|step 5 | Text, HTML, CSV, PDF, and Configuration file outputs for the | +| | SPEC CPU 2006 metrics are fetched from the VM and stored | +| | under /tmp/result folder. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | None. SPEC CPU 2006 results are collected and stored. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/tools/run_tests.sh b/tools/run_tests.sh index 633c93859..32c4f19e4 100755 --- a/tools/run_tests.sh +++ b/tools/run_tests.sh @@ -29,9 +29,16 @@ run_tests() { echo "Running unittest ... " if [ $FILE_OPTION == "f" ]; then python -m unittest discover -v -s tests/unit > $logfile 2>&1 + if [ $? -ne 0 ]; then + echo "FAILED, results in $logfile" + exit 1 + fi python -m unittest discover -v -s yardstick/tests/unit >> $logfile 2>&1 else python -m unittest discover -v -s tests/unit + if [ $? -ne 0 ]; then + exit 1 + fi python -m unittest discover -v -s yardstick/tests/unit fi |