summaryrefslogtreecommitdiffstats
path: root/docs/testing/developer/devguide
diff options
context:
space:
mode:
authorGeorg Kunz <georg.kunz@ericsson.com>2022-12-12 22:19:21 +0100
committerGeorg Kunz <georg.kunz@ericsson.com>2022-12-13 15:12:14 +0100
commit70b880e98abbc1f51006633dc6ce9805d01c2f6f (patch)
treeef31fc38f7575acdb702a27b114822036e4d885e /docs/testing/developer/devguide
parent336083df2e5ed55b32d204d9e797c5e6684e238b (diff)
[pcac] clean-up of broken links and outdated text
This change fixes all broken links and updates outdated text. It also replaces OPNFV with Anuket in the testing documentation. Signed-off-by: Georg Kunz <georg.kunz@ericsson.com> Change-Id: I5187f19a5345ce19cfe03d1935b6213ff89d56a3
Diffstat (limited to 'docs/testing/developer/devguide')
-rw-r--r--docs/testing/developer/devguide/dev-guide.rst142
1 files changed, 7 insertions, 135 deletions
diff --git a/docs/testing/developer/devguide/dev-guide.rst b/docs/testing/developer/devguide/dev-guide.rst
index c1d39dd45..5cf9b94d2 100644
--- a/docs/testing/developer/devguide/dev-guide.rst
+++ b/docs/testing/developer/devguide/dev-guide.rst
@@ -1,10 +1,6 @@
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. SPDX-License-Identifier: CC-BY-4.0
-***********************************************************
-NOTE - This file will be updated during the Lakelse Release
-***********************************************************
-
***********************
Testing developer guide
***********************
@@ -18,12 +14,9 @@ Testing developer guide
Introduction
============
-The OPNFV testing ecosystem is wide.
-
The goal of this guide consists in providing some guidelines for new developers
involved in test areas.
-For the description of the ecosystem, see `[DEV1]`_.
=================
Developer journey
@@ -42,11 +35,11 @@ resource accross the different projects.
If you develop new test cases, the best practice is to contribute upstream as
much as possible. You may contact the testing group to know which project - in
-OPNFV or upstream - would be the best place to host the test cases. Such
+Anuket or upstream - would be the best place to host the test cases. Such
contributions are usually directly connected to a specific project, more details
can be found in the user guides of the testing projects.
-Each OPNFV testing project provides test cases and the framework to manage them.
+Each Anuket testing project provides test cases and the framework to manage them.
As a developer, you can obviously contribute to them. The developer guide of
the testing projects shall indicate the procedure to follow.
@@ -59,18 +52,6 @@ event is organized after each release. Most of the test projects are present.
The summit is also a good opportunity to meet most of the actors `[DEV4]`_.
-Be involved in the testing group
-================================
-
-The testing group is a self organized working group. The OPNFV projects dealing
-with testing are invited to participate in order to elaborate and consolidate a
-consistant test strategy (test case definition, scope of projects, resources for
-long duration, documentation, ...) and align tooling or best practices.
-
-A weekly meeting is organized, the agenda may be amended by any participant.
-2 slots have been defined (US/Europe and APAC). Agendas and minutes are public.
-See `[DEV3]`_ for details.
-The testing group IRC channel is #opnfv-testperf
Best practices
==============
@@ -134,8 +115,7 @@ possible to prepare the environement and run tests through a CLI.
Dockerization
-------------
Dockerization has been introduced in Brahmaputra and adopted by most of the test
-projects. Docker containers are pulled on the jumphost of OPNFV POD.
-<TODO Jose/Mark/Alec>
+projects.
Code quality
------------
@@ -145,8 +125,7 @@ and more precisely to implement some verifications before any merge:
* pep8
* pylint
-* unit tests (python 2.7)
-* unit tests (python 3.5)
+* unit tests
The code of the test project must be covered by unit tests. The coverage
shall be reasonable and not decrease when adding new features to the framework.
@@ -164,39 +143,13 @@ and/or traffic generation. Some of the tools can be listed as follows:
+---------------+----------------------+------------------------------------+
| Project | Tool | Comments |
+===============+======================+====================================+
-| Bottlenecks | TODO | |
-+---------------+----------------------+------------------------------------+
| Functest | Tempest | OpenStack test tooling |
| | Rally | OpenStack test tooling |
| | Refstack | OpenStack test tooling |
| | RobotFramework | Used for ODL tests |
+---------------+----------------------+------------------------------------+
-| QTIP | Unixbench | |
-| | RAMSpeed | |
-| | nDPI | |
-| | openSSL | |
-| | inxi | |
-+---------------+----------------------+------------------------------------+
-| Storperf | TODO | |
-+---------------+----------------------+------------------------------------+
| VSPERF | TODO | |
+---------------+----------------------+------------------------------------+
-| Yardstick | Moongen | Traffic generator |
-| | Trex | Traffic generator |
-| | Pktgen | Traffic generator |
-| | IxLoad, IxNet | Traffic generator |
-| | SPEC | Compute |
-| | Unixbench | Compute |
-| | RAMSpeed | Compute |
-| | LMBench | Compute |
-| | Iperf3 | Network |
-| | Netperf | Network |
-| | Pktgen-DPDK | Network |
-| | Testpmd | Network |
-| | L2fwd | Network |
-| | Fio | Storage |
-| | Bonnie++ | Storage |
-+---------------+----------------------+------------------------------------+
======================================
@@ -216,7 +169,7 @@ categories can be used to group test suites.
+----------------+-------------------------------------------------------------+
| Smoke | Set of smoke test cases/suites to validate the release |
+----------------+-------------------------------------------------------------+
-| Features | Test cases that validate a specific feature on top of OPNFV.|
+| Features | Test cases that validate a specific feature on top of Anuket|
| | Those come from Feature projects and need a bit of support |
| | for integration |
+----------------+-------------------------------------------------------------+
@@ -279,85 +232,19 @@ impairments to transmission.
These kinds of "load" will cause "disruption" which could be easily found in
system logs. It is the purpose to raise such "load" to evaluate the SUT if it
could provide an acceptable level of service or level of confidence during such
-circumstances. In Danube and Euphrates, we only considered the stress test with
-excess load over OPNFV Platform.
-
-In Danube, Bottlenecks and Yardstick project jointly implemented 2 stress tests
-(concurrently create/destroy VM pairs and do ping, system throughput limit)
-while Bottlenecks acts as the load manager calling yardstick to execute each
-test iteration. These tests are designed to test for breaking points and provide
-level of confidence of the system to users. Summary of the test cases are listed
-in the following addresses:
-
- * https://wiki.opnfv.org/display/bottlenecks/Stress+Testing+over+OPNFV+Platform
- * https://wiki.opnfv.org/download/attachments/2926539/Testing%20over%20Long%20Duration%20POD.pptx?version=2&modificationDate=1502943821000&api=v2
-
-**Stress test cases** for OPNFV Euphrates (OS Ocata) release can be seen as
-extension/enhancement of those in D release. These tests are located in
-Bottlenecks/Yardstick repo (Bottlenecks as load manager while Yardstick execute
-each test iteration):
-
- * VNF scale out/up tests (also plan to measure storage usage simultaneously): https://wiki.opnfv.org/pages/viewpage.action?pageId=12390101
- * Life-cycle event with throughputs (measure NFVI to support concurrent
- network usage from different VM pairs):
- https://wiki.opnfv.org/display/DEV/Intern+Project%3A+Baseline+Stress+Test+Case+for+Bottlenecks+E+Release
-
-In OPNFV E release, we also plan to do **long duration testing** over OS Ocata.
-A separate CI pipe testing OPNFV XCI (OSA) is proposed to accomplish the job.
-We have applied specific pod for the testing.
-Proposals and details are listed below:
-
-* https://wiki.opnfv.org/display/testing/Euphrates+Testing+needs
-* https://wiki.opnfv.org/download/attachments/2926539/testing%20evolution%20v1_4.pptx?version=1&modificationDate=1503937629000&api=v2
-* https://wiki.opnfv.org/download/attachments/2926539/Testing%20over%20Long%20Duration%20POD.pptx?version=2&modificationDate=1502943821000&api=v2
-
-The long duration testing is supposed to be started when OPNFV E release is
-published.
-A simple monitoring module for these tests is also planned to be added:
-https://wiki.opnfv.org/display/DEV/Intern+Project%3A+Monitoring+Stress+Testing+for+Bottlenecks+E+Release
+circumstances.
=======
How TOs
=======
-Where can I find information on the different test projects?
-============================================================
-On http://docs.opnfv.org! A section is dedicated to the testing projects. You
-will find the overview of the ecosystem and the links to the project documents.
-
-Another source is the testing wiki on https://wiki.opnfv.org/display/testing
-
-You may also contact the testing group on the IRC channel #opnfv-testperf or by
-mail at test-wg AT lists.opnfv.org (testing group) or opnfv-tech-discuss AT
-lists.opnfv.org (generic technical discussions).
-
-
How can I contribute to a test project?
=======================================
As any project, the best solution is to contact the project. The project
members with their email address can be found under
https://git.opnfv.org/<project>/tree/INFO
-You may also send a mail to the testing mailing list or use the IRC channel
-#opnfv-testperf
-
-
-Where can I find hardware resources?
-====================================
-You should discuss this topic with the project you are working with. If you need
-access to an OPNFV community POD, it is possible to contact the infrastructure
-group. Depending on your needs (scenario/installer/tooling), it should be
-possible to find free time slots on one OPNFV community POD from the Pharos
-federation. Create a JIRA ticket to describe your needs on
-https://jira.opnfv.org/projects/INFRA.
-You must already be an OPNFV contributor. See
-https://wiki.opnfv.org/display/DEV/Developer+Getting+Started.
-
-Please note that lots of projects have their own "how to contribute" or
-"get started" page on the OPNFV wiki.
-
-
How do I integrate my tests in CI?
==================================
It shall be discussed directly with the project you are working with. It is
@@ -403,8 +290,6 @@ The architecture and associated API is described in previous chapter.
If you want to push your results from CI, you just have to call the API
at the end of your script.
-You can also reuse a python function defined in functest_utils.py `[DEV2]`_
-
Where can I find the documentation on the test API?
===================================================
@@ -412,8 +297,6 @@ Where can I find the documentation on the test API?
The Test API is now documented in this document (see sections above).
You may also find autogenerated documentation in
http://artifacts.opnfv.org/releng/docs/testapi.html
-A web protal is also under construction for certification at
-http://testresults.opnfv.org/test/#/
I have tests, to which category should I declare them?
======================================================
@@ -448,20 +331,9 @@ http://artifacts.opnfv.org/<project name>
References
==========
-`[DEV1]`_: OPNFV Testing Ecosystem
-
-`[DEV2]`_: Python code sample to push results into the Database
-
-`[DEV3]`_: Testing group wiki page
-
`[DEV4]`_: Conversation with the testing community, OPNFV Beijing Summit
`[DEV5]`_: GS NFV 003
-.. _`[DEV1]`: http://docs.opnfv.org/en/latest/testing/ecosystem/index.html
-.. _`[DEV2]`: https://git.opnfv.org/functest/tree/functest/utils/functest_utils.py#176
-.. _`[DEV3]`: https://wiki.opnfv.org/display/meetings/Test+Working+Group+Weekly+Meeting
.. _`[DEV4]`: https://www.youtube.com/watch?v=f9VAUdEqHoA
-.. _`[DEV5]`: http://www.etsi.org/deliver/etsi_gs/NFV/001_099/003/01.01.01_60/gs_NFV003v010101p.pdf
-
-IRC support chan: #opnfv-testperf
+.. _`[DEV5]`: http://www.etsi.org/deliver/etsi_gs/NFV/001_099/003/01.01.01_60/gs_NFV003v010101p.pdf \ No newline at end of file