From 8962423227e13f45ef9f7dfbf48ed4847d3f3865 Mon Sep 17 00:00:00 2001 From: mbeierl Date: Fri, 4 May 2018 14:04:51 -0400 Subject: Adds Volume Count and Dynamic Reload Adds a volume_count parameter that allows the user to change the number of volumes. Test execution still uses each volume specified and appends it to the host ip address in the graphite metrics. Changes the cache policy of the stack from SNAPS-OO so that changes made to the stack are refelcted in StorPerf. Fixes the documentation path and updates the docs for the above changes. Change-Id: I5b9160b469e80d4f7d6906ac270bb3b75411fb5a JIRA: STORPERF-242 Signed-off-by: mbeierl (cherry picked from commit 1942e374fba718d1d4b6bc5388803c75e71aa197) --- docs/testing/developer/devguide/gerrit.rst | 203 +++++++++++++++++++++++++ docs/testing/developer/devguide/ide.rst | 110 ++++++++++++++ docs/testing/developer/devguide/index.rst | 18 +++ docs/testing/developer/devguide/initial.rst | 42 +++++ docs/testing/developer/devguide/unit_tests.rst | 88 +++++++++++ 5 files changed, 461 insertions(+) create mode 100644 docs/testing/developer/devguide/gerrit.rst create mode 100644 docs/testing/developer/devguide/ide.rst create mode 100644 docs/testing/developer/devguide/index.rst create mode 100644 docs/testing/developer/devguide/initial.rst create mode 100644 docs/testing/developer/devguide/unit_tests.rst (limited to 'docs/testing/developer/devguide') diff --git a/docs/testing/developer/devguide/gerrit.rst b/docs/testing/developer/devguide/gerrit.rst new file mode 100644 index 0000000..b227d21 --- /dev/null +++ b/docs/testing/developer/devguide/gerrit.rst @@ -0,0 +1,203 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +====== +Gerrit +====== + +Installing and configuring Git and Git-Review is necessary in order to follow +this guide. The `Getting Started `_ page will provide you with some help for that. + +Committing the code with Gerrit +=============================== + +* Open a terminal window and set the project's directory to the working + directory using the cd command. In this case "/home/tim/OPNFV/storperf" is + the path to the StorPerf project folder on my computer. Replace this with the + path of your own project. + +.. code-block:: bash + + cd /home/tim/OPNFV/storperf + +* Start a new topic for your change. + +.. code-block:: bash + + git checkout -b TOPIC-BRANCH + +* Tell Git which files you would like to take into account for the next commit. + This is called 'staging' the files, by placing them into the staging area, + using the 'git add' command (or the synonym 'git stage' command). + +.. code-block:: bash + + git add storperf/utilities/math.py + git add storperf/tests/utilities/math.py + ... + +* Alternatively, you can choose to stage all files that have been modified + (that is the files you have worked on) since the last time you generated a + commit, by using the -a argument. + +.. code-block:: bash + + git add -a + +* Git won't let you push (upload) any code to Gerrit if you haven't pulled the + latest changes first. So the next step is to pull (download) the latest + changes made to the project by other collaborators using the 'pull' command. + +.. code-block:: bash + + git pull + +* Now that you have the latest version of the project and you have staged the + files you wish to push, it is time to actually commit your work to your local + Git repository. + +.. code-block:: bash + + git commit --signoff -m "Title of change + + Test of change that describes in high level what + was done. There is a lot of documentation in code + so you do not need to repeat it here. + + JIRA: STORPERF-54" + +The message that is required for the commit should follow a specific set of +rules. This practice allows to standardize the description messages attached to +the commits, and eventually navigate among the latter more easily. This +`document `_ happened to be very clear +and useful to get started with that. + + +Pushing the code to Git for review +================================== + +* Now that the code has been comitted into your local Git repository the + following step is to push it online to Gerrit for it to be reviewed. The + command we will use is 'git review'. + +.. code-block:: bash + + git review + +* This will automatically push your local commit into Gerrit, and the command + should get back to you with a Gerrit URL that looks like this : + +.. image:: ../images/git_review.png + +* The OPNFV-Gerrit-Bot in #opnfv-storperf IRC channel will send a message with + the URL as well. + +.. image:: ../images/gerrit_bot.png + +* Copy/Paste the URL into a web browser to get to the Gerrit code review you + have just generated, and click the 'add' button to add reviewers to review + your changes : + +.. image:: ../images/add_reviewers.png + +.. note:: + + Check out this `section `_ if the git + review command returns to you with an "access denied" error. + + +Fetching a Git review +===================== + +If you want to collaborate with another developer, you can fetch their review +by the Gerrit change id (which is part of the URL, and listed in the top left +as Change NNNNN). + +.. code-block:: bash + + git review -d 16213 + +would download the patchset for change 16213. If there were a topic branch +associated with it, it would switch you to that branch, allowing you to look at +different patch sets locally at the same time without conflicts. + + +Modifying the code under review in Gerrit +========================================= + +At the same time the code is being reviewed in Gerrit, you may need to edit it +to make some changes and then send it back for review. The following steps go +through the procedure. + +* Once you have modified/edited your code files under your IDE, you will have + to stage them. The 'status' command is very helpful at this point as it + provides an overview of Git's current state. + +.. code-block:: bash + + git status + +.. image:: ../images/git_status.png + +* The output of the command provides us with the files that have been modified + after the latest commit (in this case I modified storperf/tests/utilities/ + math.py and storperf/utilities/math.py). + +* We can now stage the files that have been modified as part of the Gerrit code + review edition/modification/improvement : + +.. code-block:: bash + + git add storperf/tests/utilities/math.py + git add storperf/utilities/math.py + +* The 'git status' command should take this into consideration : + +.. image:: ../images/git_status_2.png + +* It is now time to commit the newly modified files, but the objective here is + not to create a new commit, we simply want to inject the new changes into the + previous commit. We can achieve that with the '--amend' option on the + 'commit' command : + +.. code-block:: bash + + git commit --amend + +.. image:: ../images/amend_commit.png + +* If the commit was successful, the 'status' command should not return the + updated files as about to be committed. + +* The final step consists in pushing the newly modified commit to Gerrit. + +.. code-block:: bash + + git review + +.. image:: ../images/git_review_2.png + +The Gerrit code review should be updated, which results in a 'patch set 2' +notification appearing in the history log. 'patch set 1' being the original +code review proposition. + + +If Gerrit upload is denied +========================== + +The 'git review' command might return to you with an "access denied" error that +looks like this : + +.. image:: ../images/Access_denied.png + +In this case, you need to make sure your Gerrit account has been added as a +member of the StorPerf contributors group : ldap/opnfv-gerrit-storperf- +contributors. You also want to check that have signed the CLA (Contributor +License Agreement), if not you can sign it in the "Agreements" section of your +Gerrit account : + +.. image:: ../images/CLA_agreement.png + diff --git a/docs/testing/developer/devguide/ide.rst b/docs/testing/developer/devguide/ide.rst new file mode 100644 index 0000000..3af4b6c --- /dev/null +++ b/docs/testing/developer/devguide/ide.rst @@ -0,0 +1,110 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +=== +IDE +=== + +While PyCharm as an excellent IDE, some aspects of it require licensing, and +the PyDev Plugin for Eclipse (packaged as LiClipse) is fully open source +(although donations are welcome). Therefore this section focuses on using +LiClipse for StorPerf development. + + +Download +============ + +.. code-block:: bash + + http://www.liclipse.com/download.html + + +Storperf virtualenv Interpretor +================================= + +Setting up interpreter under PyDev (LiClipse): + +* Go to Project -> Properties, PyDev Interpreter: + +.. image:: ../images/PyDev_Interpreter.jpeg + +* Click to configure an interpreter not listed. + +.. image:: ../images/PyDev_Interpreters_List.jpeg + +* Click New, and create a new interpreter called StorPerf that points to your + Virtual Env. + +.. image:: ../images/PyDev_New_Interpreter.jpeg + +* You should get a pop up similar to: + +.. image:: ../images/PyDev_Interpreter_Folders.jpeg + +* And then you can change the Interpreter to StorPerf. + +.. image:: ../images/PyDev_StorPerf_Interpreter.jpeg + + +Code Formatting +=============== + +Pep8 and Flake8 rule. These are part of the Gerrit checks and I'm going to +start enforcing style guidelines soon. + +* Go to Window -> Preferences, under PyDev, Editor, Code Style, Code Formatter + and select autopep8.py for code formatting. + +.. image:: ../images/Code_formatter.jpeg + +* Next, under Save Actions, enable "Auto-format editor contents before saving", + and "Sort imports on save". + +.. image:: ../images/Save_Actions.jpeg + +* And under Imports, select Delete unused imports. + +.. image:: ../images/Unused_imports.jpeg + +* Go to PyDev -> Editor -> Code Analysis and under the pycodestye.py (pep8), + select Pep8 as Error. This flag highlight badly formatted lines as errors. + These must be fixed before Jenkins will +1 any review. + +.. image:: ../images/Code_analysis.jpeg + + +Import Storperf as Git Project +============================== + +I prefer to do the git clone from the command line, and then import that as a +local project in LiClipse. + +* From the menu: File -> Import Project + +.. image:: ../images/Import_Project.png + +| + +.. image:: ../images/Local_Repo.png + +| + +.. image:: ../images/Add_git.png + +| + +* Browse to the directory where you cloned StorPerf + +.. image:: ../images/Browse.png + +| + +* You should now have storperf as a valid local git repo: + +.. image:: ../images/Git_Selection.png + +| + +* Choose Import as general project + diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst new file mode 100644 index 0000000..48000cf --- /dev/null +++ b/docs/testing/developer/devguide/index.rst @@ -0,0 +1,18 @@ +.. _storperf-devguide: + +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +====================== +StorPerf Dev Guide +====================== + +.. toctree:: + :maxdepth: 2 + + initial.rst + ide.rst + unit_tests.rst + gerrit.rst diff --git a/docs/testing/developer/devguide/initial.rst b/docs/testing/developer/devguide/initial.rst new file mode 100644 index 0000000..04b1c45 --- /dev/null +++ b/docs/testing/developer/devguide/initial.rst @@ -0,0 +1,42 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +================ +Initial Set up +================ + +Getting the Code +================ + +Replace your LFID with your actual Linux Foundation ID. + +.. code-block:: bash + + git clone ssh://YourLFID@gerrit.opnfv.org:29418/storperf + + +Virtual Environment +======================= +It is preferred to use virtualenv for Python dependencies. This way it is known +exactly what libraries are needed, and can restart from a clean state at any +time to ensure any library is not missing. Simply running the script: + +.. code-block:: bash + + ci/verify.sh + +from inside the storperf directory will automatically create a virtualenv in +the home directory called 'storperf_venv'. This will be used as the Python +interpreter for the IDE. + + +Docker Version +======================= +In order to run the full set of StorPerf services, docker and docker-compose +are required to be installed. This requires docker 17.05 at a minimum. + +.. code-block:: bash + + https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/ + diff --git a/docs/testing/developer/devguide/unit_tests.rst b/docs/testing/developer/devguide/unit_tests.rst new file mode 100644 index 0000000..98ed3ce --- /dev/null +++ b/docs/testing/developer/devguide/unit_tests.rst @@ -0,0 +1,88 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +========== +Unit Tests +========== + +Running from CLI +================ + +You technically already did when you ran: + +.. code-block:: bash + + ci/verify.sh + +The shortcut to running the unit tests again from the command line is: + +.. code-block:: bash + + source ~/storperf_venv/bin/activate + nosetests --with-xunit \ + --with-coverage \ + --cover-package=storperf\ + --cover-xml \ + storperf + +.. note:: + + You must be in the top level storperf directory in order to run the tests. + + +Set up under LiClipse +===================== + +Running the tests: + +Right click on the tests folder and select Run as Python Unit Test. Chances +are, you'll get: + +.. code-block:: bash + + Traceback (most recent call last): + File "/home/mark/Documents/EMC/git/opnfv/storperf/storperf/tests/storperf_master_test.py", line 24, in setUp + self.storperf = StorPerfMaster() + File "/home/mark/Documents/EMC/git/opnfv/storperf/storperf/storperf_master.py", line 38, in __init__ + template_file = open("storperf/resources/hot/agent-group.yaml") + IOError: [Errno 2] No such file or directory: 'storperf/resources/hot/agent-group.yaml' + +This means we need to set the working directory of the run configuration. + +* Under the menu: Run -> Run Configurations: + +.. image:: ../images/StorPerf_Tests-Main.jpeg + +* Go to the Arguments tab and change the radio button for Working Directory to + "Default" + +.. image:: ../images/StorPerf_Tests-Arguments.jpeg + +* And on interpreter tab, change the interpreter to StorPerf: + +.. image:: ../images/StorPerf_Tests-Interpreter.jpeg + +* Click Apply. From now on, the run should be clean: + +.. image:: ../images/StorPerf_Tests-Console.jpeg + +| + +.. image:: ../images/StorPerf_Tests-PyUnit.jpeg + + +Adding builtins +=============== + +For some reason, sqlite needs to be added as a builtin. + +* Go to Window -> Preferences, PyDev > Interpreters > Python Interpreter and + select the StorPerf interpreter: + +.. image:: ../images/Python_Interpreters.jpeg + +* Go to the Forced Builtins tab, click New and add sqlite3. + +.. image:: ../images/Forced_Builtins.jpeg + -- cgit 1.2.3-korg