diff options
Diffstat (limited to 'docs')
36 files changed, 472 insertions, 0 deletions
diff --git a/docs/dev/gerrit.rst b/docs/dev/gerrit.rst new file mode 100644 index 0000000..b227d21 --- /dev/null +++ b/docs/dev/gerrit.rst @@ -0,0 +1,203 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +====== +Gerrit +====== + +Installing and configuring Git and Git-Review is necessary in order to follow +this guide. The `Getting Started <https://wiki.opnfv.org/display/DEV/ +Developer+Getting+Started>`_ page will provide you with some help for that. + +Committing the code with Gerrit +=============================== + +* Open a terminal window and set the project's directory to the working + directory using the cd command. In this case "/home/tim/OPNFV/storperf" is + the path to the StorPerf project folder on my computer. Replace this with the + path of your own project. + +.. code-block:: bash + + cd /home/tim/OPNFV/storperf + +* Start a new topic for your change. + +.. code-block:: bash + + git checkout -b TOPIC-BRANCH + +* Tell Git which files you would like to take into account for the next commit. + This is called 'staging' the files, by placing them into the staging area, + using the 'git add' command (or the synonym 'git stage' command). + +.. code-block:: bash + + git add storperf/utilities/math.py + git add storperf/tests/utilities/math.py + ... + +* Alternatively, you can choose to stage all files that have been modified + (that is the files you have worked on) since the last time you generated a + commit, by using the -a argument. + +.. code-block:: bash + + git add -a + +* Git won't let you push (upload) any code to Gerrit if you haven't pulled the + latest changes first. So the next step is to pull (download) the latest + changes made to the project by other collaborators using the 'pull' command. + +.. code-block:: bash + + git pull + +* Now that you have the latest version of the project and you have staged the + files you wish to push, it is time to actually commit your work to your local + Git repository. + +.. code-block:: bash + + git commit --signoff -m "Title of change + + Test of change that describes in high level what + was done. There is a lot of documentation in code + so you do not need to repeat it here. + + JIRA: STORPERF-54" + +The message that is required for the commit should follow a specific set of +rules. This practice allows to standardize the description messages attached to +the commits, and eventually navigate among the latter more easily. This +`document <https://chris.beams.io/posts/git-commit/>`_ happened to be very clear +and useful to get started with that. + + +Pushing the code to Git for review +================================== + +* Now that the code has been comitted into your local Git repository the + following step is to push it online to Gerrit for it to be reviewed. The + command we will use is 'git review'. + +.. code-block:: bash + + git review + +* This will automatically push your local commit into Gerrit, and the command + should get back to you with a Gerrit URL that looks like this : + +.. image:: ../images/git_review.png + +* The OPNFV-Gerrit-Bot in #opnfv-storperf IRC channel will send a message with + the URL as well. + +.. image:: ../images/gerrit_bot.png + +* Copy/Paste the URL into a web browser to get to the Gerrit code review you + have just generated, and click the 'add' button to add reviewers to review + your changes : + +.. image:: ../images/add_reviewers.png + +.. note:: + + Check out this `section <https://wiki.opnfv.org/display/storperf/Development + +Environment#DevelopmentEnvironment-IfGerrituploadisdenied>`_ if the git + review command returns to you with an "access denied" error. + + +Fetching a Git review +===================== + +If you want to collaborate with another developer, you can fetch their review +by the Gerrit change id (which is part of the URL, and listed in the top left +as Change NNNNN). + +.. code-block:: bash + + git review -d 16213 + +would download the patchset for change 16213. If there were a topic branch +associated with it, it would switch you to that branch, allowing you to look at +different patch sets locally at the same time without conflicts. + + +Modifying the code under review in Gerrit +========================================= + +At the same time the code is being reviewed in Gerrit, you may need to edit it +to make some changes and then send it back for review. The following steps go +through the procedure. + +* Once you have modified/edited your code files under your IDE, you will have + to stage them. The 'status' command is very helpful at this point as it + provides an overview of Git's current state. + +.. code-block:: bash + + git status + +.. image:: ../images/git_status.png + +* The output of the command provides us with the files that have been modified + after the latest commit (in this case I modified storperf/tests/utilities/ + math.py and storperf/utilities/math.py). + +* We can now stage the files that have been modified as part of the Gerrit code + review edition/modification/improvement : + +.. code-block:: bash + + git add storperf/tests/utilities/math.py + git add storperf/utilities/math.py + +* The 'git status' command should take this into consideration : + +.. image:: ../images/git_status_2.png + +* It is now time to commit the newly modified files, but the objective here is + not to create a new commit, we simply want to inject the new changes into the + previous commit. We can achieve that with the '--amend' option on the + 'commit' command : + +.. code-block:: bash + + git commit --amend + +.. image:: ../images/amend_commit.png + +* If the commit was successful, the 'status' command should not return the + updated files as about to be committed. + +* The final step consists in pushing the newly modified commit to Gerrit. + +.. code-block:: bash + + git review + +.. image:: ../images/git_review_2.png + +The Gerrit code review should be updated, which results in a 'patch set 2' +notification appearing in the history log. 'patch set 1' being the original +code review proposition. + + +If Gerrit upload is denied +========================== + +The 'git review' command might return to you with an "access denied" error that +looks like this : + +.. image:: ../images/Access_denied.png + +In this case, you need to make sure your Gerrit account has been added as a +member of the StorPerf contributors group : ldap/opnfv-gerrit-storperf- +contributors. You also want to check that have signed the CLA (Contributor +License Agreement), if not you can sign it in the "Agreements" section of your +Gerrit account : + +.. image:: ../images/CLA_agreement.png + diff --git a/docs/dev/ide.rst b/docs/dev/ide.rst new file mode 100644 index 0000000..3af4b6c --- /dev/null +++ b/docs/dev/ide.rst @@ -0,0 +1,110 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +=== +IDE +=== + +While PyCharm as an excellent IDE, some aspects of it require licensing, and +the PyDev Plugin for Eclipse (packaged as LiClipse) is fully open source +(although donations are welcome). Therefore this section focuses on using +LiClipse for StorPerf development. + + +Download +============ + +.. code-block:: bash + + http://www.liclipse.com/download.html + + +Storperf virtualenv Interpretor +================================= + +Setting up interpreter under PyDev (LiClipse): + +* Go to Project -> Properties, PyDev Interpreter: + +.. image:: ../images/PyDev_Interpreter.jpeg + +* Click to configure an interpreter not listed. + +.. image:: ../images/PyDev_Interpreters_List.jpeg + +* Click New, and create a new interpreter called StorPerf that points to your + Virtual Env. + +.. image:: ../images/PyDev_New_Interpreter.jpeg + +* You should get a pop up similar to: + +.. image:: ../images/PyDev_Interpreter_Folders.jpeg + +* And then you can change the Interpreter to StorPerf. + +.. image:: ../images/PyDev_StorPerf_Interpreter.jpeg + + +Code Formatting +=============== + +Pep8 and Flake8 rule. These are part of the Gerrit checks and I'm going to +start enforcing style guidelines soon. + +* Go to Window -> Preferences, under PyDev, Editor, Code Style, Code Formatter + and select autopep8.py for code formatting. + +.. image:: ../images/Code_formatter.jpeg + +* Next, under Save Actions, enable "Auto-format editor contents before saving", + and "Sort imports on save". + +.. image:: ../images/Save_Actions.jpeg + +* And under Imports, select Delete unused imports. + +.. image:: ../images/Unused_imports.jpeg + +* Go to PyDev -> Editor -> Code Analysis and under the pycodestye.py (pep8), + select Pep8 as Error. This flag highlight badly formatted lines as errors. + These must be fixed before Jenkins will +1 any review. + +.. image:: ../images/Code_analysis.jpeg + + +Import Storperf as Git Project +============================== + +I prefer to do the git clone from the command line, and then import that as a +local project in LiClipse. + +* From the menu: File -> Import Project + +.. image:: ../images/Import_Project.png + +| + +.. image:: ../images/Local_Repo.png + +| + +.. image:: ../images/Add_git.png + +| + +* Browse to the directory where you cloned StorPerf + +.. image:: ../images/Browse.png + +| + +* You should now have storperf as a valid local git repo: + +.. image:: ../images/Git_Selection.png + +| + +* Choose Import as general project + diff --git a/docs/dev/index.rst b/docs/dev/index.rst new file mode 100644 index 0000000..48000cf --- /dev/null +++ b/docs/dev/index.rst @@ -0,0 +1,18 @@ +.. _storperf-devguide: + +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +====================== +StorPerf Dev Guide +====================== + +.. toctree:: + :maxdepth: 2 + + initial.rst + ide.rst + unit_tests.rst + gerrit.rst diff --git a/docs/dev/initial.rst b/docs/dev/initial.rst new file mode 100644 index 0000000..04b1c45 --- /dev/null +++ b/docs/dev/initial.rst @@ -0,0 +1,42 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +================ +Initial Set up +================ + +Getting the Code +================ + +Replace your LFID with your actual Linux Foundation ID. + +.. code-block:: bash + + git clone ssh://YourLFID@gerrit.opnfv.org:29418/storperf + + +Virtual Environment +======================= +It is preferred to use virtualenv for Python dependencies. This way it is known +exactly what libraries are needed, and can restart from a clean state at any +time to ensure any library is not missing. Simply running the script: + +.. code-block:: bash + + ci/verify.sh + +from inside the storperf directory will automatically create a virtualenv in +the home directory called 'storperf_venv'. This will be used as the Python +interpreter for the IDE. + + +Docker Version +======================= +In order to run the full set of StorPerf services, docker and docker-compose +are required to be installed. This requires docker 17.05 at a minimum. + +.. code-block:: bash + + https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/ + diff --git a/docs/dev/unit_tests.rst b/docs/dev/unit_tests.rst new file mode 100644 index 0000000..98ed3ce --- /dev/null +++ b/docs/dev/unit_tests.rst @@ -0,0 +1,88 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Dell EMC and others. + +========== +Unit Tests +========== + +Running from CLI +================ + +You technically already did when you ran: + +.. code-block:: bash + + ci/verify.sh + +The shortcut to running the unit tests again from the command line is: + +.. code-block:: bash + + source ~/storperf_venv/bin/activate + nosetests --with-xunit \ + --with-coverage \ + --cover-package=storperf\ + --cover-xml \ + storperf + +.. note:: + + You must be in the top level storperf directory in order to run the tests. + + +Set up under LiClipse +===================== + +Running the tests: + +Right click on the tests folder and select Run as Python Unit Test. Chances +are, you'll get: + +.. code-block:: bash + + Traceback (most recent call last): + File "/home/mark/Documents/EMC/git/opnfv/storperf/storperf/tests/storperf_master_test.py", line 24, in setUp + self.storperf = StorPerfMaster() + File "/home/mark/Documents/EMC/git/opnfv/storperf/storperf/storperf_master.py", line 38, in __init__ + template_file = open("storperf/resources/hot/agent-group.yaml") + IOError: [Errno 2] No such file or directory: 'storperf/resources/hot/agent-group.yaml' + +This means we need to set the working directory of the run configuration. + +* Under the menu: Run -> Run Configurations: + +.. image:: ../images/StorPerf_Tests-Main.jpeg + +* Go to the Arguments tab and change the radio button for Working Directory to + "Default" + +.. image:: ../images/StorPerf_Tests-Arguments.jpeg + +* And on interpreter tab, change the interpreter to StorPerf: + +.. image:: ../images/StorPerf_Tests-Interpreter.jpeg + +* Click Apply. From now on, the run should be clean: + +.. image:: ../images/StorPerf_Tests-Console.jpeg + +| + +.. image:: ../images/StorPerf_Tests-PyUnit.jpeg + + +Adding builtins +=============== + +For some reason, sqlite needs to be added as a builtin. + +* Go to Window -> Preferences, PyDev > Interpreters > Python Interpreter and + select the StorPerf interpreter: + +.. image:: ../images/Python_Interpreters.jpeg + +* Go to the Forced Builtins tab, click New and add sqlite3. + +.. image:: ../images/Forced_Builtins.jpeg + diff --git a/docs/images/Access_denied.png b/docs/images/Access_denied.png Binary files differnew file mode 100644 index 0000000..6a54c0b --- /dev/null +++ b/docs/images/Access_denied.png diff --git a/docs/images/Add_git.png b/docs/images/Add_git.png Binary files differnew file mode 100644 index 0000000..d7459fc --- /dev/null +++ b/docs/images/Add_git.png diff --git a/docs/images/Browse.png b/docs/images/Browse.png Binary files differnew file mode 100644 index 0000000..81c4fcd --- /dev/null +++ b/docs/images/Browse.png diff --git a/docs/images/CLA_agreement.png b/docs/images/CLA_agreement.png Binary files differnew file mode 100644 index 0000000..bf9574d --- /dev/null +++ b/docs/images/CLA_agreement.png diff --git a/docs/images/Code_analysis.jpeg b/docs/images/Code_analysis.jpeg Binary files differnew file mode 100644 index 0000000..1299636 --- /dev/null +++ b/docs/images/Code_analysis.jpeg diff --git a/docs/images/Code_formatter.jpeg b/docs/images/Code_formatter.jpeg Binary files differnew file mode 100644 index 0000000..84d9a49 --- /dev/null +++ b/docs/images/Code_formatter.jpeg diff --git a/docs/images/Forced_Builtins.jpeg b/docs/images/Forced_Builtins.jpeg Binary files differnew file mode 100644 index 0000000..95870dd --- /dev/null +++ b/docs/images/Forced_Builtins.jpeg diff --git a/docs/images/Git_Selection.png b/docs/images/Git_Selection.png Binary files differnew file mode 100644 index 0000000..4ca0e42 --- /dev/null +++ b/docs/images/Git_Selection.png diff --git a/docs/images/Import_Project.png b/docs/images/Import_Project.png Binary files differnew file mode 100644 index 0000000..9d7d46a --- /dev/null +++ b/docs/images/Import_Project.png diff --git a/docs/images/Local_Repo.png b/docs/images/Local_Repo.png Binary files differnew file mode 100644 index 0000000..3c917a3 --- /dev/null +++ b/docs/images/Local_Repo.png diff --git a/docs/images/PyDev_Interpreter.jpeg b/docs/images/PyDev_Interpreter.jpeg Binary files differnew file mode 100644 index 0000000..17a9e8b --- /dev/null +++ b/docs/images/PyDev_Interpreter.jpeg diff --git a/docs/images/PyDev_Interpreter_Folders.jpeg b/docs/images/PyDev_Interpreter_Folders.jpeg Binary files differnew file mode 100644 index 0000000..bd13e5f --- /dev/null +++ b/docs/images/PyDev_Interpreter_Folders.jpeg diff --git a/docs/images/PyDev_Interpreters_List.jpeg b/docs/images/PyDev_Interpreters_List.jpeg Binary files differnew file mode 100644 index 0000000..31a2f90 --- /dev/null +++ b/docs/images/PyDev_Interpreters_List.jpeg diff --git a/docs/images/PyDev_New_Interpreter.jpeg b/docs/images/PyDev_New_Interpreter.jpeg Binary files differnew file mode 100644 index 0000000..ad479b9 --- /dev/null +++ b/docs/images/PyDev_New_Interpreter.jpeg diff --git a/docs/images/PyDev_StorPerf_Interpreter.jpeg b/docs/images/PyDev_StorPerf_Interpreter.jpeg Binary files differnew file mode 100644 index 0000000..f2c6cc4 --- /dev/null +++ b/docs/images/PyDev_StorPerf_Interpreter.jpeg diff --git a/docs/images/Python_Interpreters.jpeg b/docs/images/Python_Interpreters.jpeg Binary files differnew file mode 100644 index 0000000..37d4159 --- /dev/null +++ b/docs/images/Python_Interpreters.jpeg diff --git a/docs/images/Save_Actions.jpeg b/docs/images/Save_Actions.jpeg Binary files differnew file mode 100644 index 0000000..cb261e6 --- /dev/null +++ b/docs/images/Save_Actions.jpeg diff --git a/docs/images/StorPerf_Tests-Arguments.jpeg b/docs/images/StorPerf_Tests-Arguments.jpeg Binary files differnew file mode 100644 index 0000000..be2d3fb --- /dev/null +++ b/docs/images/StorPerf_Tests-Arguments.jpeg diff --git a/docs/images/StorPerf_Tests-Console.jpeg b/docs/images/StorPerf_Tests-Console.jpeg Binary files differnew file mode 100644 index 0000000..85610a5 --- /dev/null +++ b/docs/images/StorPerf_Tests-Console.jpeg diff --git a/docs/images/StorPerf_Tests-Interpreter.jpeg b/docs/images/StorPerf_Tests-Interpreter.jpeg Binary files differnew file mode 100644 index 0000000..7882e75 --- /dev/null +++ b/docs/images/StorPerf_Tests-Interpreter.jpeg diff --git a/docs/images/StorPerf_Tests-Main.jpeg b/docs/images/StorPerf_Tests-Main.jpeg Binary files differnew file mode 100644 index 0000000..228ebad --- /dev/null +++ b/docs/images/StorPerf_Tests-Main.jpeg diff --git a/docs/images/StorPerf_Tests-PyUnit.jpeg b/docs/images/StorPerf_Tests-PyUnit.jpeg Binary files differnew file mode 100644 index 0000000..acfcca8 --- /dev/null +++ b/docs/images/StorPerf_Tests-PyUnit.jpeg diff --git a/docs/images/Unused_imports.jpeg b/docs/images/Unused_imports.jpeg Binary files differnew file mode 100644 index 0000000..d4619cd --- /dev/null +++ b/docs/images/Unused_imports.jpeg diff --git a/docs/images/add_reviewers.png b/docs/images/add_reviewers.png Binary files differnew file mode 100644 index 0000000..fdf6016 --- /dev/null +++ b/docs/images/add_reviewers.png diff --git a/docs/images/amend_commit.png b/docs/images/amend_commit.png Binary files differnew file mode 100644 index 0000000..3630ed7 --- /dev/null +++ b/docs/images/amend_commit.png diff --git a/docs/images/gerrit_bot.png b/docs/images/gerrit_bot.png Binary files differnew file mode 100644 index 0000000..bcc5186 --- /dev/null +++ b/docs/images/gerrit_bot.png diff --git a/docs/images/git_review.png b/docs/images/git_review.png Binary files differnew file mode 100644 index 0000000..4efd3c7 --- /dev/null +++ b/docs/images/git_review.png diff --git a/docs/images/git_review_2.png b/docs/images/git_review_2.png Binary files differnew file mode 100644 index 0000000..a4838f0 --- /dev/null +++ b/docs/images/git_review_2.png diff --git a/docs/images/git_status.png b/docs/images/git_status.png Binary files differnew file mode 100644 index 0000000..8fd4898 --- /dev/null +++ b/docs/images/git_status.png diff --git a/docs/images/git_status_2.png b/docs/images/git_status_2.png Binary files differnew file mode 100644 index 0000000..1331c77 --- /dev/null +++ b/docs/images/git_status_2.png diff --git a/docs/index.rst b/docs/index.rst index 9729a44..ae9802d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -37,6 +37,17 @@ StorPerf User Guide ./testing/user/introduction.rst ./testing/user/test-usage.rst +****************************** +StorPerf Dev Guide +****************************** + +.. toctree:: + :caption: StorPerf Dev Guide + :maxdepth: 5 + :numbered: 5 + + ./dev/index.rst + Indices ======= |
