Adds StorPerf developer guide

The wiki for Development Environment has been converted in .rst format.
JIRA: STORPERF-201

Change-Id: I4e2bc792a8704e4f933f0ea61533c1bf5955c64d
Signed-off-by: Shrenik <shrenik.jain@research.iiit.ac.in>

36 files changed, 472 insertions, 0 deletions

diff --git a/docs/dev/gerrit.rst b/docs/dev/gerrit.rst
new file mode 100644
index 0000000..b227d21
--- /dev/null
+++ b/docs/dev/gerrit.rst
@@ -0,0 +1,203 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Dell EMC and others.
+
+======
+Gerrit
+======
+
+Installing and configuring Git and Git-Review is necessary in order to follow
+this guide. The `Getting Started <https://wiki.opnfv.org/display/DEV/
+Developer+Getting+Started>`_ page will provide you with some help for that.
+
+Committing the code with Gerrit
+===============================
+
+* Open a terminal window and set the project's directory to the working
+  directory using the cd command. In this case "/home/tim/OPNFV/storperf" is
+  the path to the StorPerf project folder on my computer. Replace this with the
+  path of your own project.
+
+.. code-block:: bash
+
+    cd /home/tim/OPNFV/storperf
+
+* Start a new topic for your change.
+
+.. code-block:: bash
+
+    git checkout -b TOPIC-BRANCH
+
+* Tell Git which files you would like to take into account for the next commit.
+  This is called 'staging' the files, by placing them into the staging area,
+  using the 'git add' command (or the synonym 'git stage' command).
+
+.. code-block:: bash
+
+    git add storperf/utilities/math.py
+    git add storperf/tests/utilities/math.py
+    ...
+
+* Alternatively, you can choose to stage all files that have been modified
+  (that is the files you have worked on) since the last time you generated a
+  commit, by using the -a argument.
+
+.. code-block:: bash
+
+    git add -a
+
+* Git won't let you push (upload) any code to Gerrit if you haven't pulled the
+  latest changes first. So the next step is to pull (download) the latest
+  changes made to the project by other collaborators using the 'pull' command.
+
+.. code-block:: bash
+
+    git pull
+
+* Now that you have the latest version of the project and you have staged the
+  files you wish to push, it is time to actually commit your work to your local
+  Git repository.
+
+.. code-block:: bash
+
+    git commit --signoff -m "Title of change
+
+    Test of change that describes in high level what
+    was done. There is a lot of documentation in code
+    so you do not need to repeat it here.
+
+    JIRA: STORPERF-54"
+
+The message that is required for the commit should follow a specific set of
+rules. This practice allows to standardize the description messages attached to
+the commits, and eventually navigate among the latter more easily. This
+`document <https://chris.beams.io/posts/git-commit/>`_ happened to be very clear
+and useful to get started with that.
+
+
+Pushing the code to Git for review
+==================================
+
+* Now that the code has been comitted into your local Git repository the
+  following step is to push it online to Gerrit for it to be reviewed. The
+  command we will use is 'git review'.
+
+.. code-block:: bash
+
+    git review
+
+* This will automatically push your local commit into Gerrit, and the command
+  should get back to you with a Gerrit URL that looks like this :
+
+.. image:: ../images/git_review.png
+
+* The OPNFV-Gerrit-Bot in #opnfv-storperf IRC channel will send a message with
+  the URL as well.
+
+.. image:: ../images/gerrit_bot.png
+
+* Copy/Paste the URL into a web browser to get to the Gerrit code review you
+  have just generated, and click the 'add' button to add reviewers to review
+  your changes :
+
+.. image:: ../images/add_reviewers.png
+
+.. note::
+
+    Check out this `section <https://wiki.opnfv.org/display/storperf/Development
+    +Environment#DevelopmentEnvironment-IfGerrituploadisdenied>`_ if the git
+    review command returns to you with an "access denied" error.
+
+
+Fetching a Git review
+=====================
+
+If you want to collaborate with another developer, you can fetch their review
+by the Gerrit change id (which is part of the URL, and listed in the top left
+as Change NNNNN).
+
+.. code-block:: bash
+
+    git review -d 16213
+
+would download the patchset for change 16213.  If there were a topic branch
+associated with it, it would switch you to that branch, allowing you to look at
+different patch sets locally at the same time without conflicts.
+
+
+Modifying the code under review in Gerrit
+=========================================
+
+At the same time the code is being reviewed in Gerrit, you may need to edit it
+to make some changes and then send it back for review. The following steps go
+through the procedure.
+
+* Once you have modified/edited your code files under your IDE, you will have
+  to stage them. The 'status' command is very helpful at this point as it
+  provides an overview of Git's current state.
+
+.. code-block:: bash
+
+    git status
+
+.. image:: ../images/git_status.png
+
+* The output of the command provides us with the files that have been modified
+  after the latest commit (in this case I modified storperf/tests/utilities/
+  math.py and storperf/utilities/math.py).
+
+* We can now stage the files that have been modified as part of the Gerrit code
+  review edition/modification/improvement :
+
+.. code-block:: bash
+
+    git add storperf/tests/utilities/math.py
+    git add storperf/utilities/math.py
+
+* The 'git status' command should take this into consideration :
+
+.. image:: ../images/git_status_2.png
+
+* It is now time to commit the newly modified files, but the objective here is
+  not to create a new commit, we simply want to inject the new changes into the
+  previous commit. We can achieve that with the '--amend' option on the
+  'commit' command :
+
+.. code-block:: bash
+
+    git commit --amend
+
+.. image:: ../images/amend_commit.png
+
+* If the commit was successful, the 'status' command should not return the
+  updated files as about to be committed.
+
+* The final step consists in pushing the newly modified commit to Gerrit.
+
+.. code-block:: bash
+
+    git review
+
+.. image:: ../images/git_review_2.png
+
+The Gerrit code review should be updated, which results in a 'patch set 2'
+notification appearing in the history log. 'patch set 1' being the original
+code review proposition.
+
+
+If Gerrit upload is denied
+==========================
+
+The 'git review' command might return to you with an "access denied" error that
+looks like this :
+
+.. image:: ../images/Access_denied.png
+
+In this case, you need to make sure your Gerrit account has been added as a
+member of the StorPerf contributors group : ldap/opnfv-gerrit-storperf-
+contributors. You also want to check that have signed the CLA (Contributor
+License Agreement), if not you can sign it in the "Agreements" section of your
+Gerrit account :
+
+.. image:: ../images/CLA_agreement.png
+
diff --git a/docs/dev/ide.rst b/docs/dev/ide.rst
new file mode 100644
index 0000000..3af4b6c
--- /dev/null
+++ b/docs/dev/ide.rst
@@ -0,0 +1,110 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Dell EMC and others.
+
+===
+IDE
+===
+
+While PyCharm as an excellent IDE, some aspects of it require licensing, and
+the PyDev Plugin for Eclipse (packaged as LiClipse) is fully open source
+(although donations are welcome). Therefore this section focuses on using
+LiClipse for StorPerf development.
+
+
+Download
+============
+
+.. code-block:: bash
+
+    http://www.liclipse.com/download.html
+
+
+Storperf virtualenv Interpretor
+=================================
+
+Setting up interpreter under PyDev (LiClipse):
+
+* Go to Project -> Properties, PyDev Interpreter:
+
+.. image:: ../images/PyDev_Interpreter.jpeg
+
+* Click to configure an interpreter not listed.
+
+.. image:: ../images/PyDev_Interpreters_List.jpeg
+
+* Click New, and create a new interpreter called StorPerf that points to your
+  Virtual Env.
+
+.. image:: ../images/PyDev_New_Interpreter.jpeg
+
+* You should get a pop up similar to:
+
+.. image:: ../images/PyDev_Interpreter_Folders.jpeg
+
+* And then you can change the Interpreter to StorPerf.
+
+.. image:: ../images/PyDev_StorPerf_Interpreter.jpeg
+
+
+Code Formatting
+===============
+
+Pep8 and Flake8 rule. These are part of the Gerrit checks and I'm going to
+start enforcing style guidelines soon.
+
+* Go to Window -> Preferences, under PyDev, Editor, Code Style, Code Formatter
+  and select autopep8.py for code formatting.
+
+.. image:: ../images/Code_formatter.jpeg
+
+* Next, under Save Actions, enable "Auto-format editor contents before saving",
+  and "Sort imports on save".
+
+.. image:: ../images/Save_Actions.jpeg
+
+* And under Imports, select Delete unused imports.
+
+.. image:: ../images/Unused_imports.jpeg
+
+* Go to PyDev -> Editor -> Code Analysis and under the pycodestye.py (pep8),
+  select Pep8 as Error.  This flag highlight badly formatted lines as errors.
+  These must be fixed before Jenkins will +1 any review.
+
+.. image:: ../images/Code_analysis.jpeg
+
+
+Import Storperf as Git Project
+==============================
+
+I prefer to do the git clone from the command line, and then import that as a
+local project in LiClipse.
+
+* From the menu: File -> Import Project
+
+.. image:: ../images/Import_Project.png
+
+|
+
+.. image:: ../images/Local_Repo.png
+
+|
+
+.. image:: ../images/Add_git.png
+
+|
+
+* Browse to the directory where you cloned StorPerf
+
+.. image:: ../images/Browse.png
+
+|
+
+* You should now have storperf as a valid local git repo:
+
+.. image:: ../images/Git_Selection.png
+
+|
+
+* Choose Import as general project
+
diff --git a/docs/dev/index.rst b/docs/dev/index.rst
new file mode 100644
index 0000000..48000cf
--- /dev/null
+++ b/docs/dev/index.rst
@@ -0,0 +1,18 @@
+.. _storperf-devguide:
+
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Dell EMC and others.
+
+======================
+StorPerf Dev Guide
+======================
+
+.. toctree::
+   :maxdepth: 2
+
+   initial.rst
+   ide.rst
+   unit_tests.rst
+   gerrit.rst
diff --git a/docs/dev/initial.rst b/docs/dev/initial.rst
new file mode 100644
index 0000000..04b1c45
--- /dev/null
+++ b/docs/dev/initial.rst
@@ -0,0 +1,42 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Dell EMC and others.
+
+================
+Initial Set up
+================
+
+Getting the Code
+================
+
+Replace your LFID with your actual Linux Foundation ID.
+
+.. code-block:: bash
+
+    git clone ssh://YourLFID@gerrit.opnfv.org:29418/storperf
+
+
+Virtual Environment
+=======================
+It is preferred to use virtualenv for Python dependencies. This way it is known
+exactly what libraries are needed, and can restart from a clean state at any
+time to ensure any library is not missing.  Simply running the script:
+
+.. code-block:: bash
+
+    ci/verify.sh
+
+from inside the storperf directory will automatically create a virtualenv in
+the home directory called 'storperf_venv'. This will be used as the Python
+interpreter for the IDE.
+
+
+Docker Version
+=======================
+In order to run the full set of StorPerf services, docker and docker-compose
+are required to be installed. This requires docker 17.05 at a minimum.
+
+.. code-block:: bash
+
+    https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/
+
diff --git a/docs/dev/unit_tests.rst b/docs/dev/unit_tests.rst
new file mode 100644
index 0000000..98ed3ce
--- /dev/null
+++ b/docs/dev/unit_tests.rst
@@ -0,0 +1,88 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Dell EMC and others.
+
+==========
+Unit Tests
+==========
+
+Running from CLI
+================
+
+You technically already did when you ran:
+
+.. code-block:: bash
+
+    ci/verify.sh
+
+The shortcut to running the unit tests again from the command line is:
+
+.. code-block:: bash
+
+    source ~/storperf_venv/bin/activate
+    nosetests --with-xunit \
+          --with-coverage \
+          --cover-package=storperf\
+          --cover-xml \
+          storperf
+
+.. note::
+
+    You must be in the top level storperf directory in order to run the tests.
+
+
+Set up under LiClipse
+=====================
+
+Running the tests:
+
+Right click on the tests folder and select Run as Python Unit Test. Chances
+are, you'll get:
+
+.. code-block:: bash
+
+    Traceback (most recent call last):
+      File "/home/mark/Documents/EMC/git/opnfv/storperf/storperf/tests/storperf_master_test.py", line 24, in setUp
+        self.storperf = StorPerfMaster()
+      File "/home/mark/Documents/EMC/git/opnfv/storperf/storperf/storperf_master.py", line 38, in __init__
+        template_file = open("storperf/resources/hot/agent-group.yaml")
+    IOError: [Errno 2] No such file or directory: 'storperf/resources/hot/agent-group.yaml'
+
+This means we need to set the working directory of the run configuration.
+
+* Under the menu: Run -> Run Configurations:
+
+.. image:: ../images/StorPerf_Tests-Main.jpeg
+
+* Go to the Arguments tab and change the radio button for Working Directory to
+  "Default"
+
+.. image:: ../images/StorPerf_Tests-Arguments.jpeg
+
+* And on interpreter tab, change the interpreter to StorPerf:
+
+.. image:: ../images/StorPerf_Tests-Interpreter.jpeg
+
+* Click Apply. From now on, the run should be clean:
+
+.. image:: ../images/StorPerf_Tests-Console.jpeg
+
+|
+
+.. image:: ../images/StorPerf_Tests-PyUnit.jpeg
+
+
+Adding builtins
+===============
+
+For some reason, sqlite needs to be added as a builtin.
+
+* Go to Window -> Preferences, PyDev > Interpreters > Python Interpreter and
+  select the StorPerf interpreter:
+
+.. image:: ../images/Python_Interpreters.jpeg
+
+* Go to the Forced Builtins tab, click New and add sqlite3.
+
+.. image:: ../images/Forced_Builtins.jpeg
+
diff --git a/docs/images/Access_denied.png b/docs/images/Access_denied.png
new file mode 100644
index 0000000..6a54c0b
--- /dev/null
+++ b/docs/images/Access_denied.png
diff --git a/docs/images/Add_git.png b/docs/images/Add_git.png
new file mode 100644
index 0000000..d7459fc
--- /dev/null
+++ b/docs/images/Add_git.png
diff --git a/docs/images/Browse.png b/docs/images/Browse.png
new file mode 100644
index 0000000..81c4fcd
--- /dev/null
+++ b/docs/images/Browse.png
diff --git a/docs/images/CLA_agreement.png b/docs/images/CLA_agreement.png
new file mode 100644
index 0000000..bf9574d
--- /dev/null
+++ b/docs/images/CLA_agreement.png
diff --git a/docs/images/Code_analysis.jpeg b/docs/images/Code_analysis.jpeg
new file mode 100644
index 0000000..1299636
--- /dev/null
+++ b/docs/images/Code_analysis.jpeg
diff --git a/docs/images/Code_formatter.jpeg b/docs/images/Code_formatter.jpeg
new file mode 100644
index 0000000..84d9a49
--- /dev/null
+++ b/docs/images/Code_formatter.jpeg
diff --git a/docs/images/Forced_Builtins.jpeg b/docs/images/Forced_Builtins.jpeg
new file mode 100644
index 0000000..95870dd
--- /dev/null
+++ b/docs/images/Forced_Builtins.jpeg
diff --git a/docs/images/Git_Selection.png b/docs/images/Git_Selection.png
new file mode 100644
index 0000000..4ca0e42
--- /dev/null
+++ b/docs/images/Git_Selection.png
diff --git a/docs/images/Import_Project.png b/docs/images/Import_Project.png
new file mode 100644
index 0000000..9d7d46a
--- /dev/null
+++ b/docs/images/Import_Project.png
diff --git a/docs/images/Local_Repo.png b/docs/images/Local_Repo.png
new file mode 100644
index 0000000..3c917a3
--- /dev/null
+++ b/docs/images/Local_Repo.png
diff --git a/docs/images/PyDev_Interpreter.jpeg b/docs/images/PyDev_Interpreter.jpeg
new file mode 100644
index 0000000..17a9e8b
--- /dev/null
+++ b/docs/images/PyDev_Interpreter.jpeg
diff --git a/docs/images/PyDev_Interpreter_Folders.jpeg b/docs/images/PyDev_Interpreter_Folders.jpeg
new file mode 100644
index 0000000..bd13e5f
--- /dev/null
+++ b/docs/images/PyDev_Interpreter_Folders.jpeg
diff --git a/docs/images/PyDev_Interpreters_List.jpeg b/docs/images/PyDev_Interpreters_List.jpeg
new file mode 100644
index 0000000..31a2f90
--- /dev/null
+++ b/docs/images/PyDev_Interpreters_List.jpeg
diff --git a/docs/images/PyDev_New_Interpreter.jpeg b/docs/images/PyDev_New_Interpreter.jpeg
new file mode 100644
index 0000000..ad479b9
--- /dev/null
+++ b/docs/images/PyDev_New_Interpreter.jpeg
diff --git a/docs/images/PyDev_StorPerf_Interpreter.jpeg b/docs/images/PyDev_StorPerf_Interpreter.jpeg
new file mode 100644
index 0000000..f2c6cc4
--- /dev/null
+++ b/docs/images/PyDev_StorPerf_Interpreter.jpeg
diff --git a/docs/images/Python_Interpreters.jpeg b/docs/images/Python_Interpreters.jpeg
new file mode 100644
index 0000000..37d4159
--- /dev/null
+++ b/docs/images/Python_Interpreters.jpeg
diff --git a/docs/images/Save_Actions.jpeg b/docs/images/Save_Actions.jpeg
new file mode 100644
index 0000000..cb261e6
--- /dev/null
+++ b/docs/images/Save_Actions.jpeg
diff --git a/docs/images/StorPerf_Tests-Arguments.jpeg b/docs/images/StorPerf_Tests-Arguments.jpeg
new file mode 100644
index 0000000..be2d3fb
--- /dev/null
+++ b/docs/images/StorPerf_Tests-Arguments.jpeg
diff --git a/docs/images/StorPerf_Tests-Console.jpeg b/docs/images/StorPerf_Tests-Console.jpeg
new file mode 100644
index 0000000..85610a5
--- /dev/null
+++ b/docs/images/StorPerf_Tests-Console.jpeg
diff --git a/docs/images/StorPerf_Tests-Interpreter.jpeg b/docs/images/StorPerf_Tests-Interpreter.jpeg
new file mode 100644
index 0000000..7882e75
--- /dev/null
+++ b/docs/images/StorPerf_Tests-Interpreter.jpeg
diff --git a/docs/images/StorPerf_Tests-Main.jpeg b/docs/images/StorPerf_Tests-Main.jpeg
new file mode 100644
index 0000000..228ebad
--- /dev/null
+++ b/docs/images/StorPerf_Tests-Main.jpeg
diff --git a/docs/images/StorPerf_Tests-PyUnit.jpeg b/docs/images/StorPerf_Tests-PyUnit.jpeg
new file mode 100644
index 0000000..acfcca8
--- /dev/null
+++ b/docs/images/StorPerf_Tests-PyUnit.jpeg
diff --git a/docs/images/Unused_imports.jpeg b/docs/images/Unused_imports.jpeg
new file mode 100644
index 0000000..d4619cd
--- /dev/null
+++ b/docs/images/Unused_imports.jpeg
diff --git a/docs/images/add_reviewers.png b/docs/images/add_reviewers.png
new file mode 100644
index 0000000..fdf6016
--- /dev/null
+++ b/docs/images/add_reviewers.png
diff --git a/docs/images/amend_commit.png b/docs/images/amend_commit.png
new file mode 100644
index 0000000..3630ed7
--- /dev/null
+++ b/docs/images/amend_commit.png
diff --git a/docs/images/gerrit_bot.png b/docs/images/gerrit_bot.png
new file mode 100644
index 0000000..bcc5186
--- /dev/null
+++ b/docs/images/gerrit_bot.png
diff --git a/docs/images/git_review.png b/docs/images/git_review.png
new file mode 100644
index 0000000..4efd3c7
--- /dev/null
+++ b/docs/images/git_review.png
diff --git a/docs/images/git_review_2.png b/docs/images/git_review_2.png
new file mode 100644
index 0000000..a4838f0
--- /dev/null
+++ b/docs/images/git_review_2.png
diff --git a/docs/images/git_status.png b/docs/images/git_status.png
new file mode 100644
index 0000000..8fd4898
--- /dev/null
+++ b/docs/images/git_status.png
diff --git a/docs/images/git_status_2.png b/docs/images/git_status_2.png
new file mode 100644
index 0000000..1331c77
--- /dev/null
+++ b/docs/images/git_status_2.png
diff --git a/docs/index.rst b/docs/index.rst
index 9729a44..ae9802d 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -37,6 +37,17 @@ StorPerf User Guide
    ./testing/user/introduction.rst
    ./testing/user/test-usage.rst
 
+******************************
+StorPerf Dev Guide
+******************************
+
+.. toctree::
+   :caption: StorPerf Dev Guide
+   :maxdepth: 5
+   :numbered: 5
+
+   ./dev/index.rst
+
 
 Indices
 =======