Updated docs/enable_docu_gen.rst to include the new updated script

JIRA: Updated docs/enable_docu_gen.rst to include the new updated script
and added **Documentation trackin** to all docs in opnfvdocs

Change-Id: Ifc0390b8f5b96290f37143a49b543ce5e123ed24
Signed-off-by: Victor Laza <vlaza@cloudbasesolutions.com>

7 files changed, 170 insertions, 69 deletions

diff --git a/docs/enable_docu_gen.rst b/docs/enable_docu_gen.rst
index db746bb6c..d80b75bc7 100644
--- a/docs/enable_docu_gen.rst
+++ b/docs/enable_docu_gen.rst
@@ -12,16 +12,22 @@ How to setup the workflow of automatic documentation build for your project
 
 **Inside the repository create the following structure:**::
 
-   gerrit.opnfv.org/<project> - docs/
-                                    /main.rst
-                                    /other-docus.rst
-                                    /images/*.png|*.jpg
+   gerrit.opnfv.org/<project>/docs/some-project-description.rst
+                                  /other-doc-1.rst
+                                  /images/*.png|*.jpg
+
+                              docs/release/some-release-doc.rst
+
+                              requirements/requirements.rst
+
+                              design_docs/some-design-doc.rst
+
 
 More details about the default structure you can find `here <https://wiki.opnfv.org/documentation>`_ at paragraph "How and where to store the document content files in your repository".
 
 **In order to obtain a nice .html & .pdf at then end you must write you documentation using reSt markup**
 
-quick guides: 
+quick guides:
 
 * http://docutils.sourceforge.net/docs/user/rst/quickref.html
 * http://rest-sphinx-memo.readthedocs.org/en/latest/ReST.html
@@ -40,23 +46,89 @@ Enter the project settings::
  cd releng/jjb/<project>/
 
 
-**Create build-docu.sh**
+**Create build-upload-docu.sh**
 
 The script is the same for most of the projects and you can just copy it under your project in releng/jjb/<project>/
 
 example: cp releng/jjb/opnfvdocs/build-docu.sh releng/jjb/<your-project>/::
 
  #!/bin/bash
- set -xv
- for file in $(find . -type f -iname '*.rst'); do
-        file_cut="${{file%.*}}"
-        html_file=$file_cut".html"
-        pdf_file=$file_cut".pdf"
-        rst2html $file > $html_file
-        rst2pdf $file -o $pdf_file
+ project="functest"
+ export PATH=$PATH:/usr/local/bin/
+
+ git_sha1="$(git rev-parse HEAD)"
+ docu_build_date="$(date)"
+
+ files=()
+ while read -r -d ''; do
+         files+=("$REPLY")
+ done < <(find * -type f -iname '*.rst' -print0)
+
+ for file in "${{files[@]}}"; do
+
+         file_cut="${{file%.*}}"
+         gs_cp_folder="${{file_cut}}"
+
+         # sed part
+         sed -i "s/_sha1_/$git_sha1/g" $file
+         sed -i "s/_date_/$docu_build_date/g" $file
+
+         # rst2html part
+         echo "rst2html $file"
+         rst2html $file | gsutil cp -L gsoutput.txt - \
+         gs://artifacts.opnfv.org/"$project"/"$gs_cp_folder".html
+         gsutil setmeta -h "Content-Type:text/html" \
+                        -h "Cache-Control:private, max-age=0, no-transform" \
+                        gs://artifacts.opnfv.org/"$project"/"$gs_cp_folder".html
+         cat gsoutput.txt
+         rm -f gsoutput.txt
+
+         echo "rst2pdf $file"
+         rst2pdf $file -o - | gsutil cp -L gsoutput.txt - \
+         gs://artifacts.opnfv.org/"$project"/"$gs_cp_folder".pdf
+         gsutil setmeta -h "Content-Type:application/pdf" \
+                        -h "Cache-Control:private, max-age=0, no-transform" \
+                        gs://artifacts.opnfv.org/"$project"/"$gs_cp_folder".pdf
+         cat gsoutput.txt
+         rm -f gsoutput.txt
+
+ done
+
+ #the double {{ in file_cut="${{file%.*}}" is to escape jjb's yaml
+
+
+**Create build-docu-verify.sh**::
+
+ #!/bin/bash
+ project="opnfvdocs"
+ export PATH=$PATH:/usr/local/bin/
+
+ git_sha1="$(git rev-parse HEAD)"
+ docu_build_date="$(date)"
+
+ files=()
+ while read -r -d ''; do
+         files+=("$REPLY")
+ done < <(find * -type f -iname '*.rst' -print0)
+
+ for file in "${{files[@]}}"; do
+
+         file_cut="${{file%.*}}"
+         gs_cp_folder="${{file_cut}}"
+
+         # sed part
+         sed -i "s/_sha1_/$git_sha1/g" $file
+         sed -i "s/_date_/$docu_build_date/g" $file
+
+         # rst2html part
+         echo "rst2html $file"
+         rst2html $file > $file_cut".html"
+
+         echo "rst2pdf $file"
+         rst2pdf $file -o $file_cut".pdf"
+
  done
 
- 
  #the double {{ in file_cut="${{file%.*}}" is to escape jjb's yaml
 
 
@@ -74,28 +146,21 @@ example: less releng/jjb/opnfvdocs/opnfvdocs.yml (pay extra attention at the "bu
     ...
     builders:
         - shell:
-            !include-raw build-docu.sh
-        - shell: |
-            echo $PATH
-            /usr/local/bin/gsutil cp docs/*.pdf gs://artifacts.opnfv.org/opnfvdocs/docs/
-            /usr/local/bin/gsutil cp docs/*.html gs://artifacts.opnfv.org/opnfvdocs/docs/
+            !include-raw build-upload-docu.sh
 
  - job-template:
     name: 'opnfvdocs-verify'
     ...
     builders:
         - shell:
-            !include-raw build-docu.sh
+            !include-raw build-docu-verify.sh
 
  - job-template:
     name: 'opnfvdocs-merge'
     ...
     builders:
         - shell:
-            !include-raw build-docu.sh
-        - shell: |
-           /usr/local/bin/gsutil cp docs/*.pdf gs://artifacts.opnfv.org/opnfvdocs/docs/
-           /usr/local/bin/gsutil cp docs/*.html gs://artifacts.opnfv.org/opnfvdocs/docs/
+            !include-raw build-upload-docu.sh
 
 
 Please reffer to the releng repository for the correct indentation as JJB is very picky with those and also for the rest of the code that is missing in the example code and replaced by "...".
@@ -148,6 +213,19 @@ Please try to write documentation as accurate and clear as possible as once revi
 If you want to see on wiki what code is scraped from the built artifacts click "Show pagesource" in the right (it will appear if you hover over the magnifier icon); this way you know what is written straight on wiki and what is embedded with "scrape". By knowing these details you will be able to prevent damages by manually updating wiki.
 
 
+**How to track documentation**
+
+You must include at the bottom of every document that you want to track the following::
+
+ **Documentation tracking**
+
+ Revision: _sha1
+
+ Build date:  _date
+
+ # add one "_" at the end of each trigger variable (they have also a prefix "_") when inserting them into documents to enable auto-replacement
+
+
 
 NOTE:
 ------
@@ -156,4 +234,10 @@ In order to generate html & pdf documentation the needed packages are rst2pdf &
 For html generation it also supports css styles if needed.
 
 
+**Documentation tracking**
+
+Revision: _sha1_
+
+Build date:  _date_
+
 
diff --git a/docs/opnfv-jjb-usage.rst b/docs/opnfv-jjb-usage.rst
index ccb932301..495d36404 100644
--- a/docs/opnfv-jjb-usage.rst
+++ b/docs/opnfv-jjb-usage.rst
@@ -163,3 +163,9 @@ The Current merge and verify jobs for jenkins job builder as pulled from the rep
                 jenkins-jobs update --delete-old jjb/
 
 
+
+**Documentation tracking**
+
+Revision: _sha1_
+
+Build date:  _date_
diff --git a/docs/main.rst b/docs/opnfvdocs_info.rst
index cc0b5577c..1c78eeaf0 100644
--- a/docs/main.rst
+++ b/docs/opnfvdocs_info.rst
@@ -94,7 +94,8 @@ Proposed Release Schedule:
 
 **Documentation tracking**
 
- Revision: _sha1_
- Build date:  _date_
+Revision: _sha1_
+
+Build date:  _date_
 
 
diff --git a/docs/other_options_for_docu_gen.rst b/docs/other_options_for_docu_gen.rst
index 2a050a0d3..73d38dd92 100644
--- a/docs/other_options_for_docu_gen.rst
+++ b/docs/other_options_for_docu_gen.rst
@@ -5,72 +5,71 @@ Other options to generate documentation that we tested
 
 Description: This was the first discovered method
 
-- html: using Doxygen plugin + HTML publisher
+* html: using Doxygen plugin + HTML publisher
   It involves some customization at doxygen level + custom html header/footer
 
-- pdf: it generates a .pdf using latex
+* pdf: it generates a .pdf using latex
+* Input files: .md , .rst
+* Output: .html & .pdf
+* Pros:
 
-- Input files: .md , .rst
+        - standard tools: doxygen, html publisher, LaTeX suite
+        - doxygen plugin available in Jenkins, you just need to install it; html publisher plugin available in Jenkins, you just need to install it
+        - destination files are generated fast
+        - standard reStructuredText or Markdown
 
-- Output: .html & .pdf
+* Cons:
 
-- Pros:
+        - takes some time to customize the output in matters of template, requires custom html header/footer
+        - latex suite is quite substantial in amount of packages and consumed space (around 1.2 GB)
 
- - standard tools: doxygen, html publisher, LaTeX suite
- - doxygen plugin available in Jenkins, you just need to install it; html publisher plugin available in Jenkins, you just need to install it
- - destination files are generated fast
- - standard reStructuredText or Markdown
-
-- Cons:
-
- - takes some time to customize the output in matters of template, requires custom html header/footer
- - latex suite is quite substantial in amount of packages and consumed space (around 1.2 GB)
-
-- Tested: roughly, functional tests only
+* Tested: roughly, functional tests only
 
 **Maven & clouddocs-maven-plugin (actually used to generate openstack-manuals)**
 
 Description: It represents the standard tool to generate Openstack documentation manuals, uses maven, maven plugins, clouddocs-maven-plugins; location of finally generated files is the object of a small Bash script that will reside as Post-actions
 
-- Input files: .xml
+* Input files: .xml
+* Output: .html & .pdf
+* Pros:
 
-- Output: .html & .pdf
+        - quite easy for initial setup
+        - uses openstack documentation generation flows as for openstack-manuals (clouddocs-maven-plugin), maven installs all you need generate the documentation
 
-- Pros:
+* Cons:
 
- - quite easy for initial setup
- - uses openstack documentation generation flows as for openstack-manuals (clouddocs-maven-plugin), maven installs all you need generate the documentation
+        - could be tricky to generate a custom layout, knowledge about Maven plugins required, .pom editing
+        - dependent of multiple maven plugins
+        - input files are .xml and xml editing knowledge is required
 
-- Cons:
-
- - could be tricky to generate a custom layout, knowledge about Maven plugins required, .pom editing
- - dependent of multiple maven plugins
- - input files are .xml and xml editing knowledge is required
-
-- Tested: roughly, functional tests only
+* Tested: roughly, functional tests only
 
 **Sphinx & LaTeX suite**
 
 Description: The easiest to install, the cleanest in matter of folder & files structure, uses standard tools available in repositories; location of finally generated files is the object of a small Bash script that will reside as Post-actions
 
-- Input files: .rst as default
+* Input files: .rst as default
+* Output: .html & .pdf
+* Pros:
+
+        - standard tools: Python Sphinx, LaTeX suite
+        - destination files are generated fast
+        - standard reStructuredText as default; other inputs can be configured
+        - Sphinx's installation is very clean in matters of folder structure; the cleanest from all tested variants
+        - latex suite is also easy to install via yum/apt and available in general repos
+        - everyone is migration from other tools to Spinx lately; it provides more control and better looking documentation
+        - can be used also for source-code documentation, specially if you use Python
 
-- Output: .html & .pdf
+* Cons:
 
-- Pros:
+        - takes some time to customize the output in matters of template, requires custom html header/footer
+        - latex suite is quite substantial in amount of packages and consumed space (around 1.2 GB)
 
- - standard tools: Python Sphinx, LaTeX suite
- - destination files are generated fast
- - standard reStructuredText as default; other inputs can be configured
- - Sphinx's installation is very clean in matters of folder structure; the cleanest from all tested variants
- - latex suite is also easy to install via yum/apt and available in general repos
- - everyone is migration from other tools to Spinx lately; it provides more control and better looking documentation
- - can be used also for source-code documentation, specially if you use Python
+* Tested: roughly, functional tests only
 
-- Cons:
 
- - takes some time to customize the output in matters of template, requires custom html header/footer
- - latex suite is quite substantial in amount of packages and consumed space (around 1.2 GB)
+**Documentation tracking**
 
-- Tested: roughly, functional tests only
+Revision: _sha1_
 
+Build date:  _date_
diff --git a/docs/release/main.rst b/docs/release/main.rst
deleted file mode 100644
index 7bfa94bc5..000000000
--- a/docs/release/main.rst
+++ /dev/null
@@ -1 +0,0 @@
-Just a dummy file
diff --git a/docs/templates/installation-instructions.rst b/docs/templates/installation-instructions.rst
index 495adb2ac..166452df7 100644
--- a/docs/templates/installation-instructions.rst
+++ b/docs/templates/installation-instructions.rst
@@ -1,6 +1,11 @@
 :Authors: Jonas Bjurel (Ericsson)
 :Version: 0.1
 
+Revision: _sha1_
+
+Build date:  _date_
+
+
 ================================================================
 OPNFV Installation instructions for - < Component denomination >
 ================================================================
@@ -217,3 +222,5 @@ Change the Host OS password by......
 
 9.4    Fuel
 ------------
+
+
diff --git a/docs/templates/release-notes.rst b/docs/templates/release-notes.rst
index d6b21cbeb..9104d37e3 100644
--- a/docs/templates/release-notes.rst
+++ b/docs/templates/release-notes.rst
@@ -1,6 +1,11 @@
 :Authors: Jonas Bjurel (Ericsson)
 :Version: 0.1
 
+Revision: _sha1_
+
+Build date:  _date_
+
+
 ======================================================================
 OPNFV Release Note for "Arno-SRx release" - < Component denomination >
 ======================================================================