summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rwxr-xr-xci/README13
-rwxr-xr-xci/jenkins-slave.init37
-rw-r--r--docs/images/ci_infra.pngbin0 -> 123943 bytes
-rw-r--r--docs/opnfv-ci-infrastructure.rst66
-rw-r--r--docs/opnfv-jenkins-slave-connection.rst83
5 files changed, 164 insertions, 35 deletions
diff --git a/ci/README b/ci/README
index a830076..d1bb1b1 100755
--- a/ci/README
+++ b/ci/README
@@ -9,16 +9,17 @@
Before use this script please make sure things below you have done:
1.Install JDK on server
2.Create local user on server: jenkins
-3.Create the local workspace configured in jenkins
-3.Download slave.jar and put it into $HOME directory of jenkins user
-4.Your slave has been configured in OPNFV Jenkins
-5.Obtain slave's token from Linux Foundation Helpdesk
+3.Your slave has been configured in OPNFV Jenkins master
+4.Create slave's local workspace that has been configured in jenkins master
+5.Download slave.jar and put it into the folder of /home/jenkins/
+6.Obtain slave's token from Linux Foundation Helpdesk
Consult wiki : https://wiki.opnfv.org/wiki/jenkins#how_to_connect_servers_from_labs_to_opnfv_jenkins
Follow steps below to launch your slave to OPNFV Jenkins:
1. cp jenkins-slave.init /etc/init.d/jenkins-slave
2. chmod +x /etc/init.d/jenkins-slave
-3. define $SLAVENAME&$TOKEN in config file(jenkins-slave)
+3. Define $SLAVENAME&$TOKEN in config file(jenkins-slave)
4. cp jenkins-slave /etc/default/jenkins-slave
5. service jenkins-slave start
-Finally, run "service jenkins-slave status" command to check the process is running, see the log(/home/jenkins/slave.log) to confirm the connect to OPNFV Jenkins. \ No newline at end of file
+
+Finally, run "service jenkins-slave status" command to check the process is running, see the log(/home/jenkins/jenkins-slave/slave.log) to confirm the connection to OPNFV Jenkins.
diff --git a/ci/jenkins-slave.init b/ci/jenkins-slave.init
index 8402b67..35dc2fd 100755
--- a/ci/jenkins-slave.init
+++ b/ci/jenkins-slave.init
@@ -12,27 +12,44 @@ set -o nounset
set -o pipefail
SLAVE_CONF="/etc/default/jenkins-slave"
-if [ -f ${SLAVE_CONF} ]; then
+
+if [ -f ${SLAVE_CONF} ]
+then
. ${SLAVE_CONF}
else
- echo "Fatal : Configuration file dosenot exist, details please refer to README."
+ echo "Error : Configuration file ($SLAVE_CONF) does not exist, details please refer to README."
exit
fi
if [ -z "${SLAVENAME}" ] || [ -z "${TOKEN}" ]
then
- echo "Fatal : Lack of slavename and token"
+ echo "Error : Please define SLAVENAME and TOKEN values in the Configuration file ($SLAVE_CONF), details please refer to README."
exit
fi
+SLAVE_JNLP="-jnlpUrl https://build.opnfv.org/ci/computer/${SLAVENAME}/slave-agent.jnlp"
JENKINS_HOME="/home/jenkins"
-PID_FILE="${JENKINS_HOME}/slave.pid"
SLAVE_JAR="${JENKINS_HOME}/slave.jar"
-SLAVE_JNLP="-jnlpUrl https://build.opnfv.org/ci/computer/${SLAVENAME}/slave-agent.jnlp"
+JENKINS_SLAVE_ROOTDIR="${JENKINS_HOME}/jenkins-slave"
+PID_FILE="${JENKINS_SLAVE_ROOTDIR}/slave.pid"
+SLAVE_LOG="${JENKINS_SLAVE_ROOTDIR}/slave.log"
USER="jenkins"
-SLAVE_LOG="${JENKINS_HOME}/slave.log"
+GROUP="jenkins"
JAVA_BIN="/usr/bin/java"
+if [ ! -f ${SLAVE_JAR} ]
+then
+ echo "Error : ${SLAVE_JAR} dosenot exist, details please refer to README."
+ exit
+fi
+
+[ -e ${JENKINS_SLAVE_ROOTDIR} ] || { mkdir ${JENKINS_SLAVE_ROOTDIR}; chown ${USER}:${GROUP} ${JENKINS_SLAVE_ROOTDIR};};
+if [ ! -d ${JENKINS_SLAVE_ROOTDIR} ]
+then
+ echo "Error : ${JENKINS_SLAVE_ROOTDIR} can not be created! Maybe there is a file with the same name of 'jenkins-slave'."
+ exit
+fi
+
usage ()
{
cat<<EOF
@@ -74,7 +91,7 @@ status()
then
echo "jenkins slave is not running!"
else
- echo "jenkins slave is running, pid is $pid"
+ echo "jenkins slave is running, pid is $pid"
fi
}
@@ -82,7 +99,7 @@ if [ $# -gt 0 ]
then
OPTION=$1
else
- echo "None valid argument!"
+ echo "Please input your argument!"
usage
exit 1
fi
@@ -103,8 +120,8 @@ case "$OPTION" in
status
;;
*)
- echo "$OPTION is not a valid argument!"
- usage
+ echo "Error: $OPTION is not a valid argument"
+ usage
exit 1
;;
esac
diff --git a/docs/images/ci_infra.png b/docs/images/ci_infra.png
new file mode 100644
index 0000000..53c329c
--- /dev/null
+++ b/docs/images/ci_infra.png
Binary files differ
diff --git a/docs/opnfv-ci-infrastructure.rst b/docs/opnfv-ci-infrastructure.rst
new file mode 100644
index 0000000..6038fa1
--- /dev/null
+++ b/docs/opnfv-ci-infrastructure.rst
@@ -0,0 +1,66 @@
+===========================================
+OPNFV Continuous Integration Infrastructure
+===========================================
+
+This document covers the Continuous Integration(CI) infrastructure used in the day to day operation of the OPNFV, which may be of interest to people who want to help develop this infrastructure or integrate their tools into it.
+
+Infra-overview
+===============
+
+The OPNFV CI infrastructure includes hardware/tools to check, build, deploy and test, etc, in pipeline. Below diagram shows the hardware/tools resources used in Octopus project,
+
+.. image:: images/ci_infra.png
+ :scale: 150%
+ :alt: OPNFV CI Infrastructure
+
+The table below lists the tools/resources that are used in OPNFV CI,
+
++-----------------------------+-----------------------+
+| Tools/Resources | Name |
++=============================+=======================+
+| CI Engine | Jenkins |
++-----------------------------+-----------------------+
+| Source Code Management(SCM) | Git |
++-----------------------------+-----------------------+
+| Code Review | Gerrit |
++-----------------------------+-----------------------+
+| Artifact Repository | Google Storage |
++-----------------------------+-----------------------+
+| Hardware Resources | servers supplied by LF|
++-----------------------------+-----------------------+
+
+The hardware resources are located in Linux Foundation(LF) lab and community labs, for more details, please read the Pharos project, which is shown in https://wiki.opnfv.org/pharos, a guide is provided to describe how to connect your hosting to LF Jenkins, shown in https://wiki.opnfv.org/octopus/jenkins_slave_connection.
+
+Major Infrastructures
+======================
+
+Git/Gerrit
+-----------
+
+Git is the famous open source distributed version control software, it was initially designed and developed for Linux Kernel development, and has become the most widely adopted version control system for software development. Git puts emphasis on speed, data integrity, and support for distributed, non-linear workflows.
+
+Jenkins git plugin is used as the Source Code Manager(SCM), in some way, the Git features make it stand out apart from nearly every other SCM, for more details, please refer to http://git-scm.com/about/. For developers, you can refer to http://git-scm.com/docs for its usage.
+
+Gerrit is used here for facilitating online code reviews for our git version control system. As a reviewer, you can see the changes are shown in a side-by-side style and add some inline comments.
+
+Jenkins and JJB
+----------------
+
+Jenkins is a Continuous Integration system that runs tests and automates some parts of project operations. Jenkins mainly focuses on building/testing software projects continuously and monitoring executions of externally-run jobs. Upstream documentation is available at https://jenkins-ci.org/.
+
+Jenkins supports plugins, which allows to be extended to meet specific requirements. Numbers of plugins have been installed, and new ones can be installed when requirements arise.
+
+The Jenkins jobs are defined by Jenkins Job Builder(JJB) in human readable YAML format. The jobs defined are the key points of CI pipeline. To make clear how the jobs in pipeline run to complete the build, deploy, test works, you can refer to https://wiki.opnfv.org/octopus/pipelines. Moreover, to start your own job in Jenkins, you can write a JJB under the guide of https://wiki.opnfv.org/octopus/jenkins_wow.
+
+Artifact Repository
+-------------------
+
+An artifact repository is a collection of binary software artifacts and metadata stored in a defined directory structure, it is a kin to what subversion is to source code, i.e., it is a way of versioning artifacts produced by build systems, CI, and so on. At this moment, since there is not enough storage space of LF environment, Google Cloud Storage is used as the OPNFV artifact repository temporarily.
+
+If you want to further find out what the artifact repository is and how to use OPNFV artifact repository, the wiki link https://wiki.opnfv.org/octopus/artifact_repository provides a good reference.
+
+**Documentation tracking**
+
+Revision: _sha1_
+
+Build date: _date_
diff --git a/docs/opnfv-jenkins-slave-connection.rst b/docs/opnfv-jenkins-slave-connection.rst
index 178d55d..2adea18 100644
--- a/docs/opnfv-jenkins-slave-connection.rst
+++ b/docs/opnfv-jenkins-slave-connection.rst
@@ -1,14 +1,60 @@
-Jenkins
+:Authors: Fatih Degirmenci (Ericsson AB)
+:Version: 0.1.0
+
+================================
+Connecting Labs to OPNFV Jenkins
+================================
+
+Abstract
+========
+
+This document describes how to connect resources (servers) located in Linux Foundation (LF) lab and labs provided by the OPNFV Community to OPNFV Jenkins.
+
+License
=======
+Connecting Labs to OPNFV Jenkins (c) by Fatih Degirmenci (Ericsson AB)
+
+Connecting Labs to OPNFV Jenkins document is licensed under a Creative Commons Attribution 4.0 International License. You should have received a copy of the license along with this. If not, see <http://creativecommons.org/licenses/by/4.0/>.
+
+**Contents**
+
+1 Version history
+
+2 Jenkins
+
+3 Jenkins Slaves
+
+4 Connecting Slaves to OPNFV Jenkins
+
+5 References
+
+
+1 Version History
+===================
+
++--------------------+--------------------+--------------------+--------------------+
+| **Date** | **Ver.** | **Author** | **Comment** |
+| | | | |
++--------------------+--------------------+--------------------+--------------------+
+| 2015-05-05 | 0.1.0 | Fatih Degirmenci | First draft |
+| | | | |
++--------------------+--------------------+--------------------+--------------------+
+
+2 Jenkins
+===========
Jenkins is an extensible open source Continuous Integration (CI) server. [1]
Linux Foundation (LF) hosts and operates `OPNFV Jenkins <https://build.opnfv.org/ci/>`_.
-Jenkins Slaves
-==============
+3 Jenkins Slaves
+==================
+
+**Slaves** are computers that are set up to build projects for a **Jenkins Master**. [2]
-**Slaves** are computers that are set up to build projects for a **Jenkins Master**. Jenkins runs a separate program called "**slave agent**" on slaves. When slaves are registered to a master, the master starts distributing loads to slaves. Term **Node** is used to refer to all machines that are part of Jenkins grid, slaves and master. [2]
+Jenkins runs a separate program called "**slave agent**" on slaves. When slaves are registered to a master, the master starts distributing loads to slaves. [2]
+
+Term **Node** is used to refer to all machines that are part of Jenkins grid, slaves and master. [2]
Two types of slaves are currently connected to OPNFV Jenkins and handling different tasks depending on the purpose of connecting the slave.
@@ -16,22 +62,22 @@ Two types of slaves are currently connected to OPNFV Jenkins and handling differ
* Slaves hosted in `Community Test Labs <https://wiki.opnfv.org/pharos#community_test_labs>`_
The slaves connected to OPNFV Jenkins can be seen using this link: https://build.opnfv.org/ci/computer/
-The ones that don't have red cross next to computer icon are fully functional.
+Slaves without red cross next to computer icon are fully functional.
-How to Connect Slaves to OPNFV Jenkins
+4 Connecting Slaves to OPNFV Jenkins
======================================
The method that is normally used for connecting slaves to Jenkins requires direct SSH access to servers. [3] This is the method that is used for connecting slaves hosted in LF Lab.
Connecting slaves using direct SSH access can become a challenge given that OPNFV Project has number of different labs provided by community as mentioned in previous section. All these labs have different security requirements which can increase the effort and the time needed for connecting slaves to Jenkins. In order to reduce the effort and the time needed for connecting slaves and streamline the process, it has been decided to connect slaves using `Java Network Launch Protocol (JNLP) <https://docs.oracle.com/javase/tutorial/deployment/deploymentInDepth/jnlp.html>`_.
-Connecting Slaves from LF Lab to OPNFV Jenkins
-----------------------------------------------
+4.1 Connecting Slaves from LF Lab to OPNFV Jenkins
+--------------------------------------------------
Slaves hosted in LF Lab are handled by LF. All the requests and questions regarding these slaves should be submitted to `OPNFV LF Helpdesk <opnfv-helpdesk@rt.linuxfoundation.org>`_.
-Connecting Slaves from Community Labs to OPNFV Jenkins
-------------------------------------------------------
+4.2 Connecting Slaves from Community Labs to OPNFV Jenkins
+----------------------------------------------------------
As noted in corresponding section, slaves from Community Labs are connected using JNLP. Via JNLP, slaves open connection towards Jenkins Master instead of Jenkins Master accessing to them directly.
@@ -44,19 +90,18 @@ Please follow below steps to connect a slave to OPNFV Jenkins.
3. Create a local user on server you want to connect to OPNFV Jenkins. (named **jenkins** for example)
4. Download slave.jar using https://build.opnfv.org/ci/jnlpJars/slave.jar and place it to somewhere so jenkins user created in previous step can access.
5. Create a directory /home/jenkins/opnfv_slave_root.
-6. Ping/contact LF(Aric) via chat as getting the server connected requires Aric's help.
-7. Ask Aric to create a slave on OPNFV Jenkins. The slave should use JNLP as method.
-8. Provide needed information to Aric such as the IP of the server, name of the slave, and slave root. (/home/jenkins/opnfv_slave_root for example)
-9. Aric should provide you the key/token you need to use.
-10. Try to see if you can establish connection towards OPNFV Jenkins by using below command.
+6. Contact LF via chat as getting the server connected requires help from LF. (aricg on freenode)
+7. Provide needed information to LF such as the IP of the server, name of the slave, and slave root. (/home/jenkins/opnfv_slave_root for example)
+8. LF will provide you the key/token you need to use.
+9. Try to see if you can establish connection towards OPNFV Jenkins by using below command.
``java -jar slave.jar -jnlpUrl https://build.opnfv.org/ci/computer/<slave_name>/slave-agent.jnlp -secret <token>``
-11. Navigate to OPNFV Jenkins and look for your slave. It should have some executors in “Idle” state if the connection is successful.
-12. Once you reach this step, you have the server connection to OPNFV Jenkins completed. You can script the command you used above so the connection between slave and Jenkins can be kept open.
+10. Navigate to OPNFV Jenkins and look for your slave. It should have some executors in “Idle” state if the connection is successful.
+11. Once you reach this step, you have the server connection to OPNFV Jenkins completed. You can script the command you used above so the connection between slave and Jenkins can be kept open.
-References
-==========
+5 References
+==============
* `What is Jenkins <https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins>`_
* `Jenkins Terminology <https://wiki.jenkins-ci.org/display/JENKINS/Terminology>`_
* `Jenkins SSH Slaves Plugin <https://wiki.jenkins-ci.org/display/JENKINS/SSH+Slaves+plugin>`_