summaryrefslogtreecommitdiffstats
path: root/xci/scripts
AgeCommit message (Collapse)AuthorFilesLines
2017-10-10Merge "xci: scripts: start-new-vm.sh: Do not run multiple package manager ↵Markos Chandras1-7/+29
processes"
2017-10-10Merge "xci: scripts: start-new-vm.sh: Add option to clean VM on failure"Markos Chandras1-0/+15
2017-10-09xci: scripts: start-new-vm.sh: Do not run multiple package manager processesMarkos Chandras1-7/+29
Package managers get upset if multiple instances run in parallel since the last one will fail to aquire some sort of DB locking and fail. As such, lets fix that and only run one package manager process at any time. Change-Id: I4ab13f61eee861ef70cb4115403d0e239ec14308 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-09xci: scripts: start-new-vm.sh: Add option to clean VM on failureMarkos Chandras1-0/+15
Sometimes we may want to remove the clean VM in case of failure. This is mostly true for CI tests because developers do not normally have access to the deployed VMs to debug problems since they can easily recreate the same environment using this script. If they ever want direct access to the slaves then that option can easily be turned off and retrigger the job. In the meantime, this patch allows slaves to free resources when a job fails instead of keeping the broken tests around. Change-Id: I07d56661e45fc64e1ec0db7a359522e4ebd2891a Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-07xci: scripts: start-new-vm.sh: Do not destroy user's configuration fileMarkos Chandras1-16/+20
As we move towards making this script useful for developers, we shouldn't destroy the existing ssh configuration file but rather create a new one just for XCI. This also move the destruction of previous instances sooner, even before building the OS image so we can free resources as soon as possible. Finally, this also uses the hostname to ssh to the new VM instead of its IP to ensure that everything is setup up properly on the physical host. Change-Id: I185befd8943fdb8ebcdb635aefb4d4b03473258b Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-05xci: scripts: build-dib-os.sh: chmod and chown files using sudoMarkos Chandras2-2/+14
Some of the files may be owned by root so we need to ensure that everything is reset properly. Moreover, only clean the image files for the one we are building. Change-Id: I45b78db6715534187f19d9f513e8288fd076cb6b Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-04xci: scripts: start-new-vm.sh: Copy hidden files and directoriesMarkos Chandras1-1/+1
Fix the rsync command to also copy the hidden files and directories. Copying the .git directory allows us to quickly inspect any file changes when investigating bug reports and other problems. Change-Id: I4e6feeea77d5c3b23362a216206c5970b4900169 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-04xci: scripts: vm: Add ability to use cached images for clean VMMarkos Chandras2-38/+109
Make it possible to use the hosted dib images from artifacts.opnfv.org instead of building a new one on every CI run. This way, we can reduce the time it takes to start the actual XCI deployment. Change-Id: Idb1f5e3929cc14502c3f7383a61d2fbd0a6eefaf Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-03xci: scripts: start-new-vm.sh: Set JENKINS_HOME on local runsMarkos Chandras1-0/+5
If we are creating a new virtual machine for local testing, then we should set JENKINS_HOME since some playbooks use this variable to determine if the deployment happens on a CI or not. That script aims to simulate a CI deployment so it should have this variable set. Change-Id: I1165c8b2cd467ba92fcf6f6c178a69b07834894e Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-03xci: scripts: start-new-vm.sh: Do not break on idle SSH connectionsMarkos Chandras1-0/+2
Add ServerAliveInterval and ServerAliveCountMax directives in order to try and keep SSH connections alive. This aims to solve ssh timeouts like the following one on busy hypervisors: packet_write_wait: Connection to UNKNOWN port 0: Broken pipe Change-Id: I58029b41226098d6a44181434ad8653f72384e9d Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-02Merge "xci: scripts: start-new-vm.sh: Use Docker to build OS images"Markos Chandras2-104/+85
2017-10-02xci: scripts: start-new-vm.sh: Use Docker to build OS imagesMarkos Chandras2-104/+85
Use a docker container to build the OS images so we can build images and start new virtual machines on all supported operating systems. This way all developers can now launch a virtual machine to quickly reproduce Jenkins results which should assist with debugging problems. Since the container runs with elevated privileges it's best to ensure that we have exclusive access to devices. Finally, we remove the build-dib-os.sh script which is now part of the container itself. The build image process now becomes more stable since it runs on clean evnironment all the time so the only external factor is the upstream distribution repositories. Change-Id: I6b443192419ee2546a23430f421b152766d16333 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-02xci: scripts: start-new-vm.sh: Fix default test caseMarkos Chandras1-1/+2
Export XCI_FLAVOR to mini to minic the CI. Moreover, fix the script to not use relative paths. Change-Id: I16947c596b8bbff20371b6d345dffd215f76aecc Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-28xci: scripts: build-dib-os.sh: Pin diskimage-builderMarkos Chandras1-1/+1
diskimage-builder does very frequent releases and sometimes things break. There is no particular reason to always fetch the latest version so lets do it in a controlled way after proper testing. Change-Id: Ie9685a9a15fa7fefa1c5ecea49090ac9d92e63ff Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: scripts: start-new-vm.sh: Fix paths for default XCI testMarkos Chandras1-2/+2
The XCI test is located in ~/releng-xci inside the new VM so we need to look there for an existing one or for executing it after the default one has been created and copied to the remote host. Change-Id: Ieb195293da7832bcb5d29c8a28cc3477205f2e5e Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: scripts: start-new-vm.sh: Print dib output when running on CIMarkos Chandras2-3/+7
Until we are able to fetch the dib images from external resources, we need to build them as part of the job. diskimage-builder can sometimes fail so we need to be able to see the log for debug purposes. Change-Id: Iab8bfba08daa7095cf76537f629c8e7bf6330b17 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: scripts: start-new-vm.sh: Use 'unsafe' cache for the clean vm diskMarkos Chandras1-1/+1
This is not a production virtual machine and we don't care if we lose data in case it's killed unexpectedly so we can just ignore the 'flush' commands from the guest. This will also improve the overall I/O in the VMs. Change-Id: Ib486a073f667e22f4a1e65a220ac553874f708cc Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: scripts: start-new-vm.sh: Do not allow multiple build-dib-os processesMarkos Chandras2-3/+13
apt-get is really unhappy when another instance is running and fails with the following error: E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable) E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it? Change-Id: I2fe343fdab8438cb112cce0a4f81c7e3977c55f9 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-26xci: scripts: Fix script to run on Jenkins CIMarkos Chandras2-15/+33
A couple of tweaks are necessary to get it working with Jenkins - Jenkins jobs could contain the 'xci' string so make the regexp more accurate. - Rename VMs to use a more accurate name - Fix ssh public key location - Create a fresh /etc/hosts since distro may not have one. - Set hostname on VM Change-Id: I332a424bc8b2de98d7b326c192996b7b12c79dd7 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-25xci: scripts: Add script to use clean virtual machines for testingMarkos Chandras4-0/+281
These scripts will prepare a clean OS image based on diskimage-builder and launch a new VM with it. The purpose of that is to use this virtual machine as a 'host' to run XCI on it. This way we can ensure that all tests are being executed on the same clean environment which is ideal for reproducing and debugging purposes. We also commit the ssh keypair so we can build it in the image and use it to control the virtual machine without user interaction. Needless to say that this keypair MUST NOT be used for anything else apart from the dib build process. Change-Id: Idec0fc06c82435adc1a4d569b8e578616998de5f Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-06xci: scripts: Update wording for commentsMarkos Chandras1-1/+1
Update wording so it matches the upstream one and minimize the noise on a-r-r updates. Change-Id: I216c1ee1ac4f24ab2c5a382bfe5e2aacf324024a Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: scripts: Update SHA datesMarkos Chandras1-2/+4
Update the dates in the pinned-versions file whenever we update the SHAs. Change-Id: I23ebd90a3fc688a7501fcb2d32217357f8c31657 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: update-osa-version-files: Add support for updating bifrost SHAsMarkos Chandras1-3/+8
Add a second optional argument to update the bifrost SHA instead of doing this bit manually. Change-Id: I0fab8745ae08ad4f54a2a1f5b0b7a64fde4b8295 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Update SHAs for OSA and its rolesMarkos Chandras1-3/+2
This also updates the second (pre_release) argument of 'update_ansible_role_requirements' from 'false' to 'true' so that non-openstack roles can be updated as well. Change-Id: Id424499f44c9b51c02d56e5d93580faeea50ad12 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: scripts: Add update-osa-version-files.sh scriptMarkos Chandras1-0/+85
Add new prototypes/xci/scripts/update-osa-version-files.sh which can be used to update the XCI ansible-role-requirements.yml file as well as the OSA pinned SHA string. This file is using the upstream 'sources-branch-updater-lib.sh' library from the openstack-ansible repository. Change-Id: I51b88c50cb2bffe0cf1b7aa054a5b237103fd92f Signed-off-by: Markos Chandras <mchandras@suse.de>