summaryrefslogtreecommitdiffstats
path: root/xci/scripts
AgeCommit message (Collapse)AuthorFilesLines
2018-03-01Do not fail the if virsh command failsFatih Degirmenci1-2/+2
Change-Id: Id9c55b45719399c6c78c0d3d8b97926a2907b7f8 Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-02-28Ensure the ssh config files are created per distroFatih Degirmenci1-4/+7
If 2 jobs for different distros start on the same host, the generated xci-vm-config files might be rewritten by both jobs causing trouble for the other. This change creates ssh config per distro to prevent potential issues. Change-Id: If6c0a86fd51bf3ba920e9206849ed1537894c7e1 Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-02-28Fix the proxy command and hostname for opnfv vm in ssh configFatih Degirmenci1-1/+2
Change-Id: Ic036a4f5ef8b24b88154f9314957339d3954839a Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-02-27xci: script: start-new-vm: Switch to using 'curl' instead of 'wget'Markos Chandras1-3/+4
curl supports the 'retry' argument to handle all sorts of connection problems so we can use it to retry if we encounter a broken connection. Change-Id: I3c8afd2be36ee36c5477f05495704bbbee5b9757 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-27xci: scripts: start-new-vm: Mask the pgrep exit codeMarkos Chandras1-1/+1
pgrep will exit with non-zero exit code if it didn't find a matching process. This breaks the script since we set 'errexit' so we need to mask it by simply piping it to a 'cat' command. Change-Id: I80dcf06b597a769242bd59d628bbd8e09f0f199e Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-27Merge "xci: scripts: vm: start-new-vm.sh: Re-arrange debug messages"Markos Chandras1-2/+9
2018-02-26Merge "xci: Switch default CPU model to 'host-model'"Markos Chandras1-1/+1
2018-02-23xci: scripts: start-new-vm.sh: Disable nested virtualization in clean VMMarkos Chandras1-0/+3
When using the clean VM for the tests, we will end up with 3 levels of virtualization when functest is executed (1st level = clean vm, 2nd level = compute node, 3rd level = functest VMs). This makes the functest VM terribly slow leading to all sorts of random failures. It appears that KVM can't handle this case, so we need to switch to the QEMU interpreter instead which is slower but functional. This only affects the Jenkins jobs, so deployments on baremetal will still use 2 levels of nested KVM virtualization which should work fine. Change-Id: If274129fbf347526982ac4cf577d216173eb4d1b Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-23xci: scripts: start-new-vm: Set pipefail optionMarkos Chandras1-0/+1
In commit I75a8cb8c05957bb205e63210d6cafaf54d989f32 the 'ts' utility was added in order to print timestamps for deployment. However, this breaks the exit code since 'pipefail' was not set so every job was returning 'success'. Change-Id: Ia5f47ea96a4a5be9ed0664f5c7100accc30cde51 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-23xci: Switch default CPU model to 'host-model'Markos Chandras1-1/+1
From upstream docs: 'The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports' It's probably safer to use that instead of 'host-passthrough' so we can get a CPU (both for main VM and the nested ones) that libvirt understands. Moreover, it's important to present a CPU that the guest OS understands as well. Change-Id: I25a8ff0e8635df9804c793d184f048cc86059ce0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-21Merge "xci: xci-deploy.sh: Calculate and print the total deployment time"Markos Chandras1-1/+1
2018-02-20xci: Switch VM disk cache to 'unsafe' and use 'iothreads' for I/OMarkos Chandras1-2/+2
According to the docs[1] "writeback: This mode causes the hypervisor to interact with the disk image file or block device with neither O_DSYNC nor O_DIRECT semantics. The host page cache is used and writes are reported to the guest as completed when they are placed in the host page cache. The normal page cache management will handle commitment to the storage device. Additionally, the guest's virtual storage adapter is informed of the writeback cache, so the guest would be expected to send down flush commands as needed to manage data integrity. Analogous to a raid controller with RAM cache." and "writeback: This mode informs the guest of the presence of a write cache, and relies on the guest to send flush commands as needed to maintain data integrity within its disk image. This is a common storage design which is completely accounted for within modern file systems. This mode exposes the guest to data loss in the unlikely case of a host failure, because there is a window of time between the time a write is reported as completed, and that write being committed to the storage device." "unsafe: This mode is similar to writeback caching except for the following: the guest flush commands are ignored, nullifying the data integrity control of these flush commands, and resulting in a higher risk of data loss because of host failure. The name “unsafe” should serve as a warning that there is a much higher potential for data loss because of a host failure than with the other modes. As the guest terminates, the cached data is flushed at that time." It's beneficial to use the host page cache to cache I/O from the guest instead of waiting for data to reach the actual disk device. We do not normally care about data integrity so data loss is not a problem. Moreover, we drop the cache configuration from the flavor files since it's independent of the flavor that's being deployed. [1] https://www.suse.com/documentation/sles-12/singlehtml/book_virt/book_virt.html#cha.cachemodes Change-Id: I118ffdf84b1be672185b3eff60fe5d0b5f1a590d Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-20xci: scripts: vm: start-new-vm.sh: Re-arrange debug messagesMarkos Chandras1-2/+9
This also improves (again!) the check for parallel packages managers to print some useful messages for debugging. Change-Id: I7756db80aa3ed42cd3ef37293f7116b5a9c150c1 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-19xci: xci-deploy.sh: Calculate and print the total deployment timeMarkos Chandras1-1/+1
It's useful to know how XCI performs in terms of duration time so use the bash $SECONDS variable to calculate the total amount of time it takes for a full XCI deployment. Moreover, use the 'ts' command to print a timestamp for every command executed. This will help developers identify tasks which take a lot of time to complete and possibly optimize them. Change-Id: I75a8cb8c05957bb205e63210d6cafaf54d989f32 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-13bifrost: Add ability to pin Ironic-{,client,inspector,inspector-client} SHAsMarkos Chandras1-2/+17
We should pin the SHAs for Ironic components as part of the regular bifrost bump to ensure maximum long term stability. This also bumps the bifrost SHA to the latest available one. Change-Id: Ia66d8ab6566db58e2ddab5134eb8445bd84d2a8f Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-07xci: scripts: vm: Use LVM backend if availableMarkos Chandras1-5/+21
The CI nodes come with a big secondary mechanical disk which can be attached directly to the clean VM. As such, we split the disk into one logical volume per distribution and flush the qcow2 image to it before attaching it to the VM. If the XCI volume group is not available, then we revert back to using a file-based storage backend. Finally, we hide stdout/stderr from the package installation process to minimize the script output. Change-Id: I11ea82899c411f8a4168e5f23cbbd8187cc891df Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-01xci: scripts: vm: Install additional packages to clean VMMarkos Chandras1-3/+5
Install some additional packages to the clean VM which can be useful during debugging. Moreover, we move the common distribution packages into a variable in order to simplify the code a little bit. Change-Id: I0405820900a15fec730a2bbfc95a1d1f3485bf0f Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-01-23xci: scripts: vm: Retry when timer expiresMarkos Chandras1-12/+4
We should look again for active package managers when the timer expires to avoid locking issues. Change-Id: I88f0299a87befce17d11ba7581f0330979a25345 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-01-08xci: Rename 'nfvi' to 'installer'Markos Chandras1-7/+7
Using 'installer' to describe the tool that will deploy the foundations of a particular XCI scenario is more appropriate than NFVI which normally describes both the physical and virtual resources needed by an NFV deployment. Change-Id: Ib8b1aac58673bf705ce2ff053574fd10cb390d71 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-21xci: OSA: Move all the OSA specific playbooks to the NFVI directoryMarkos Chandras1-7/+7
Move all the playbooks that only make sense for OpenStack-Ansible deployments to the NFVI/OSA directory where they belong. This further disassociates XCI from OSA. Change-Id: Iab8b6dc81d9025a1d85608a98fb1eee0f1c6a69f Signed-off-by: Markos Chandras <mchandras@suse.de> Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2017-12-18Increase ram of XCI VM and change disk cache type for all VMsFatih Degirmenci1-2/+2
These options seem to help with the functest execution within XCI VM. Change-Id: I3b9b6b02fc571feb42543025f8ab179c6db18e0a Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2017-12-15xci: scripts: start-new-vm: Use host-passthrough to use the real host CPUMarkos Chandras1-1/+1
The host-passthrough mode will allow the VM to use all the host cpu features so it should provide the best possible CPU performance. Change-Id: I6ecd0363d046aaefa08f9e97144aa9ca9f1afae5 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-14xci: scripts: vm: Use virtio for disk and network I/OMarkos Chandras1-1/+1
This ensures that the virtio paravirtualization driver is used for both disk and network I/O which should provide the best performance since we no longer need to emulate a real bus or driver. Change-Id: Ic46a38f51b06cb4b848d87d890d23fec022b2f15 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-11xci: Make it easier to use OpenStack-Ansible stable branchesMarkos Chandras1-1/+1
It seems it is desirable for projects to use stable OpenStak-Ansible and OpenStack releases in general. However, merely overriding OPENSTACK_OSA_VERSION doesn't work because role requirements and OpenStack services have to be adapted too. Despite the Documentation warning users about this, seems like nobody remembers to read and do the necessary steps. However, this leads to numerous invalid bug reports due to this stable/master mix. Lets try to help improving this situation by doing all the necessary ground work if we detect that we are using a stable branch. Change-Id: I2ba5d7e7942109973b6a6523aae2c01b65e61fe7 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-11-23Merge "xci: scripts: start-new-vm.sh: Rework fix for parallel pkg mgr instances"Markos Chandras1-23/+18
2017-11-23xci: scripts: start-new-vm.sh: Rework fix for parallel pkg mgr instancesMarkos Chandras1-23/+18
The current way of detecting whether multiple package manager processes are running is not ideal so just wait a random amount of time if we detect that another instance of our package manager command is running. Change-Id: Id94e3ed5293bf223f7250f92525a6e02bb1ea842 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-11-22Add the libvirt DNS to resolv.confVictor Morales1-2/+1
This change adds the gateway into the resolv.conf for helping to resolve some routes faster during the execution of start-new-vm.sh main script. Change-Id: I08f361f5e55589f388e6ff6613d213eb79bbbce9 Signed-off-by: Victor Morales <victor.morales@intel.com>
2017-11-02Bump OpenStack Ansible SHAsMarkos Chandras1-4/+9
We also need to allow root logins when the ansible-hardening role is applied which are disabled by default. Change-Id: I71f881305bafdd884d541cd34796a6fed8eee157 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-17xci: scripts: start-new-vm.sh: Remove empty lineMarkos Chandras1-1/+0
The empty line is wrong and it may cause some issues so drop it. Change-Id: I4120e00df1dd9b7061dc1175e0ea6db8e0a9df88 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-16Merge "xci: scripts: start-new-vm.sh: Make VM specs configurable"Aric Gardner1-4/+4
2017-10-16xci: scripts: start-new-vm.sh: Skip all qcow2 filesMarkos Chandras1-0/+2
We may have qcow2 files around from previous deployments so make sure they are not copied to the newly created VM. Change-Id: I35a8d72355ffff4678b611e09aedc8667f29a7f1 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-13xci: scripts: start-new-vm.sh: Make VM specs configurableMarkos Chandras1-4/+4
We should be able to configure the VM specs so we can fit it in smaller environments. Change-Id: I56850df2f6842b6ef757500fecdc21bd1382b7e7 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-10Merge "xci: scripts: start-new-vm.sh: Do not run multiple package manager ↵Markos Chandras1-7/+29
processes"
2017-10-10Merge "xci: scripts: start-new-vm.sh: Add option to clean VM on failure"Markos Chandras1-0/+15
2017-10-09xci: scripts: start-new-vm.sh: Do not run multiple package manager processesMarkos Chandras1-7/+29
Package managers get upset if multiple instances run in parallel since the last one will fail to aquire some sort of DB locking and fail. As such, lets fix that and only run one package manager process at any time. Change-Id: I4ab13f61eee861ef70cb4115403d0e239ec14308 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-09xci: scripts: start-new-vm.sh: Add option to clean VM on failureMarkos Chandras1-0/+15
Sometimes we may want to remove the clean VM in case of failure. This is mostly true for CI tests because developers do not normally have access to the deployed VMs to debug problems since they can easily recreate the same environment using this script. If they ever want direct access to the slaves then that option can easily be turned off and retrigger the job. In the meantime, this patch allows slaves to free resources when a job fails instead of keeping the broken tests around. Change-Id: I07d56661e45fc64e1ec0db7a359522e4ebd2891a Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-07xci: scripts: start-new-vm.sh: Do not destroy user's configuration fileMarkos Chandras1-16/+20
As we move towards making this script useful for developers, we shouldn't destroy the existing ssh configuration file but rather create a new one just for XCI. This also move the destruction of previous instances sooner, even before building the OS image so we can free resources as soon as possible. Finally, this also uses the hostname to ssh to the new VM instead of its IP to ensure that everything is setup up properly on the physical host. Change-Id: I185befd8943fdb8ebcdb635aefb4d4b03473258b Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-05xci: scripts: build-dib-os.sh: chmod and chown files using sudoMarkos Chandras2-2/+14
Some of the files may be owned by root so we need to ensure that everything is reset properly. Moreover, only clean the image files for the one we are building. Change-Id: I45b78db6715534187f19d9f513e8288fd076cb6b Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-04xci: scripts: start-new-vm.sh: Copy hidden files and directoriesMarkos Chandras1-1/+1
Fix the rsync command to also copy the hidden files and directories. Copying the .git directory allows us to quickly inspect any file changes when investigating bug reports and other problems. Change-Id: I4e6feeea77d5c3b23362a216206c5970b4900169 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-04xci: scripts: vm: Add ability to use cached images for clean VMMarkos Chandras2-38/+109
Make it possible to use the hosted dib images from artifacts.opnfv.org instead of building a new one on every CI run. This way, we can reduce the time it takes to start the actual XCI deployment. Change-Id: Idb1f5e3929cc14502c3f7383a61d2fbd0a6eefaf Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-03xci: scripts: start-new-vm.sh: Set JENKINS_HOME on local runsMarkos Chandras1-0/+5
If we are creating a new virtual machine for local testing, then we should set JENKINS_HOME since some playbooks use this variable to determine if the deployment happens on a CI or not. That script aims to simulate a CI deployment so it should have this variable set. Change-Id: I1165c8b2cd467ba92fcf6f6c178a69b07834894e Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-03xci: scripts: start-new-vm.sh: Do not break on idle SSH connectionsMarkos Chandras1-0/+2
Add ServerAliveInterval and ServerAliveCountMax directives in order to try and keep SSH connections alive. This aims to solve ssh timeouts like the following one on busy hypervisors: packet_write_wait: Connection to UNKNOWN port 0: Broken pipe Change-Id: I58029b41226098d6a44181434ad8653f72384e9d Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-02Merge "xci: scripts: start-new-vm.sh: Use Docker to build OS images"Markos Chandras2-104/+85
2017-10-02xci: scripts: start-new-vm.sh: Use Docker to build OS imagesMarkos Chandras2-104/+85
Use a docker container to build the OS images so we can build images and start new virtual machines on all supported operating systems. This way all developers can now launch a virtual machine to quickly reproduce Jenkins results which should assist with debugging problems. Since the container runs with elevated privileges it's best to ensure that we have exclusive access to devices. Finally, we remove the build-dib-os.sh script which is now part of the container itself. The build image process now becomes more stable since it runs on clean evnironment all the time so the only external factor is the upstream distribution repositories. Change-Id: I6b443192419ee2546a23430f421b152766d16333 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-02xci: scripts: start-new-vm.sh: Fix default test caseMarkos Chandras1-1/+2
Export XCI_FLAVOR to mini to minic the CI. Moreover, fix the script to not use relative paths. Change-Id: I16947c596b8bbff20371b6d345dffd215f76aecc Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-28xci: scripts: build-dib-os.sh: Pin diskimage-builderMarkos Chandras1-1/+1
diskimage-builder does very frequent releases and sometimes things break. There is no particular reason to always fetch the latest version so lets do it in a controlled way after proper testing. Change-Id: Ie9685a9a15fa7fefa1c5ecea49090ac9d92e63ff Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: scripts: start-new-vm.sh: Fix paths for default XCI testMarkos Chandras1-2/+2
The XCI test is located in ~/releng-xci inside the new VM so we need to look there for an existing one or for executing it after the default one has been created and copied to the remote host. Change-Id: Ieb195293da7832bcb5d29c8a28cc3477205f2e5e Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: scripts: start-new-vm.sh: Print dib output when running on CIMarkos Chandras2-3/+7
Until we are able to fetch the dib images from external resources, we need to build them as part of the job. diskimage-builder can sometimes fail so we need to be able to see the log for debug purposes. Change-Id: Iab8bfba08daa7095cf76537f629c8e7bf6330b17 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: scripts: start-new-vm.sh: Use 'unsafe' cache for the clean vm diskMarkos Chandras1-1/+1
This is not a production virtual machine and we don't care if we lose data in case it's killed unexpectedly so we can just ignore the 'flush' commands from the guest. This will also improve the overall I/O in the VMs. Change-Id: Ib486a073f667e22f4a1e65a220ac553874f708cc Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: scripts: start-new-vm.sh: Do not allow multiple build-dib-os processesMarkos Chandras2-3/+13
apt-get is really unhappy when another instance is running and fails with the following error: E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable) E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it? Change-Id: I2fe343fdab8438cb112cce0a4f81c7e3977c55f9 Signed-off-by: Markos Chandras <mchandras@suse.de>