Age | Commit message (Collapse) | Author | Files | Lines |
|
The DEFAULT_XCI_TEST value wasn't checked and it could actually contain
anything leading to all sort of failures so we need to check it against
well known values.
Change-Id: Ib12c944adebfbcb3097b42af98d2bacefbd8374c
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
All clean VM operations should be performed within the root directory of
the releng-xci repo so make sure we are there.
Change-Id: I96f9d91439210faa1ac7f0e43db3992776293e9e
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
The qemu-kvm package was duplicated for opensuse distro in the
start-new-vm.sh script. It doesn't affect its functionality but
reduce the list of additional packages to install.
Change-Id: Id5f24793295d7ccd9b19b64a07b24a1150a780aa
Signed-off-by: Victor Morales <victor.morales@intel.com>
|
|
Change-Id: If53b51fc26d642afa39c5e618afcd23fab395acc
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
|
|
We need to make sure that the local package cache is updated before we
try to install packages. Moreover, from time to time, something in the
repos may be broken so we don't want to break the CI job. As such, make
that step non-fatal as simply send the output to stdout so we can see
what's wrong.
Change-Id: I148e3b5435c604bb0d54339935255740c8ce07c0
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
The XCI deployment script may wipe the PXE directories so we need
to place the OS images after this has happened so we don't download them
every time.
Change-Id: Iccf943bc75592e7f5dddea59dfee14b2b080e7ef
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
We already have images for all 3 distros which we could simply re-use
for all the XCI VM deployments instead of building new ones everytime
with DIB. The images will be copied to the new VM from the cache
directory if they are available otherwise we will simply download them
during the XCI execution phase.
Change-Id: I2a8391650558511668654c6b54a10db316f867a2
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
Change-Id: I21998a31a26a201d767734d574ade8db7fd50c92
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
|
|
Change-Id: Id9c55b45719399c6c78c0d3d8b97926a2907b7f8
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
|
|
If 2 jobs for different distros start on the same host, the
generated xci-vm-config files might be rewritten by both jobs
causing trouble for the other. This change creates ssh config
per distro to prevent potential issues.
Change-Id: If6c0a86fd51bf3ba920e9206849ed1537894c7e1
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
|
|
Change-Id: Ic036a4f5ef8b24b88154f9314957339d3954839a
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
|
|
curl supports the 'retry' argument to handle all sorts of connection
problems so we can use it to retry if we encounter a broken connection.
Change-Id: I3c8afd2be36ee36c5477f05495704bbbee5b9757
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
pgrep will exit with non-zero exit code if it didn't find a matching
process. This breaks the script since we set 'errexit' so we need to
mask it by simply piping it to a 'cat' command.
Change-Id: I80dcf06b597a769242bd59d628bbd8e09f0f199e
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
|
|
|
|
When using the clean VM for the tests, we will end up with 3 levels of
virtualization when functest is executed (1st level = clean vm, 2nd
level = compute node, 3rd level = functest VMs). This makes the functest
VM terribly slow leading to all sorts of random failures. It appears
that KVM can't handle this case, so we need to switch to the QEMU
interpreter instead which is slower but functional. This only affects
the Jenkins jobs, so deployments on baremetal will still use 2 levels of
nested KVM virtualization which should work fine.
Change-Id: If274129fbf347526982ac4cf577d216173eb4d1b
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
In commit I75a8cb8c05957bb205e63210d6cafaf54d989f32 the 'ts' utility
was added in order to print timestamps for deployment. However, this
breaks the exit code since 'pipefail' was not set so every job was
returning 'success'.
Change-Id: Ia5f47ea96a4a5be9ed0664f5c7100accc30cde51
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
From upstream docs:
'The host-model mode is essentially a shortcut to copying host CPU
definition from capabilities XML into domain XML. Since the CPU
definition is copied just before starting a domain, exactly the
same XML can be used on different hosts while still providing
the best guest CPU each host supports'
It's probably safer to use that instead of 'host-passthrough' so
we can get a CPU (both for main VM and the nested ones) that libvirt
understands. Moreover, it's important to present a CPU that the
guest OS understands as well.
Change-Id: I25a8ff0e8635df9804c793d184f048cc86059ce0
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
|
|
According to the docs[1]
"writeback: This mode causes the hypervisor to interact with the disk
image file or block device with neither O_DSYNC nor O_DIRECT semantics.
The host page cache is used and writes are reported to the guest as
completed when they are placed in the host page cache. The normal page
cache management will handle commitment to the storage device.
Additionally, the guest's virtual storage adapter is informed of the
writeback cache, so the guest would be expected to send down flush
commands as needed to manage data integrity. Analogous to a raid
controller with RAM cache."
and
"writeback: This mode informs the guest of the presence of a write
cache, and relies on the guest to send flush commands as needed to
maintain data integrity within its disk image. This is a common
storage design which is completely accounted for within modern file
systems. This mode exposes the guest to data loss in the unlikely case
of a host failure, because there is a window of time between the time
a write is reported as completed, and that write being committed to the
storage device."
"unsafe: This mode is similar to writeback caching except for the
following: the guest flush commands are ignored, nullifying the data
integrity control of these flush commands, and resulting in a higher
risk of data loss because of host failure. The name “unsafe” should
serve as a warning that there is a much higher potential for data
loss because of a host failure than with the other modes. As the
guest terminates, the cached data is flushed at that time."
It's beneficial to use the host page cache to cache I/O from the guest
instead of waiting for data to reach the actual disk device. We do not
normally care about data integrity so data loss is not a problem.
Moreover, we drop the cache configuration from the flavor files since
it's independent of the flavor that's being deployed.
[1] https://www.suse.com/documentation/sles-12/singlehtml/book_virt/book_virt.html#cha.cachemodes
Change-Id: I118ffdf84b1be672185b3eff60fe5d0b5f1a590d
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
This also improves (again!) the check for parallel packages managers
to print some useful messages for debugging.
Change-Id: I7756db80aa3ed42cd3ef37293f7116b5a9c150c1
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
It's useful to know how XCI performs in terms of duration time so
use the bash $SECONDS variable to calculate the total amount of time
it takes for a full XCI deployment. Moreover, use the 'ts' command to
print a timestamp for every command executed. This will help developers
identify tasks which take a lot of time to complete and possibly
optimize them.
Change-Id: I75a8cb8c05957bb205e63210d6cafaf54d989f32
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
The CI nodes come with a big secondary mechanical disk which can be
attached directly to the clean VM. As such, we split the disk into one
logical volume per distribution and flush the qcow2 image to it before
attaching it to the VM. If the XCI volume group is not available, then
we revert back to using a file-based storage backend. Finally, we hide
stdout/stderr from the package installation process to minimize the
script output.
Change-Id: I11ea82899c411f8a4168e5f23cbbd8187cc891df
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
Install some additional packages to the clean VM which can be useful
during debugging. Moreover, we move the common distribution packages
into a variable in order to simplify the code a little bit.
Change-Id: I0405820900a15fec730a2bbfc95a1d1f3485bf0f
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
We should look again for active package managers when the timer expires
to avoid locking issues.
Change-Id: I88f0299a87befce17d11ba7581f0330979a25345
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
These options seem to help with the functest execution
within XCI VM.
Change-Id: I3b9b6b02fc571feb42543025f8ab179c6db18e0a
Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
|
|
The host-passthrough mode will allow the VM to use all the host cpu
features so it should provide the best possible CPU performance.
Change-Id: I6ecd0363d046aaefa08f9e97144aa9ca9f1afae5
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
This ensures that the virtio paravirtualization driver is used for both
disk and network I/O which should provide the best performance since we
no longer need to emulate a real bus or driver.
Change-Id: Ic46a38f51b06cb4b848d87d890d23fec022b2f15
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
|
|
The current way of detecting whether multiple package manager processes
are running is not ideal so just wait a random amount of time if we
detect that another instance of our package manager command is running.
Change-Id: Id94e3ed5293bf223f7250f92525a6e02bb1ea842
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
This change adds the gateway into the resolv.conf for helping
to resolve some routes faster during the execution of start-new-vm.sh
main script.
Change-Id: I08f361f5e55589f388e6ff6613d213eb79bbbce9
Signed-off-by: Victor Morales <victor.morales@intel.com>
|
|
The empty line is wrong and it may cause some issues so drop it.
Change-Id: I4120e00df1dd9b7061dc1175e0ea6db8e0a9df88
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
|
|
We may have qcow2 files around from previous deployments so make sure
they are not copied to the newly created VM.
Change-Id: I35a8d72355ffff4678b611e09aedc8667f29a7f1
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
We should be able to configure the VM specs so we can fit it
in smaller environments.
Change-Id: I56850df2f6842b6ef757500fecdc21bd1382b7e7
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
processes"
|
|
|
|
Package managers get upset if multiple instances run in parallel since
the last one will fail to aquire some sort of DB locking and fail. As
such, lets fix that and only run one package manager process at any
time.
Change-Id: I4ab13f61eee861ef70cb4115403d0e239ec14308
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
Sometimes we may want to remove the clean VM in case of failure. This
is mostly true for CI tests because developers do not normally have access
to the deployed VMs to debug problems since they can easily recreate the
same environment using this script. If they ever want direct access to
the slaves then that option can easily be turned off and retrigger the
job. In the meantime, this patch allows slaves to free resources when a
job fails instead of keeping the broken tests around.
Change-Id: I07d56661e45fc64e1ec0db7a359522e4ebd2891a
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
As we move towards making this script useful for developers, we
shouldn't destroy the existing ssh configuration file but rather
create a new one just for XCI. This also move the destruction of
previous instances sooner, even before building the OS image so we
can free resources as soon as possible. Finally, this also uses the
hostname to ssh to the new VM instead of its IP to ensure that
everything is setup up properly on the physical host.
Change-Id: I185befd8943fdb8ebcdb635aefb4d4b03473258b
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
Some of the files may be owned by root so we need to ensure that
everything is reset properly. Moreover, only clean the image files
for the one we are building.
Change-Id: I45b78db6715534187f19d9f513e8288fd076cb6b
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
Fix the rsync command to also copy the hidden files and directories.
Copying the .git directory allows us to quickly inspect any file
changes when investigating bug reports and other problems.
Change-Id: I4e6feeea77d5c3b23362a216206c5970b4900169
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
Make it possible to use the hosted dib images from artifacts.opnfv.org
instead of building a new one on every CI run. This way, we can reduce
the time it takes to start the actual XCI deployment.
Change-Id: Idb1f5e3929cc14502c3f7383a61d2fbd0a6eefaf
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
If we are creating a new virtual machine for local testing, then we
should set JENKINS_HOME since some playbooks use this variable to
determine if the deployment happens on a CI or not. That script aims
to simulate a CI deployment so it should have this variable set.
Change-Id: I1165c8b2cd467ba92fcf6f6c178a69b07834894e
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
Add ServerAliveInterval and ServerAliveCountMax directives in order
to try and keep SSH connections alive. This aims to solve ssh timeouts
like the following one on busy hypervisors:
packet_write_wait: Connection to UNKNOWN port 0: Broken pipe
Change-Id: I58029b41226098d6a44181434ad8653f72384e9d
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
|
|
Use a docker container to build the OS images so we can build images and
start new virtual machines on all supported operating systems. This way
all developers can now launch a virtual machine to quickly reproduce
Jenkins results which should assist with debugging problems.
Since the container runs with elevated privileges it's best to ensure
that we have exclusive access to devices. Finally, we remove the
build-dib-os.sh script which is now part of the container itself.
The build image process now becomes more stable since it runs on clean
evnironment all the time so the only external factor is the upstream
distribution repositories.
Change-Id: I6b443192419ee2546a23430f421b152766d16333
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
Export XCI_FLAVOR to mini to minic the CI. Moreover, fix the
script to not use relative paths.
Change-Id: I16947c596b8bbff20371b6d345dffd215f76aecc
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
diskimage-builder does very frequent releases and sometimes things
break. There is no particular reason to always fetch the latest version
so lets do it in a controlled way after proper testing.
Change-Id: Ie9685a9a15fa7fefa1c5ecea49090ac9d92e63ff
Signed-off-by: Markos Chandras <mchandras@suse.de>
|
|
The XCI test is located in ~/releng-xci inside the new VM so we need
to look there for an existing one or for executing it after the default
one has been created and copied to the remote host.
Change-Id: Ieb195293da7832bcb5d29c8a28cc3477205f2e5e
Signed-off-by: Markos Chandras <mchandras@suse.de>
|