summaryrefslogtreecommitdiffstats
path: root/xci/scripts
AgeCommit message (Collapse)AuthorFilesLines
2019-08-31Suppress package manager stdout in utility scriptsFatih Degirmenci1-4/+4
installer-type:kubespray deploy-scenario:k8-calico-nofeature JIRA: XCI-22 Change-Id: I59e072f25185fe1824469d94926b7064a3bb04ff Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-10-26Change variable in start-new-vm.shManuel Buil1-1/+1
Since the patch: https://gerrit.opnfv.org/gerrit/#/c/63173/ The variable BIFROST_USE_PREBUILT_IMAGES has changed to BIFROST_CREATE_IMAGE_VIA_DIB. As jenkins does not trigger testing jobs when editing file in xci/scripts/, this change is done in a separate patch Change-Id: I3bc285936fae5b7514272ca0ad2418b60446e4aa Signed-off-by: Manuel Buil <mbuil@suse.com>
2018-09-07xci: osa: Update OSA SHAs to the HEAD of the Rocky branchMarkos Chandras1-1/+1
Change-Id: I012f829c124d665d7d8d9ebb6b18a72d6d73258f Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-08-31xci: Bump Bifrost components to the HEAD of Rocky releaseMarkos Chandras1-4/+4
The bifrost and ironic SHAs are from the Stein release which is still in early days of development. We should use Rocky SHAs for the time being. However, bifrost from Rocky does not support Ansible > 2.4 so we have to use bifrost from master branch but all the ironic components have to be from the Rocky release. Change-Id: I1acb28e4a292c774d681519517be19a2da230851 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-08-08scripts: start-new-vm: Do not upgrade packages on the hostMarkos Chandras1-7/+4
Upgrading the host could bring undesired side effects so we should not do that as part of CI deployment. Change-Id: Ib776cdf96943e22155c2dd47fdadb3e8a86f6143 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-06-25Merge "xci: vm: start-new-vm: Copy deployment image to clean VM"Markos Chandras1-0/+3
2018-06-22xci: vm: start-new-vm: Copy deployment image to clean VMMarkos Chandras1-0/+3
In order to save us some time and bandwidth we can copy the deployment image to the clean VM in order to be used by bifrost to prepare the virtual machines. Moreover, we also put our key there so we can ssh as the root user. Change-Id: Id21e19aaefde71862c21180129c664c1bf121c32 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-06-06xci: osa: switch back from GitHub to OpenStack OSA reposMarkos Chandras1-4/+0
Switching to github.com for the gitclones did not make much of a difference since we are still seeing randomg SSL failures during git clones. So we can switch back to git://git.openstack.org hoping that the git protocol will be more efficient than https. Moreover some projects are moving away from github mirroring and this breaks our tests. Change-Id: I8188ec7cbfbf16eeb4634bd9a44b12a104ce1059 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-05-11xci: scripts: start-new-vm.sh: Fix ID for openSUSE TumbleweedMarkos Chandras1-1/+1
Recent openSUSE Tumbleweed releases are now using 'opensuse-tumbleweed' as distro ID so we need to take that into consideration in order to be able to use the script on such systems. ~$ source /etc/os-release && echo ${ID} opensuse-tumbleweed Change-Id: Ibdfb4292d53dc4d20376230a4131b212f9492e54 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-22xci: scripts: start-new-vm.sh: Improve do_copy()Victor Morales1-7/+11
The copy function of start-new-vm.sh script only excludes the image of the current OS which is some cases can contain other images. This change excludes any image. Change-Id: I50075cb56e7784d090582fe5d0c11c906773a174 Signed-off-by: Victor Morales <victor.morales@intel.com>
2018-03-17xci: scripts: start-new-vm.sh: Make DEFAULT_XCI_TEST stricterMarkos Chandras1-1/+1
The DEFAULT_XCI_TEST value wasn't checked and it could actually contain anything leading to all sort of failures so we need to check it against well known values. Change-Id: Ib12c944adebfbcb3097b42af98d2bacefbd8374c Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-17xci: scripts: start-new-vm.sh: Ensure we enter the correct root directoryMarkos Chandras1-0/+2
All clean VM operations should be performed within the root directory of the releng-xci repo so make sure we are there. Change-Id: I96f9d91439210faa1ac7f0e43db3992776293e9e Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-16Remove qemu-kvm duplicate packageVictor Morales1-1/+1
The qemu-kvm package was duplicated for opensuse distro in the start-new-vm.sh script. It doesn't affect its functionality but reduce the list of additional packages to install. Change-Id: Id5f24793295d7ccd9b19b64a07b24a1150a780aa Signed-off-by: Victor Morales <victor.morales@intel.com>
2018-03-15Remove opnfv vm from known_hostsFatih Degirmenci1-3/+3
Change-Id: If53b51fc26d642afa39c5e618afcd23fab395acc Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-03-13xci: osa: Switch git repositories to github.comMarkos Chandras1-0/+4
CI sometimes fails with the following error "msg": "fatal: unable to access 'https://git.openstack.org/openstack/dragonflow/': Failed to connect to git.openstack.org port 443: Connection timed out" We assume that the upstream OpenStack mirrors may get very busy at times or the connectivity between the CI and the OpenStack repos is not very stable. As such, lets switch to github.com for the majority of OSA repositories such as the roles and the services which may improve the situation. Change-Id: Ia4668d692077a785c76adeda128eb4cf9f4516a7 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-10xci: scripts: vm: Update package manager cache and ignore errorsMarkos Chandras1-4/+10
We need to make sure that the local package cache is updated before we try to install packages. Moreover, from time to time, something in the repos may be broken so we don't want to break the CI job. As such, make that step non-fatal as simply send the output to stdout so we can see what's wrong. Change-Id: I148e3b5435c604bb0d54339935255740c8ce07c0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-08bifrost: Ensure OS images are present in the PXE directoryMarkos Chandras1-4/+3
The XCI deployment script may wipe the PXE directories so we need to place the OS images after this has happened so we don't download them every time. Change-Id: Iccf943bc75592e7f5dddea59dfee14b2b080e7ef Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-07bifrost: Use the pre-built XCI OS images instead of dibMarkos Chandras1-3/+6
We already have images for all 3 distros which we could simply re-use for all the XCI VM deployments instead of building new ones everytime with DIB. The images will be copied to the new VM from the cache directory if they are available otherwise we will simply download them during the XCI execution phase. Change-Id: I2a8391650558511668654c6b54a10db316f867a2 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-01Remove opnfv vm from known_hostsFatih Degirmenci1-0/+4
Change-Id: I21998a31a26a201d767734d574ade8db7fd50c92 Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-03-01Do not fail the if virsh command failsFatih Degirmenci1-2/+2
Change-Id: Id9c55b45719399c6c78c0d3d8b97926a2907b7f8 Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-02-28Ensure the ssh config files are created per distroFatih Degirmenci1-4/+7
If 2 jobs for different distros start on the same host, the generated xci-vm-config files might be rewritten by both jobs causing trouble for the other. This change creates ssh config per distro to prevent potential issues. Change-Id: If6c0a86fd51bf3ba920e9206849ed1537894c7e1 Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-02-28Fix the proxy command and hostname for opnfv vm in ssh configFatih Degirmenci1-1/+2
Change-Id: Ic036a4f5ef8b24b88154f9314957339d3954839a Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-02-27xci: script: start-new-vm: Switch to using 'curl' instead of 'wget'Markos Chandras1-3/+4
curl supports the 'retry' argument to handle all sorts of connection problems so we can use it to retry if we encounter a broken connection. Change-Id: I3c8afd2be36ee36c5477f05495704bbbee5b9757 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-27xci: scripts: start-new-vm: Mask the pgrep exit codeMarkos Chandras1-1/+1
pgrep will exit with non-zero exit code if it didn't find a matching process. This breaks the script since we set 'errexit' so we need to mask it by simply piping it to a 'cat' command. Change-Id: I80dcf06b597a769242bd59d628bbd8e09f0f199e Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-27Merge "xci: scripts: vm: start-new-vm.sh: Re-arrange debug messages"Markos Chandras1-2/+9
2018-02-26Merge "xci: Switch default CPU model to 'host-model'"Markos Chandras1-1/+1
2018-02-23xci: scripts: start-new-vm.sh: Disable nested virtualization in clean VMMarkos Chandras1-0/+3
When using the clean VM for the tests, we will end up with 3 levels of virtualization when functest is executed (1st level = clean vm, 2nd level = compute node, 3rd level = functest VMs). This makes the functest VM terribly slow leading to all sorts of random failures. It appears that KVM can't handle this case, so we need to switch to the QEMU interpreter instead which is slower but functional. This only affects the Jenkins jobs, so deployments on baremetal will still use 2 levels of nested KVM virtualization which should work fine. Change-Id: If274129fbf347526982ac4cf577d216173eb4d1b Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-23xci: scripts: start-new-vm: Set pipefail optionMarkos Chandras1-0/+1
In commit I75a8cb8c05957bb205e63210d6cafaf54d989f32 the 'ts' utility was added in order to print timestamps for deployment. However, this breaks the exit code since 'pipefail' was not set so every job was returning 'success'. Change-Id: Ia5f47ea96a4a5be9ed0664f5c7100accc30cde51 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-23xci: Switch default CPU model to 'host-model'Markos Chandras1-1/+1
From upstream docs: 'The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports' It's probably safer to use that instead of 'host-passthrough' so we can get a CPU (both for main VM and the nested ones) that libvirt understands. Moreover, it's important to present a CPU that the guest OS understands as well. Change-Id: I25a8ff0e8635df9804c793d184f048cc86059ce0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-21Merge "xci: xci-deploy.sh: Calculate and print the total deployment time"Markos Chandras1-1/+1
2018-02-20xci: Switch VM disk cache to 'unsafe' and use 'iothreads' for I/OMarkos Chandras1-2/+2
According to the docs[1] "writeback: This mode causes the hypervisor to interact with the disk image file or block device with neither O_DSYNC nor O_DIRECT semantics. The host page cache is used and writes are reported to the guest as completed when they are placed in the host page cache. The normal page cache management will handle commitment to the storage device. Additionally, the guest's virtual storage adapter is informed of the writeback cache, so the guest would be expected to send down flush commands as needed to manage data integrity. Analogous to a raid controller with RAM cache." and "writeback: This mode informs the guest of the presence of a write cache, and relies on the guest to send flush commands as needed to maintain data integrity within its disk image. This is a common storage design which is completely accounted for within modern file systems. This mode exposes the guest to data loss in the unlikely case of a host failure, because there is a window of time between the time a write is reported as completed, and that write being committed to the storage device." "unsafe: This mode is similar to writeback caching except for the following: the guest flush commands are ignored, nullifying the data integrity control of these flush commands, and resulting in a higher risk of data loss because of host failure. The name “unsafe” should serve as a warning that there is a much higher potential for data loss because of a host failure than with the other modes. As the guest terminates, the cached data is flushed at that time." It's beneficial to use the host page cache to cache I/O from the guest instead of waiting for data to reach the actual disk device. We do not normally care about data integrity so data loss is not a problem. Moreover, we drop the cache configuration from the flavor files since it's independent of the flavor that's being deployed. [1] https://www.suse.com/documentation/sles-12/singlehtml/book_virt/book_virt.html#cha.cachemodes Change-Id: I118ffdf84b1be672185b3eff60fe5d0b5f1a590d Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-20xci: scripts: vm: start-new-vm.sh: Re-arrange debug messagesMarkos Chandras1-2/+9
This also improves (again!) the check for parallel packages managers to print some useful messages for debugging. Change-Id: I7756db80aa3ed42cd3ef37293f7116b5a9c150c1 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-19xci: xci-deploy.sh: Calculate and print the total deployment timeMarkos Chandras1-1/+1
It's useful to know how XCI performs in terms of duration time so use the bash $SECONDS variable to calculate the total amount of time it takes for a full XCI deployment. Moreover, use the 'ts' command to print a timestamp for every command executed. This will help developers identify tasks which take a lot of time to complete and possibly optimize them. Change-Id: I75a8cb8c05957bb205e63210d6cafaf54d989f32 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-13bifrost: Add ability to pin Ironic-{,client,inspector,inspector-client} SHAsMarkos Chandras1-2/+17
We should pin the SHAs for Ironic components as part of the regular bifrost bump to ensure maximum long term stability. This also bumps the bifrost SHA to the latest available one. Change-Id: Ia66d8ab6566db58e2ddab5134eb8445bd84d2a8f Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-07xci: scripts: vm: Use LVM backend if availableMarkos Chandras1-5/+21
The CI nodes come with a big secondary mechanical disk which can be attached directly to the clean VM. As such, we split the disk into one logical volume per distribution and flush the qcow2 image to it before attaching it to the VM. If the XCI volume group is not available, then we revert back to using a file-based storage backend. Finally, we hide stdout/stderr from the package installation process to minimize the script output. Change-Id: I11ea82899c411f8a4168e5f23cbbd8187cc891df Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-01xci: scripts: vm: Install additional packages to clean VMMarkos Chandras1-3/+5
Install some additional packages to the clean VM which can be useful during debugging. Moreover, we move the common distribution packages into a variable in order to simplify the code a little bit. Change-Id: I0405820900a15fec730a2bbfc95a1d1f3485bf0f Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-01-23xci: scripts: vm: Retry when timer expiresMarkos Chandras1-12/+4
We should look again for active package managers when the timer expires to avoid locking issues. Change-Id: I88f0299a87befce17d11ba7581f0330979a25345 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-01-08xci: Rename 'nfvi' to 'installer'Markos Chandras1-7/+7
Using 'installer' to describe the tool that will deploy the foundations of a particular XCI scenario is more appropriate than NFVI which normally describes both the physical and virtual resources needed by an NFV deployment. Change-Id: Ib8b1aac58673bf705ce2ff053574fd10cb390d71 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-21xci: OSA: Move all the OSA specific playbooks to the NFVI directoryMarkos Chandras1-7/+7
Move all the playbooks that only make sense for OpenStack-Ansible deployments to the NFVI/OSA directory where they belong. This further disassociates XCI from OSA. Change-Id: Iab8b6dc81d9025a1d85608a98fb1eee0f1c6a69f Signed-off-by: Markos Chandras <mchandras@suse.de> Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2017-12-18Increase ram of XCI VM and change disk cache type for all VMsFatih Degirmenci1-2/+2
These options seem to help with the functest execution within XCI VM. Change-Id: I3b9b6b02fc571feb42543025f8ab179c6db18e0a Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2017-12-15xci: scripts: start-new-vm: Use host-passthrough to use the real host CPUMarkos Chandras1-1/+1
The host-passthrough mode will allow the VM to use all the host cpu features so it should provide the best possible CPU performance. Change-Id: I6ecd0363d046aaefa08f9e97144aa9ca9f1afae5 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-14xci: scripts: vm: Use virtio for disk and network I/OMarkos Chandras1-1/+1
This ensures that the virtio paravirtualization driver is used for both disk and network I/O which should provide the best performance since we no longer need to emulate a real bus or driver. Change-Id: Ic46a38f51b06cb4b848d87d890d23fec022b2f15 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-11xci: Make it easier to use OpenStack-Ansible stable branchesMarkos Chandras1-1/+1
It seems it is desirable for projects to use stable OpenStak-Ansible and OpenStack releases in general. However, merely overriding OPENSTACK_OSA_VERSION doesn't work because role requirements and OpenStack services have to be adapted too. Despite the Documentation warning users about this, seems like nobody remembers to read and do the necessary steps. However, this leads to numerous invalid bug reports due to this stable/master mix. Lets try to help improving this situation by doing all the necessary ground work if we detect that we are using a stable branch. Change-Id: I2ba5d7e7942109973b6a6523aae2c01b65e61fe7 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-11-23Merge "xci: scripts: start-new-vm.sh: Rework fix for parallel pkg mgr instances"Markos Chandras1-23/+18
2017-11-23xci: scripts: start-new-vm.sh: Rework fix for parallel pkg mgr instancesMarkos Chandras1-23/+18
The current way of detecting whether multiple package manager processes are running is not ideal so just wait a random amount of time if we detect that another instance of our package manager command is running. Change-Id: Id94e3ed5293bf223f7250f92525a6e02bb1ea842 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-11-22Add the libvirt DNS to resolv.confVictor Morales1-2/+1
This change adds the gateway into the resolv.conf for helping to resolve some routes faster during the execution of start-new-vm.sh main script. Change-Id: I08f361f5e55589f388e6ff6613d213eb79bbbce9 Signed-off-by: Victor Morales <victor.morales@intel.com>
2017-11-02Bump OpenStack Ansible SHAsMarkos Chandras1-4/+9
We also need to allow root logins when the ansible-hardening role is applied which are disabled by default. Change-Id: I71f881305bafdd884d541cd34796a6fed8eee157 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-17xci: scripts: start-new-vm.sh: Remove empty lineMarkos Chandras1-1/+0
The empty line is wrong and it may cause some issues so drop it. Change-Id: I4120e00df1dd9b7061dc1175e0ea6db8e0a9df88 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-16Merge "xci: scripts: start-new-vm.sh: Make VM specs configurable"Aric Gardner1-4/+4
2017-10-16xci: scripts: start-new-vm.sh: Skip all qcow2 filesMarkos Chandras1-0/+2
We may have qcow2 files around from previous deployments so make sure they are not copied to the newly created VM. Change-Id: I35a8d72355ffff4678b611e09aedc8667f29a7f1 Signed-off-by: Markos Chandras <mchandras@suse.de>