summaryrefslogtreecommitdiffstats
path: root/bifrost
AgeCommit message (Collapse)AuthorFilesLines
2018-03-13Merge changes from topics 'allow-pass-ansible-arguments', ↵Markos Chandras1-3/+3
'allow-pass-ansible-arguments-force-verify' * changes: xci: OSA: Fix warning about missing inventory file xci: Pass the XCI_PATH variable to all Ansible calls xci: Rename XCI_ANSIBLE_VERBOSITY to XCI_ANSIBLE_PARAMS
2018-03-12remove .ansible directoryPeriyasamy Palanisamy1-9/+10
It's needed to remove .ansible directory which could be a leftovers by some other ansible execution enviroment which would affect XCI deployment. Change-Id: Ic334ef1ca53878721762b23b238fe9a8aeefa0e0 Signed-off-by: Periyasamy Palanisamy <periyasamy.palanisamy@ericsson.com>
2018-03-12xci: Rename XCI_ANSIBLE_VERBOSITY to XCI_ANSIBLE_PARAMSMarkos Chandras1-3/+3
We don't quite need a specific env variable just for Ansible verbosity so we can rename this variable to make it clear that it can be used to pass any Ansible option to XCI. Change-Id: Ie20517d4b563bfc6daeb27848168d36da7014cee Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-10Merge "bifrost: Do not run virsh commands if it's not installed"Markos Chandras1-2/+4
2018-03-10Merge "bifrost: Move DIB configuration variables to bifrost"Markos Chandras1-0/+20
2018-03-09bifrost: Optimize the OPNFV bifrost playbookMarkos Chandras1-31/+11
We could combine several plays into a single one when they are executed to the same set of nodes. This has the added benefit that we avoid all the Ansible warm-up operations when a new play is started. Change-Id: Ibcad07d1b74f1d7cc5a9c60e4b2a6130fb6178fd Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-09bifrost: Do not run virsh commands if it's not installedMarkos Chandras1-2/+4
Check if 'virsh' is available before trying to remove the VMs: Fixes the following warning ./scripts/destroy-env.sh: line 36: virsh: command not found ./scripts/destroy-env.sh: line 37: virsh: command not found ./scripts/destroy-env.sh: line 36: virsh: command not found ./scripts/destroy-env.sh: line 37: virsh: command not found ./scripts/destroy-env.sh: line 36: virsh: command not found ./scripts/destroy-env.sh: line 37: virsh: command not found Change-Id: I9c82c8b8b40462584c6324871284ba12d660ae9d Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-09bifrost: Move DIB configuration variables to bifrostMarkos Chandras1-0/+20
The DIB variables only make sense on bifrost so they should live in the bifrost provision script. Change-Id: I05baca7b9c6889a0d401a9288998991324d270af Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-08bifrost: Ensure OS images are present in the PXE directoryMarkos Chandras2-2/+8
The XCI deployment script may wipe the PXE directories so we need to place the OS images after this has happened so we don't download them every time. Change-Id: Iccf943bc75592e7f5dddea59dfee14b2b080e7ef Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-07bifrost: Use the pre-built XCI OS images instead of dibMarkos Chandras2-3/+49
We already have images for all 3 distros which we could simply re-use for all the XCI VM deployments instead of building new ones everytime with DIB. The images will be copied to the new VM from the cache directory if they are available otherwise we will simply download them during the XCI execution phase. Change-Id: I2a8391650558511668654c6b54a10db316f867a2 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-03-07Bump bifrost sha and set UPPER_CONSTRAINTS_FILEFatih Degirmenci1-0/+3
The lack of UC means that pip is attempting to install libvirt-python 4.1.0, which is failing on Ubuntu due to https://bugs.launchpad.net/openstack-requirements/+bug/1753539. Currently upper-constraints limits libvirt-python to 4.0.0, which does not have this issue. Change-Id: Idc97a0d9928cebafc9167614723236d2584cc93f Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2018-02-23xci: Switch default CPU model to 'host-model'Markos Chandras1-2/+2
From upstream docs: 'The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports' It's probably safer to use that instead of 'host-passthrough' so we can get a CPU (both for main VM and the nested ones) that libvirt understands. Moreover, it's important to present a CPU that the guest OS understands as well. Change-Id: I25a8ff0e8635df9804c793d184f048cc86059ce0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-20xci: Switch VM disk cache to 'unsafe' and use 'iothreads' for I/OMarkos Chandras1-1/+1
According to the docs[1] "writeback: This mode causes the hypervisor to interact with the disk image file or block device with neither O_DSYNC nor O_DIRECT semantics. The host page cache is used and writes are reported to the guest as completed when they are placed in the host page cache. The normal page cache management will handle commitment to the storage device. Additionally, the guest's virtual storage adapter is informed of the writeback cache, so the guest would be expected to send down flush commands as needed to manage data integrity. Analogous to a raid controller with RAM cache." and "writeback: This mode informs the guest of the presence of a write cache, and relies on the guest to send flush commands as needed to maintain data integrity within its disk image. This is a common storage design which is completely accounted for within modern file systems. This mode exposes the guest to data loss in the unlikely case of a host failure, because there is a window of time between the time a write is reported as completed, and that write being committed to the storage device." "unsafe: This mode is similar to writeback caching except for the following: the guest flush commands are ignored, nullifying the data integrity control of these flush commands, and resulting in a higher risk of data loss because of host failure. The name “unsafe” should serve as a warning that there is a much higher potential for data loss because of a host failure than with the other modes. As the guest terminates, the cached data is flushed at that time." It's beneficial to use the host page cache to cache I/O from the guest instead of waiting for data to reach the actual disk device. We do not normally care about data integrity so data loss is not a problem. Moreover, we drop the cache configuration from the flavor files since it's independent of the flavor that's being deployed. [1] https://www.suse.com/documentation/sles-12/singlehtml/book_virt/book_virt.html#cha.cachemodes Change-Id: I118ffdf84b1be672185b3eff60fe5d0b5f1a590d Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-19bifrost: scripts: bifrost-provision.sh: Set defaults for Ironic variablesMarkos Chandras1-0/+5
When running the bifrost jobs for the OPNFV 3rd Party CI, the Ironic variables are not defined and we get the following error: ./scripts/bifrost-provision.sh: line 112: BIFROST_IRONIC_INSPECTOR_VERSION: unbound variable As such, we need to provide reasonable defaults when we are running this script outside of XCI. Change-Id: I9cc3ae1b70494637a85558b9cae99fc1eeaebc74 Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-13bifrost: Add ability to pin Ironic-{,client,inspector,inspector-client} SHAsMarkos Chandras1-1/+5
We should pin the SHAs for Ironic components as part of the regular bifrost bump to ensure maximum long term stability. This also bumps the bifrost SHA to the latest available one. Change-Id: Ia66d8ab6566db58e2ddab5134eb8445bd84d2a8f Signed-off-by: Markos Chandras <mchandras@suse.de>
2018-02-13bifrost: Always deploy Ironic from masterMarkos Chandras1-1/+4
The Ironic components are currently being installed from PIP but we should install them from the git repo instead so we can always try the latest code from master or the pinned version when such functionality is implemented. Change-Id: I0138c54befe52ed9df8bf617b78e063df3cf8750 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-20bifrost: Bump bifrost SHAMarkos Chandras1-62/+0
Bump the bifrost SHA so we can get rid of the testvm.xml.j2 template which has now made it upstream. Change-Id: Ia9cdd999a5702f37125d38e0ba68bb31920d2117 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-18Merge "Increase ram of XCI VM and change disk cache type for all VMs"Fatih Degirmenci1-1/+1
2017-12-18bifrost: Set vCPU model to 'host-passthrough'Markos Chandras2-0/+64
The default CPU model of QEMU GenuineIntel/QEMU Virtual CPU is not the best option for vCPU in terms of performance. As such, lets use the real host CPU. This has also been submitted upstream and the template should be removed once the upstream patch has been merged and the SHA for bifrost is bumped. Change-Id: I23d86933604c290d5650b73f420972d0a2b23dc8 Link: https://review.openstack.org/#/c/528677/ Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-18Increase ram of XCI VM and change disk cache type for all VMsFatih Degirmenci1-1/+1
These options seem to help with the functest execution within XCI VM. Change-Id: I3b9b6b02fc571feb42543025f8ab179c6db18e0a Signed-off-by: Fatih Degirmenci <fdegir@gmail.com>
2017-12-13xci: Reduce noise during bootstrappingMarkos Chandras1-19/+3
There is too much noise when XCI is cloning repositories, installing packages etc so lets make the console output somewhat more readable. Change-Id: I15667e0349f943ad5488daf0e3cea9336c9166d1 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-12-11bifrost: destroy-env.sh: Clean dib images by defaultMarkos Chandras1-2/+2
The destroy-env.sh script is meant to clean the entire environment including the dib images that were built in a previous run. However, this step was only executed if requested exclicitly leaving old artifacts behind. As such, lets switch the default value in order to remove these images as expected. Change-Id: If3b91a621df50931ef3d666737cbc3bbea5d34b9 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-11-21xci: Use the dib pip-and-virtualenv elementMarkos Chandras1-1/+3
The element will ensure that we have a reasonable set of pip and virtualenv packages in place as well as pick the right python version for the distribution. We also make sure that we use python2 everywhere. Change-Id: I8a83af9acffbd8d6d583832956e1685100ec508a Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-20Drop Ironic DB using non-default passwordVictor Morales1-1/+3
The current implementation relays on the fact that the existing mysql deployment the root user can remove has access to the root user without its password. It's possible to drop the Ironic database using the same credentials that were used during its creation. Change-Id: I4b26455e06e5f024d2bd8c42065367838bb4aa0f Signed-off-by: Victor Morales <victor.morales@intel.com>
2017-10-10Merge "xci: Bump bifrost SHA"Markos Chandras1-1/+1
2017-10-06bifrost: bifrost-provision.sh: Always initialize XCI_ANSIBLE_VERBOSITYMarkos Chandras1-0/+3
The XCI_ANSIBLE_VERBOSITY is normally exported by the XCI bootstrapping process. However, sometimes it's necessary to run bifrost-provision.sh by itself so we can debug machine provisioning. As a result of which, we need to initialize this variable properly so we don't fail with 'unbound variable' errors. Change-Id: I81c59cf8eb3369c001e952c2c0a2846ed8d95cdf Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-10-06xci: Bump bifrost SHAMarkos Chandras1-1/+1
The latest bifrost code contains a fix when setting up DNS from a glean configuration drive and also a few virtualenv improvements. This allows us to get rid of all the workarounds we had in the past to switch Ansible versions in the middle of the XCI deployment. Change-Id: If49e290315ec96efdc07d04ff6624439c53aee19 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-27xci: Replace ANSIBLE_VERBOSITY with XCI_ANSIBLE_VERBOSITYMarkos Chandras1-4/+6
The ANSIBLE_VERBOSITY env variable is actually used internally by Ansible and exporting it actually makes Ansible super verbose even if the variable is empty. This feature is going away in the future so we introduce our own XCI_ANSIBLE_VERBOSITY variable which is simply passed to the command line to control verbosity in a proper and deterministic way. Change-Id: I8a32eeb1a86b4cb0a9872a51bbe9f9624d096e39 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-21bifrost: scripts: Look for virtualbmc on bifrost virtualenvMarkos Chandras1-6/+11
Since I94874d600a29247b7826324cc111901995df90dc, bifrost installs pip packages in virtualenv so we need to look there for virtualbmc if it's not currently installed on the host. Change-Id: Ia4c7e8a75c020dc00b920956495831cc77457876 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-21xci: Move VM provisioning to xci-deploy.shMarkos Chandras1-1/+1
The bifrost-provision.sh script will provision machines using Ansible playbooks. If we run this script in an Ansible playbook, that means that we will run Ansible with Ansible and this can only lead to further confusion when debugging issues. As such, since we already have a script to provision machines, lets use it directly from the xci-deploy.sh script. This also reverts 3f04e1fd72b14420788af64b14c35a5f74727f82 which added virtualenv support in bifrost since that causes more problems than it fixes for the time being. Change-Id: Id82b7c06a2af28b66f64f1966227888227ec276f Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-18bifrost: scripts: Do not run env-setup twiceMarkos Chandras1-6/+0
This is already done a few lines above since I94874d600a29247b7826324cc111901995df90dc so there is no need to do it twice. Change-Id: I661516d9aadd265bf6869005718b2f51c57ff84d Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-15xci: env-vars: Bump Ansible version to 2.3.2.0Markos Chandras1-9/+4
Now that bifrost uses virtual env for Ansible we can go ahead and bump the Ansible version to match the one from OSA so we can make use of the latest features. Furthremore, we stop passing ANSIBLE_VERBOSITY to the command line since it's an environment variable so Ansible already knows about it. Change-Id: I52d9a211fc0ced1f7830f6945b2943f35d38109b Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-15bifrost: Use virtual environment for bifrostMarkos Chandras1-1/+16
bifrost forces us to use an old Ansible version which means we can't take advantage of new features in XCI playbooks. As such, we move the entire bifrost step to a virtual environment so we can have a newer version of Ansible on the host. Change-Id: I94874d600a29247b7826324cc111901995df90dc Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-18Merge "prototypes: bifrost: destroy-env: Always destroy XCI VMs"Markos Chandras1-0/+6
2017-08-17prototypes: bifrost: destroy-env: Always destroy XCI VMsMarkos Chandras1-0/+6
It's possible that a previous run failed before registering the VMs to the vbmc. As such, consulting the vbmc to retrieve the VM names will return nothing and no VM will be deleted. As such, it's best to get the VMs from the currently XCI profile and make sure they are really gone. Change-Id: If11265655f42c8a1c7627c9139ac0cbfc002b1fe Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-17Merge "Refactor releng/prototypes/xci to releng-xci/xci in the code"Fatih Degirmenci1-2/+2
2017-08-16Refactor releng/prototypes/xci to releng-xci/xci in the codeDave Urschatz1-2/+2
These changes are needed as part of migrating xci from releng repo to the releng-xci repo. They cover: - README updates - Re-point configuration from releng/prototypes to releng-xci The changes have been tested as follows: - XCI_FLAVOR=aio - OPENSTACK_OSA_VERSION=adfaa66d6108f87599e0595591b39cad2c8fb658 xci: aio has been installed real 107m5.310s user 11m50.180s sys 4m11.152s ubuntu@rack-IRA2-SymKloudBlade4:~/releng-xci/xci$ Change-Id: I8fd86c2442ee3a7c6996eedb6510dbc6eebd30b0 Signed-off-by: Dave Urschatz <dave.urschatz@cengn.ca>
2017-08-16bifrost: scripts: Disable selinux on Centos7Markos Chandras1-0/+10
We are seeing issues with selinux enabled from time to time. Having selinux enabled has proven to be rather unstable and Centos7 not being a gate on upstream bifrost does not help in fully testing this scenario. As such, the best we can do right now is to disable selinux until Centos7 becomes a gate upstream. Change-Id: I93a3414cfebc3c3ef4ac063c8e623f36f615455f Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Do not use tmpfs for dib buildsMarkos Chandras1-0/+2
Using tmpfs seems somewhat unstable. For example the following error is observed on openSUSE hosts ERROR: failed to umount the /tmp/dib_image.RJBbahUd tmpfs As such, we set the dib_notmpfs variable to 'true' which mimics what upstream OpenStack scripts are doing already in project-config:tools/build-image.sh Change-Id: If050ab70303b3556fc1880a956f0ba1505d62050 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Switch Ubuntu targets to Xenial (16.04)Markos Chandras1-2/+2
Switch default target OS to Ubuntu Xenial (16.04) which is also the version which is supported by OpenStack Ansible. Change-Id: I73adc6472b0fb572177e1508cf831b4d3190245c Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Remove OPNFV specific group_vars fileMarkos Chandras1-53/+0
The only reason we had this file was because we had to define our own 'ssh_public_key_path' variable to look in the /root directory since we were running everything as root. Since we are now moving away from this approach, this file is not needed anymore. Change-Id: Ic938c4c8c877c95ce261721dc32fc5147081e236 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Write clouds.yaml file for regular usersMarkos Chandras1-6/+1
This is similar to commit 6769ae9bcbf6 ("prototypes: bifrost: Add keystone roles"). In preparation for running the bifrost jobs as normal user we need to write a clouds.yaml file to users home directory. This is similar to the bifrost upstream testing playbook. Change-Id: Iae4bc73906649707667925280a19282ec5c93998
2017-08-11prototypes: bifrost: Add keystone rolesMarkos Chandras3-2/+19
The os_client_config Ansible module used by all the ironic-* roles depends on the keystone roles so we need to make use of them even though they do not affect the end result. This fixes the following OPNFV CI problem due to not having a clouds.yaml file present to be consumed by the os_client_config Ansible module. fatal: [controller00]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} An exception occurred during task execution. The full traceback is: Traceback (most recent call last): File "/tmp/ansible_ApkCUQ/ansible_module_os_client_config.py", line 75, in <module> main() File "/tmp/ansible_ApkCUQ/ansible_module_os_client_config.py", line 63, in main for cloud in config.get_all_clouds(): File "/usr/lib/python2.7/site-packages/os_client_config/config.py", line 798, in get_all_clouds cloud, region_name=region['name'])) File "/usr/lib/python2.7/site-packages/os_client_config/config.py", line 1071, in get_one_cloud auth_plugin = loader.load_from_options(**config['auth']) File "/usr/lib/python2.7/site-packages/keystoneauth1/loading/base.py", line 162, in load_from_options raise exceptions.MissingRequiredOptions(missing_required) keystoneauth1.exceptions.auth_plugins.MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url fatal: [opnfv]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} Moreover, we cleanup the ~/openstack directory which may contain some bifrost artifacts such as a clouds.yaml file. Finally, we use 'sudo -H' because for the keystone roles we need the HOME variable to be set properly. Change-Id: I45b08bd33dd8ea9505fe10eb4b2b10956b3b683c
2017-08-11xci: Adjust and fix jobs and scriptsFatih Degirmenci1-1/+1
This change - fixes GIT_BASE in jobs - adjusts build blockers - fixes where the flavor vars should be sourced from - sources flavor vars for bifrost periodic jobs - renames the xcimaster node to opnfv to be consistent - removes obsolete xci-provision.sh script - enable additional openstack services including tempest to ensure the deployment is sane Change-Id: Ifdce1da52d68a26c2b87e1bd3b1996ec119d8d90 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: bifrost: Remove ironic templatesMarkos Chandras2-158/+0
This effectively reverts d8235c3c97f4 and 7c1b8bf89d6ce since upstream has now fixed the issues reported in https://review.openstack.org/#/c/450681/ Change-Id: I35588146654d9141bc9357faa3d44e8799c5b706 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Remove /opt/stack from previous deploymentsMarkos Chandras1-0/+3
/opt/stack is being created by bifrost so the destroy-env.sh script should be responsible for cleaning up all the remnants. That way we can use the destroy-env.sh script outside of Jenkins as well. Change-Id: I72d9737cf2446ff3ffc02f195337a6873eceb5c0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11xci: Destroy/undefine only the VMs we createdFatih Degirmenci1-6/+2
Destroying/undefining all the VMs on host might cause issues for the users. This change tries to reduce the possibility of it by removing VMs listed by vbmc. Change-Id: I5134043309722632df68a0800c6403e6f39a8d16 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: bifrost: Fix Ironic drivers for VMsMarkos Chandras1-1/+1
Commit d8235c3c97f4("Set database pool_size in ironic.conf and inspector.conf templates") added a copy of the ironic.conf.j2 template file to workaround an upstream bug but it also changed the default ironic drivers leading to broken introspection. We use virtual BMC for controlling the VMs and this expects ipmi ironic drivers so we fix that. Change-Id: Ibb2cc58738ec1693e136bdce7335cd1b861ec613 Fixes: d8235c3c97f4("Set database pool_size in ironic.conf and inspector.conf templates") Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Set database pool_size in ironic.conf and inspector.conf templatesFatih Degirmenci2-0/+158
min_pool_size and max_pool_size are undefined in ironic.conf and inspector.conf templates, causing issues during ironic-dbsync and ironic-inspector-dbsync. https://build.opnfv.org/ci/job/opnfv-bifrost-verify-trusty-virtual-master/224/console This issue is there for stable/ocata as well. These should be got rid of once the patch on openstack/bifrost is merged. https://review.openstack.org/#/c/450681/ Change-Id: I678835e763022a7168e498047bb65d140895e019 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Configure ansible verbosity based on job typeFatih Degirmenci1-6/+7
Daily jobs will use pinned versions of bifrost and openstack-ansible in order to make sure we use known good versions for them since focus of the daily jobs is OPNFV platform testing. Ansible log can easily become massive depending on the level it is set and it is not so relevant for daily jobs to have it like this due to less probability of dailies failing to provision/deploy. On the other hand, the jobs that verify/test bifrost and openstack-ansible will have max verbosity enabled to troubleshoot these components themselves. New set of jobs will be created running periodically against the HEAD of master for bifrost and openstack-ansible to catch working versions and help us promote them to use for daily platform jobs. Verbosity of these jobs will be enabled like how we are doing for bifrost-verify jobs. Change-Id: Ic9fd2ff418fdaa3ad7fbfc22d8ec2dc1174df65f Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>