summaryrefslogtreecommitdiffstats
path: root/bifrost
AgeCommit message (Collapse)AuthorFilesLines
2017-09-15xci: env-vars: Bump Ansible version to 2.3.2.0Markos Chandras1-9/+4
Now that bifrost uses virtual env for Ansible we can go ahead and bump the Ansible version to match the one from OSA so we can make use of the latest features. Furthremore, we stop passing ANSIBLE_VERBOSITY to the command line since it's an environment variable so Ansible already knows about it. Change-Id: I52d9a211fc0ced1f7830f6945b2943f35d38109b Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-09-15bifrost: Use virtual environment for bifrostMarkos Chandras1-1/+16
bifrost forces us to use an old Ansible version which means we can't take advantage of new features in XCI playbooks. As such, we move the entire bifrost step to a virtual environment so we can have a newer version of Ansible on the host. Change-Id: I94874d600a29247b7826324cc111901995df90dc Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-18Merge "prototypes: bifrost: destroy-env: Always destroy XCI VMs"Markos Chandras1-0/+6
2017-08-17prototypes: bifrost: destroy-env: Always destroy XCI VMsMarkos Chandras1-0/+6
It's possible that a previous run failed before registering the VMs to the vbmc. As such, consulting the vbmc to retrieve the VM names will return nothing and no VM will be deleted. As such, it's best to get the VMs from the currently XCI profile and make sure they are really gone. Change-Id: If11265655f42c8a1c7627c9139ac0cbfc002b1fe Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-17Merge "Refactor releng/prototypes/xci to releng-xci/xci in the code"Fatih Degirmenci1-2/+2
2017-08-16Refactor releng/prototypes/xci to releng-xci/xci in the codeDave Urschatz1-2/+2
These changes are needed as part of migrating xci from releng repo to the releng-xci repo. They cover: - README updates - Re-point configuration from releng/prototypes to releng-xci The changes have been tested as follows: - XCI_FLAVOR=aio - OPENSTACK_OSA_VERSION=adfaa66d6108f87599e0595591b39cad2c8fb658 xci: aio has been installed real 107m5.310s user 11m50.180s sys 4m11.152s ubuntu@rack-IRA2-SymKloudBlade4:~/releng-xci/xci$ Change-Id: I8fd86c2442ee3a7c6996eedb6510dbc6eebd30b0 Signed-off-by: Dave Urschatz <dave.urschatz@cengn.ca>
2017-08-16bifrost: scripts: Disable selinux on Centos7Markos Chandras1-0/+10
We are seeing issues with selinux enabled from time to time. Having selinux enabled has proven to be rather unstable and Centos7 not being a gate on upstream bifrost does not help in fully testing this scenario. As such, the best we can do right now is to disable selinux until Centos7 becomes a gate upstream. Change-Id: I93a3414cfebc3c3ef4ac063c8e623f36f615455f Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Do not use tmpfs for dib buildsMarkos Chandras1-0/+2
Using tmpfs seems somewhat unstable. For example the following error is observed on openSUSE hosts ERROR: failed to umount the /tmp/dib_image.RJBbahUd tmpfs As such, we set the dib_notmpfs variable to 'true' which mimics what upstream OpenStack scripts are doing already in project-config:tools/build-image.sh Change-Id: If050ab70303b3556fc1880a956f0ba1505d62050 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Switch Ubuntu targets to Xenial (16.04)Markos Chandras1-2/+2
Switch default target OS to Ubuntu Xenial (16.04) which is also the version which is supported by OpenStack Ansible. Change-Id: I73adc6472b0fb572177e1508cf831b4d3190245c Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Remove OPNFV specific group_vars fileMarkos Chandras1-53/+0
The only reason we had this file was because we had to define our own 'ssh_public_key_path' variable to look in the /root directory since we were running everything as root. Since we are now moving away from this approach, this file is not needed anymore. Change-Id: Ic938c4c8c877c95ce261721dc32fc5147081e236 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Write clouds.yaml file for regular usersMarkos Chandras1-6/+1
This is similar to commit 6769ae9bcbf6 ("prototypes: bifrost: Add keystone roles"). In preparation for running the bifrost jobs as normal user we need to write a clouds.yaml file to users home directory. This is similar to the bifrost upstream testing playbook. Change-Id: Iae4bc73906649707667925280a19282ec5c93998
2017-08-11prototypes: bifrost: Add keystone rolesMarkos Chandras3-2/+19
The os_client_config Ansible module used by all the ironic-* roles depends on the keystone roles so we need to make use of them even though they do not affect the end result. This fixes the following OPNFV CI problem due to not having a clouds.yaml file present to be consumed by the os_client_config Ansible module. fatal: [controller00]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} An exception occurred during task execution. The full traceback is: Traceback (most recent call last): File "/tmp/ansible_ApkCUQ/ansible_module_os_client_config.py", line 75, in <module> main() File "/tmp/ansible_ApkCUQ/ansible_module_os_client_config.py", line 63, in main for cloud in config.get_all_clouds(): File "/usr/lib/python2.7/site-packages/os_client_config/config.py", line 798, in get_all_clouds cloud, region_name=region['name'])) File "/usr/lib/python2.7/site-packages/os_client_config/config.py", line 1071, in get_one_cloud auth_plugin = loader.load_from_options(**config['auth']) File "/usr/lib/python2.7/site-packages/keystoneauth1/loading/base.py", line 162, in load_from_options raise exceptions.MissingRequiredOptions(missing_required) keystoneauth1.exceptions.auth_plugins.MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url fatal: [opnfv]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} Moreover, we cleanup the ~/openstack directory which may contain some bifrost artifacts such as a clouds.yaml file. Finally, we use 'sudo -H' because for the keystone roles we need the HOME variable to be set properly. Change-Id: I45b08bd33dd8ea9505fe10eb4b2b10956b3b683c
2017-08-11xci: Adjust and fix jobs and scriptsFatih Degirmenci1-1/+1
This change - fixes GIT_BASE in jobs - adjusts build blockers - fixes where the flavor vars should be sourced from - sources flavor vars for bifrost periodic jobs - renames the xcimaster node to opnfv to be consistent - removes obsolete xci-provision.sh script - enable additional openstack services including tempest to ensure the deployment is sane Change-Id: Ifdce1da52d68a26c2b87e1bd3b1996ec119d8d90 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: bifrost: Remove ironic templatesMarkos Chandras2-158/+0
This effectively reverts d8235c3c97f4 and 7c1b8bf89d6ce since upstream has now fixed the issues reported in https://review.openstack.org/#/c/450681/ Change-Id: I35588146654d9141bc9357faa3d44e8799c5b706 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Remove /opt/stack from previous deploymentsMarkos Chandras1-0/+3
/opt/stack is being created by bifrost so the destroy-env.sh script should be responsible for cleaning up all the remnants. That way we can use the destroy-env.sh script outside of Jenkins as well. Change-Id: I72d9737cf2446ff3ffc02f195337a6873eceb5c0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11xci: Destroy/undefine only the VMs we createdFatih Degirmenci1-6/+2
Destroying/undefining all the VMs on host might cause issues for the users. This change tries to reduce the possibility of it by removing VMs listed by vbmc. Change-Id: I5134043309722632df68a0800c6403e6f39a8d16 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: bifrost: Fix Ironic drivers for VMsMarkos Chandras1-1/+1
Commit d8235c3c97f4("Set database pool_size in ironic.conf and inspector.conf templates") added a copy of the ironic.conf.j2 template file to workaround an upstream bug but it also changed the default ironic drivers leading to broken introspection. We use virtual BMC for controlling the VMs and this expects ipmi ironic drivers so we fix that. Change-Id: Ibb2cc58738ec1693e136bdce7335cd1b861ec613 Fixes: d8235c3c97f4("Set database pool_size in ironic.conf and inspector.conf templates") Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Set database pool_size in ironic.conf and inspector.conf templatesFatih Degirmenci2-0/+158
min_pool_size and max_pool_size are undefined in ironic.conf and inspector.conf templates, causing issues during ironic-dbsync and ironic-inspector-dbsync. https://build.opnfv.org/ci/job/opnfv-bifrost-verify-trusty-virtual-master/224/console This issue is there for stable/ocata as well. These should be got rid of once the patch on openstack/bifrost is merged. https://review.openstack.org/#/c/450681/ Change-Id: I678835e763022a7168e498047bb65d140895e019 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Configure ansible verbosity based on job typeFatih Degirmenci1-6/+7
Daily jobs will use pinned versions of bifrost and openstack-ansible in order to make sure we use known good versions for them since focus of the daily jobs is OPNFV platform testing. Ansible log can easily become massive depending on the level it is set and it is not so relevant for daily jobs to have it like this due to less probability of dailies failing to provision/deploy. On the other hand, the jobs that verify/test bifrost and openstack-ansible will have max verbosity enabled to troubleshoot these components themselves. New set of jobs will be created running periodically against the HEAD of master for bifrost and openstack-ansible to catch working versions and help us promote them to use for daily platform jobs. Verbosity of these jobs will be enabled like how we are doing for bifrost-verify jobs. Change-Id: Ic9fd2ff418fdaa3ad7fbfc22d8ec2dc1174df65f Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Merge bifrost provisioning scripts and renameFatih Degirmenci2-174/+24
Jobs are adjusted as well. Change-Id: I0bc5bdda37277053d81d7da394e0156902da817a Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Adjust how bifrost inventory files are specifiedFatih Degirmenci2-7/+10
First step towards creating env-vars files based on the use of these scripts so the scripts can be merged into one. Change-Id: I4ee1d9af520aa684a5367a2645d709835bc4a5a1 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Make it possible to select component versionsFatih Degirmenci1-4/+2
By replacing the branch with version, we now can specify sha1s per project; opnfv/releng, openstack-ansible, and bifrost. If nothing is specified manually (or via promotion), the defaults will be set to branches, master and stable/ocata. Change also includes further logging. Change-Id: Iae0f8cc60391caeb0f2735610050007014374a70 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Updates to releng/bifrost to make it work on JenkinsFatih Degirmenci4-9/+68
- get rid of BAREMETAL_DATA_FILE and use BIFROST_INVENTORY_SOURCE so it works both for master and ocata for osa-bifrost. - set BIFROST_INVENTORY_SOURCE file according to branch bifrost is executed from for osa-bifrost. - explicitly set what the SSH public key file is as ansible copies the wrong public key to nodes if bifrost is executed on Jenkins using sudo. - set branches if they are not set so the scripts can be used manually as well. - rename jumphost to xcimaster so people do not mix it with actual jumphost located in Pharos PODs. Change-Id: Iff7631fa99816ad75316b62c5ac20714f67cd86a Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Remove obsolete bifrost dib ELEMENTS_PATHFatih Degirmenci2-6/+0
Change-Id: I7baa0db9c8193e4041883d5997529df98919c7af Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Enable VM provisioning using bifrost for daily runsFatih Degirmenci4-11/+14
- create script to provision VMs (log upload is not enabled yet) - remove leftover inventory files (baremetal.csv, baremetal.json) - set BIFROST_INVENTORY_SOURCE as needed by stable/ocata - rename playbook used during bifrost and adjust scripts using it - remove puppet-infracloud stuff Change-Id: I1c4412d0725c00ab9a1b3497e358d33998b4e7e6 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: bifrost: destroy-env: Delete nodes from vbmcMarkos Chandras1-0/+4
Upstream commit 12376b958e76 ("Use virtualbmc for virtual BM nodes") moved the VM IPMI management from the ssh drivers to the vbmc one. As a result, we need to clear all the nodes from vbmc during cleanup. Change-Id: I2a4e7233874c4fad8df525c8f7b526df17a8211f Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11bifrost: Create bifrost script for openstack-ansiblewutianwei2-1/+143
This change creates a separate script to provision 6 VMs for HA to use for openstack-ansible. - create 1 jumphost, 3 controller, 2 compute - set the mumber of vCPU to 8 - use 16GB RAM per VM - use xenial and add additional packages required by openstack-ansible Change-Id: I858340c5bd13a1cd35cf4334ab2b8c362c5c5e7f Signed-off-by: wutianwei <wutianwei1@huawei.com> Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: bifrost: Fix ELEMENTS_PATH for latest diskimage-builder releaseMarkos Chandras2-2/+2
Upcoming diskimage-builder version (2.0.0) changed the location on where dib elements are installed from /usr/share/diskimage-builder to /usr/lib/$python/diskimage_builder/. However, base elements are included by default so fix the script to set ELEMENTS_PATH to the path of our out-of-tree elements. Change-Id: Ib5c4e1465a67a819c9d495b0e0a64addeb0294cb Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Switch to JSON for inventoryMarkos Chandras1-2/+4
Upstream commit 3aaed64e88e3 ("Use JSON baremetal data in testing") switched from CSV to JSON for the inventory file in order to support the *_ipmitool drivers and virtualbmc. This commit mimics the upstream one. Depends-On: Ib5c4e1465a67a819c9d495b0e0a64addeb0294cb Change-Id: If66810db445cf997c5eb61090815abe186e002fa Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Collect logs even if deployment failsMarkos Chandras1-2/+5
Move the collect-test-info.sh script to the 'exit' trap so we can collect whatever logs we can even if the bifrost deployment failed. Upstream change: https://review.openstack.org/433532 Change-Id: Icfbec01d894ff33e83730a40a09dc5540008dbaa Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11bifrost: Destroy all VMs from the slave nodesMarkos Chandras1-6/+6
New patchsets may introduce (or remove) testing VMs so make sure the script destroys all of them Change-Id: Ic96142823b9a8b044306327ce3454fbdc6fa5eb4 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Cleanup old VM console logsMarkos Chandras1-1/+1
The VM logs may not be rotated by logrotate so they are accumulated over time. Make sure they are gone when we start a new job. Change-Id: Iecf7e53015740db98c17a42612da7f44f3c4a676 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Drop the entire ironic databaseMarkos Chandras1-5/+2
There is no reason to only drop a few tables from it, so just drop the entire database. Change-Id: Iea9757fc5ccd85bc223d3c86d6a4c3ecf65fa130 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11bifrost: Use 'unsafe' disk cache for test VMsMarkos Chandras1-0/+1
The VMs which are created by bifrost are not meant to be used in production so improve the overall CI job performance by switching to the 'unsafe' disk cache mode. Make it also the default mode when using bifrost outside of the Jenkins job. Change-Id: If88f6c9395617a4aa860f8d00e22abfdd18d2922 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11jjb: infra: prototypes: bifrost: Ignore mysql and ironic failuresMarkos Chandras1-8/+10
On brand new hosts ironic may not be installed yet so attemping to restart ironic services will lead to failures. Similarly, mysql may also not be installed so do not try to wipe the tables. Change-Id: Ic77d9358aeecb11f5c076a47cfea18ee4f0dff9f Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Increase default provision timeout to 3600 secondsMarkos Chandras1-1/+1
Sometimes, the current '2400 seconds' timeout is not enough to all 3 VMs to become reachable especially when the qcow2 images are becoming bigger and bigger. This leads to spurious CI failures. Increase the timeout to 3600 seconds which should be enough for VMs boot properly Change-Id: I06942952667df28341e5ce43d30619fe2e30326b Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11bifrost: test-bifrost-deployment.sh: Use the new 'openssh-server' dib elementMarkos Chandras1-3/+3
This is a follow-up for b0a9b75e7481a24676c9488f17583c13b8b8026e ("Make use of the new 'openssh-server' dib element"). Remove the openssh-server package from the default packages and use the 'openssh-server' element instead. Change-Id: Ie14cdf93769ebefcae00f35d9e1ac149a31e7711 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11bifrost: Make use of the new 'openssh-server' dib elementMarkos Chandras2-1/+5
Make use of the new 'openssh-server' dib element to achieve a consistent behavior of ssh server across all distributions. Change-Id: Ib10066a244fe44530969702cd465cdd79454b8fe Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11bifrost: test-bifrost-deployment: Allow overriding VM hardware optionsMarkos Chandras1-3/+3
Make VM disk, cpu and memory settings configurable to allow less powerful hosts to host the bifrost VMs. Change-Id: If819451536dd518690f3808c55de940169b90346 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Start using centos7 instead of centos-minimalYolanda Robla Mota1-1/+1
Current builds with centos-minimal are failing on 7, so start using centos7 element instead. Change-Id: I80d986fd7613e1807a119447a01c62e6f8a93ddf Signed-Off-By: Yolanda Robla <yroblamo@redhat.com>
2017-08-11Add step to customize create bridge templateYolanda Robla Mota1-5/+8
Change-Id: I0f4e7353773291e810f70059e4a015ac641bd2c8 Signed-Off-By: Yolanda Robla <yroblamo@redhat.com>
2017-08-11playbooks: Do not serialize ironic deploymentMarkos Chandras1-4/+4
[This is similar to upstream https://review.openstack.org/389079] This somewhat reverts 0a82e3d8d29f85229e1ed29d90de47346bc1d9b1 ("Serialize localhost actions during final host deployment"). The only point where we want to serialize the steps is when accessing common files on the host. In the bifrost-deploy-nodes-dynamic role, all tasks are accessing files based on the 'inventory_hostname' variable, which means there is no conflict there and tasks on these role can run in parallel. This allows bifrost to provision machines in parallel during pxe boot which speeds up the entire operation. Change-Id: I64b1f2832c333c89e33e8f9d19f03e4d653d74b4 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Ignore dnsmasq service failuresMarkos Chandras1-1/+1
If this is the first time we run bifrost on a host or if we haven't run bifrost since the last reboot, restarting the dnsmasq service will fail because of conflicts with the dnsmasq process running by libvirt. This is later on 'fixed' by bifrost by killing the one from libvirt. Make sure that this spurious failure will not prevent the rest of the cleanup process to move on. Change-Id: Iebf37881570c27cb07cea3cecdb0fa8ac3fc511b Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11bifrost: Serialize localhost actions during final host deploymentMarkos Chandras1-3/+8
Several tasks in the final playbooks run locally and there is a chance to encounter race conditions when multiple tasks run in parallel and all access or modify the same local files. We fix this by splitting the final steps into multiple plays and serializing those that run locally. Change-Id: I38c3465d9c4714d9db772f9ff9e32ec1d4a91db8 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Set the wait_timeout variable to the deployment scriptMarkos Chandras2-5/+3
There is no point in us keeping our own copy of the upstream bifrost-prepare-for-test-dynamic playbook just for overriding the wait_timeout variable. Delete the duplicate role and move the variable definition to the deployment script instead. Change-Id: Ib8066fefe80e9a3190d878eb5dba7b8d86a85e07 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11bifrost: Refactor bifrost jobsFatih Degirmenci1-2/+3
This change - Creates job templates to verify opnfv/bifrost patches - Introduces type in order to prepare verification on baremetal - Fixes git cloning so the job shows changes for the corresponding project - Gets rid of unnecessary slave parameters from slave-params.yml - Fixes build blocking - Updates the bifrost-verify.sh script and includes in shell so we don't have to clone releng repo separately - Updates destroy-env.sh to create /httpboot and /tftpboot with permissions 755 - Removes daily infra-bifrost job and scripts as we don't use them at the moment Change-Id: Icb5a543628d3321bfe4e549ab92c67443ac578d8 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11bifrost: Get rid of httpboot and tftpboot directoriesFatih Degirmenci1-2/+2
Change-Id: If2af5177e8614651664c7de9cde3e681ec2bb43c Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: bifrost: README: Run the deployment script using sudoMarkos Chandras1-1/+1
It's necessary to run the deployment script using 'sudo' so that exported variables within the script are preserved in playbooks once we gain elevated privileges. Change-Id: I824b44199d57b6acf14804c3ab1825d7aa80da4e Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Check we are running the destroy-env.sh as rootMarkos Chandras2-6/+14
Nearly all operations in the destroy-env.sh script need to be executed as root so make sure that's the case before we try to do anything. This also fixes the dnsmasq.leases path and we check that the file exists before we try to wipe it. The dnsmasq is also restarted to take the empty leases file into consideration. Finally, we improve the documentation to run the necessary commands using sudo. Change-Id: I56df821ca8a93307117a94a11abd0867b27d8443 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Parameterize infracloud playbook by platformsYolanda Robla Mota2-2/+22
This will allow to run on different distros depending on settings exported. Change-Id: I068d7bb2eebfd009dceeff057c8a65361483788b Signed-off-by: Yolanda Robla <yroblamo@redhat.com>