summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-08-23xci: Bump OSA SHA to Pre-PikeFatih Degirmenci9-71/+459
This change bumps osa sha to 4d39f2cc. [1] The reason for going to a later osa sha is to include ODL and Tacker commits in upstream to ease the work done for ODL and SFC scenarios. However, this osa sha seems to be having issues with the flavor ha so the users should be aware of the problems which is added into the readme as known issue and the flavor is disabled in xci-deploy.sh. Another reason to bump sha right now is that upstream changed a lot during last 2 months and waiting longer will probably result in more changes, making it harder to move to a later osa sha even harder. Other changes included by this change are - update ansible role requirements to move to shas pinned by upstream for pike. - add odl and tacker roles into ansible role requirements. - update how aio is deployed due to removal of an upstream script. (this could perhaps have been done in a better way but the time pressure required this to be fixed in the way as proposed.) - ensure facts are gathered for all the hosts and containers before running setup-infrastructure.yml playbook. - stop grepping for failed in the log of setup-hosts.yml playbook due to existence of rescue block in rabbitmq_cluster_create.yml [2]. - reduce the resource needs by updating the user_variables that are based on upstream osa gate which uses aio. The values for none-aio xci flavors will be adjusted in a separate change once the sha bump is complete. - fix a bug in ha-vars. [1] https://review.openstack.org/gitweb?p=openstack/openstack-ansible.git;a=commit;h=4d39f2cc29417153780210fc0bb86223387e9968 [2] https://review.openstack.org/gitweb?p=openstack/openstack-ansible-rabbitmq_server.git;a=blob;f=tasks/rabbitmq_cluster_create.yml;h=4aaad5adafa42684c0d791b49539069fe1b45635;hb=HEAD#l24 Change-Id: Ia7fc495e315e4a41359641f8be4b3c8bbf7b61fa Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-21xci: Update the repo directory structureFatih Degirmenci28-2208/+0
This patch - removes obsolete openstack-ansible and puppet-infracloud directories - adds upstream directory to keep the contributions that are pending to be accepted by upstream in order to have progress in OPNFV. In a perfect world, one should expect to have nothing in this directory so the items in this folder are short-lived. - adds prototypes directory to keep stuff that hasn't been discussed to be part of XCI and to share ideas and trials with the rest of the community. Change-Id: I12afe7050ff2b0ac457d4b16d21dfd7df6ac84c9 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-18Merge "prototypes: bifrost: destroy-env: Always destroy XCI VMs"Markos Chandras1-0/+6
2017-08-17prototypes: bifrost: destroy-env: Always destroy XCI VMsMarkos Chandras1-0/+6
It's possible that a previous run failed before registering the VMs to the vbmc. As such, consulting the vbmc to retrieve the VM names will return nothing and no VM will be deleted. As such, it's best to get the VMs from the currently XCI profile and make sure they are really gone. Change-Id: If11265655f42c8a1c7627c9139ac0cbfc002b1fe Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-17Merge "Refactor releng/prototypes/xci to releng-xci/xci in the code"Fatih Degirmenci12-61/+62
2017-08-17Merge "bifrost: scripts: Disable selinux on Centos7"Markos Chandras1-0/+10
2017-08-17Merge "prototypes: xci: xci-deploy.sh: Only allow XCI on Ubuntu"Markos Chandras1-0/+9
2017-08-17Merge "prototypes: xci: config: Allow users to set VM specs"Markos Chandras4-16/+16
2017-08-17prototypes: xci: xci-deploy.sh: Only allow XCI on UbuntuMarkos Chandras1-0/+9
There is no support for openSUSE or CentOS right now yet the script runs happily on these hosts until it breaks in horrible ways. The only place we explicitly ask for Ubuntu hosts is the wiki which can easily be missed by newcomers. As such, modify the script to only run on Ubuntu hosts. Change-Id: If9b355da51367fc95948a2b12a7f6c4d6b3bbbbf Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-17prototypes: xci: config: Allow users to set VM specsMarkos Chandras4-16/+16
The default VM spec may not be good enough for every host so allow users to override the default VM_* variables. Change-Id: Ie846620ab81b7fedcbe4b23da735379ab75d3cc2 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-16Refactor releng/prototypes/xci to releng-xci/xci in the codeDave Urschatz12-61/+62
These changes are needed as part of migrating xci from releng repo to the releng-xci repo. They cover: - README updates - Re-point configuration from releng/prototypes to releng-xci The changes have been tested as follows: - XCI_FLAVOR=aio - OPENSTACK_OSA_VERSION=adfaa66d6108f87599e0595591b39cad2c8fb658 xci: aio has been installed real 107m5.310s user 11m50.180s sys 4m11.152s ubuntu@rack-IRA2-SymKloudBlade4:~/releng-xci/xci$ Change-Id: I8fd86c2442ee3a7c6996eedb6510dbc6eebd30b0 Signed-off-by: Dave Urschatz <dave.urschatz@cengn.ca>
2017-08-16bifrost: scripts: Disable selinux on Centos7Markos Chandras1-0/+10
We are seeing issues with selinux enabled from time to time. Having selinux enabled has proven to be rather unstable and Centos7 not being a gate on upstream bifrost does not help in fully testing this scenario. As such, the best we can do right now is to disable selinux until Centos7 becomes a gate upstream. Change-Id: I93a3414cfebc3c3ef4ac063c8e623f36f615455f Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-15xci: Vagrantfile: Add Vagrantfile to simplify testingMarkos Chandras1-0/+90
Add a Vagrantfile based on the upstream AIO one which can be used to easily test XCI without modifying the host. Change-Id: Icced4ff4cd80c43d4b0cec2282629fedaf44cb8c Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Initial .gitignore and .gitreviewTrevor Bramwell2-0/+44
Change-Id: I71280521df7727747daf47b662356292164a076e Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
2017-08-11prototypes: bifrost: Do not use tmpfs for dib buildsMarkos Chandras1-0/+2
Using tmpfs seems somewhat unstable. For example the following error is observed on openSUSE hosts ERROR: failed to umount the /tmp/dib_image.RJBbahUd tmpfs As such, we set the dib_notmpfs variable to 'true' which mimics what upstream OpenStack scripts are doing already in project-config:tools/build-image.sh Change-Id: If050ab70303b3556fc1880a956f0ba1505d62050 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Fix missing python packages for password generationJuan Vidal Allende4-0/+10
These packages are needed by pw-token-gen.py tool Change-Id: Ib9d165274449551a469e201da9feeffac5a7a4cf Signed-off-by: Juan Vidal Allende <juan.vidal.allende@ericsson.com>
2017-08-11prototypes: xci: xci-deploy: Set default value for devel parametersMarkos Chandras1-1/+1
Commit 68412ffc734e ("prototypes: xci: xci-deploy.sh: Append trailing slash for rsync vars") modified xci-deploy.sh script to append a trailing slash to the devel variables but it broke the script when one of these variables was not set (since we run with 'set -eu') with the following error: ./xci-deploy.sh: line 45: !local_user_var: unbound variable Fixes: I0103b754585931fa1dcd3966c52d7e4a8f2f63f6 Change-Id: Ic2465ff07348b61e74e191516425cd10d80c1fce Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: xci-deploy.sh: Append trailing slash for rsync varsMarkos Chandras1-0/+9
The 'synchronize' Ansible module uses rsync and for that we need to indicate that we want to copy the contents of the development path and not the development path itself. As such we need to ensure that a trailing slash is always appended to the user variables but we need to also ensure that we will not end up with double slashes. Change-Id: I0103b754585931fa1dcd3966c52d7e4a8f2f63f6 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11xci: Fix README for setting dev path instructionsFatih Degirmenci1-2/+4
Change-Id: I56c727c242b6826f87a64246b6b4736a637c1abe Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: xci: scripts: Update SHA datesMarkos Chandras2-4/+6
Update the dates in the pinned-versions file whenever we update the SHAs. Change-Id: I23ebd90a3fc688a7501fcb2d32217357f8c31657 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: update-osa-version-files: Add support for updating bifrost SHAsMarkos Chandras2-4/+9
Add a second optional argument to update the bifrost SHA instead of doing this bit manually. Change-Id: I0fab8745ae08ad4f54a2a1f5b0b7a64fde4b8295 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Update SHAs for OSA and its rolesMarkos Chandras3-57/+60
This also updates the second (pre_release) argument of 'update_ansible_role_requirements' from 'false' to 'true' so that non-openstack roles can be updated as well. Change-Id: Id424499f44c9b51c02d56e5d93580faeea50ad12 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Talk about development mode for XCIYolanda Robla1-0/+17
Explain the usage of OPNFV_RELENG_DEV_PATH Change-Id: I19e3946c9ea78a6bb0521818b58af297a396d80d
2017-08-11prototypes: xci: config: Update bifrost SHAMarkos Chandras1-1/+1
Update bifrost SHA to 7c9bb5e07c6bc3b42c9a9e8457e5eef511075b38 which is the current HEAD on 16th of June 2017. Change-Id: I91cd98221fda7a1dd689abbfbf07474bd8aac4e1 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: scripts: Add update-osa-version-files.sh scriptMarkos Chandras1-0/+85
Add new prototypes/xci/scripts/update-osa-version-files.sh which can be used to update the XCI ansible-role-requirements.yml file as well as the OSA pinned SHA string. This file is using the upstream 'sources-branch-updater-lib.sh' library from the openstack-ansible repository. Change-Id: I51b88c50cb2bffe0cf1b7aa054a5b237103fd92f Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Add 'ansible-hardening' role to the requirements fileMarkos Chandras1-0/+4
Upstream commit 2798cf403c47d165cd8d34d7888990dce1f827f8 ("Use new ansible-hardening role") added the new 'ansible-hardening' role to the ansible-role-requirements.yml file and as such we also need to take that into consideration. The sha hash does not matter much at this point since it will be fixed the next time we tag all the roles. Change-Id: I5444af948e2b83de49863e7e28f3918fed88c90d Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Define an extra vars path and copy into playbooksYolanda Robla3-0/+13
This will allow to define the XCI_EXTRA_VARS_PATH, that can contain group_vars/all (or any other valid files), and those will be copied inside releng and bifrost playbooks. Change-Id: I95e4b0bfb67f26bfa1eb10c97096784eb7f3a87a Signed-Off-By: Yolanda Robla <yroblamo@redhat.com>
2017-08-11Explain the need of having an ssh keyYolanda Robla1-0/+6
In order for XCI to work, the user needs to have a ssh key. Explain that on the README and show the command to generate it. Change-Id: I517e15de15ba1b09977dbd62881bc2625c8b2833
2017-08-11prototypes: xci: Make CLEAN_DIB_IMAGES variable configurableMarkos Chandras2-1/+2
The CLEAN_DIB_IMAGES variable determines whether the /httpboot or /tftpboot directories will be removed before running the xci-deploy script. This forces XCI to re-create the target OS images. We moved this variable from env-vars to user-vars since this is something that users normally want to tweak in their deployments. Change-Id: Ia48bd63979e3837aab7e47c906c2501e7c7dd0e1 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Switch Ubuntu targets to Xenial (16.04)Markos Chandras1-2/+2
Switch default target OS to Ubuntu Xenial (16.04) which is also the version which is supported by OpenStack Ansible. Change-Id: I73adc6472b0fb572177e1508cf831b4d3190245c Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Fix ansible installation for non-rootYolanda Robla1-2/+2
Currently ansible was set to be installed with pip, but this fails for non-root user. Instead of that, execute pip with --user flag, so we don't need root permissions and it is only installed for the current user. Change-Id: Ib37a2a3866b4b48aca834b894cdd128ee63b31d6
2017-08-11Remove sudo usage from xci-deployYolanda Robla1-1/+1
The script was refactored to don't need sudo on the latest versions, and the documentation is still referencing it. Change-Id: I7064317ab7b756a78638f7474f2944472d3a4ba2
2017-08-11prototypes: xci: Look for SSH keys in $HOME directoryMarkos Chandras1-1/+1
We no longer run everything as root so we need to look for SSH keys in the appropriate home directory. Change-Id: Iae1f7eb80059e7d369c8e0c8b6c33c6a4f673f94 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Move host keys away from / directoryMarkos Chandras4-2/+11
Putting the host keys in '/' requires root privileges so it's best if we place them in the same directory like the rest of the XCI files. Change-Id: I030ed3d6cbb57bb984a78aeffb4eca2bd5c10bb0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Add ability to use local repositories for testingMarkos Chandras4-0/+88
When developing XCI features it's useful to be able to use the local repositories rather than cloning them from git since that makes it harder to test local modifications against XCI. As such, we add three new variables which can be used to hold local paths to the bifrost, releng and openstack-ansible repositories. We are still cloning the repositories but we then use the 'synchronize' Ansible module to copy modified files from the local repositories. Change-Id: I6d593ea48d8b9c51415d9d0848f77a498ef2f486 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Remove OPNFV specific group_vars fileMarkos Chandras1-53/+0
The only reason we had this file was because we had to define our own 'ssh_public_key_path' variable to look in the /root directory since we were running everything as root. Since we are now moving away from this approach, this file is not needed anymore. Change-Id: Ic938c4c8c877c95ce261721dc32fc5147081e236 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: configure-localhost: Use 'connection: local'Markos Chandras1-1/+1
Since we are operating on localhost, there is no need to do that via ssh so we use 'connection: local' instead. Moreover, we do not need to execute everything as root so we drop the remote user directive. Change-Id: Ib2127edad29e2da1cd1beebf42cb8f0d278ce3ad Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: provision-vm-nodes: Do not provision VMs as root.Markos Chandras1-1/+16
There is no need to provision VM nodes as root. The only thing that we need to run as root is the destroy-env.sh script and for that we move it to its own play so we can use Ansible's become directive. Moreover, since this playbook operates on localhost we can use 'connection: local' so we don't have to execute everything via ssh and possibly speed up the whole process. Change-Id: Ia9efd5f30e95385b5cf193dde352f93551846c0e Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: user-vars: Move root directory away from /optMarkos Chandras1-4/+5
Having everything in the /opt implies root privileges. However, xci should work as normal user and as such we use the /tmp directory as the root directory. Change-Id: I985ded289208e1de53ad0ba878afd5bfd745c3fd Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: xci-deploy: Do not use sudo when connecting to remote hostsMarkos Chandras1-4/+4
In preparation for running xci-deploy as normal user, we do not need to gain elevated privileges in order to connect to remote hosts. SSH keys will be placed in user's ssh directory so we can ssh as normal user. Change-Id: Ieb705515ec191955a646839e8f195d089df184bd Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Do not require root privileges in xci-deployMarkos Chandras2-11/+7
There is no good reason to run the script as root. If elevated privileges are required then we will do it on a per case basis. Change-Id: I7195fb3a0026ea20ce04e0b652dd3c1c56ca5741 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: bifrost: Write clouds.yaml file for regular usersMarkos Chandras1-6/+1
This is similar to commit 6769ae9bcbf6 ("prototypes: bifrost: Add keystone roles"). In preparation for running the bifrost jobs as normal user we need to write a clouds.yaml file to users home directory. This is similar to the bifrost upstream testing playbook. Change-Id: Iae4bc73906649707667925280a19282ec5c93998
2017-08-11prototypes: xci: Make sure Ansible dependencies are installedMarkos Chandras2-1/+137
'pip install ansible' is not enough on newly installed hosts which may lack the necessary build tools to install Ansible's dependencies. As such, we add a script similar to the bifrost/scripts/install-deps.sh one to pull in all the necessary distro-specific packages so Ansible and it's dependencies can be installed from scratch. Change-Id: I4b1e74644db9ace451ad763e4c54f1a3a43214fd
2017-08-11prototypes: bifrost: Add keystone rolesMarkos Chandras3-2/+19
The os_client_config Ansible module used by all the ironic-* roles depends on the keystone roles so we need to make use of them even though they do not affect the end result. This fixes the following OPNFV CI problem due to not having a clouds.yaml file present to be consumed by the os_client_config Ansible module. fatal: [controller00]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} An exception occurred during task execution. The full traceback is: Traceback (most recent call last): File "/tmp/ansible_ApkCUQ/ansible_module_os_client_config.py", line 75, in <module> main() File "/tmp/ansible_ApkCUQ/ansible_module_os_client_config.py", line 63, in main for cloud in config.get_all_clouds(): File "/usr/lib/python2.7/site-packages/os_client_config/config.py", line 798, in get_all_clouds cloud, region_name=region['name'])) File "/usr/lib/python2.7/site-packages/os_client_config/config.py", line 1071, in get_one_cloud auth_plugin = loader.load_from_options(**config['auth']) File "/usr/lib/python2.7/site-packages/keystoneauth1/loading/base.py", line 162, in load_from_options raise exceptions.MissingRequiredOptions(missing_required) keystoneauth1.exceptions.auth_plugins.MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url fatal: [opnfv]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} Moreover, we cleanup the ~/openstack directory which may contain some bifrost artifacts such as a clouds.yaml file. Finally, we use 'sudo -H' because for the keystone roles we need the HOME variable to be set properly. Change-Id: I45b08bd33dd8ea9505fe10eb4b2b10956b3b683c
2017-08-11xci: Ensure ansible is bootstrapped as last stepFatih Degirmenci1-8/+8
Role requirements file is used during ansible bootstrap on opnfv host so we must ensure the right ansible role requirements is there in advance. Change-Id: I2c5a1edd82a51cbbe1469c31f37b5d638d32ec27 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Always use pinned versions of openstack-ansible-requirementsJuan Vidal1-2/+0
Pinned versions used to be used only if XCI_LOOP == 'daily', but it seems to be required, otherwise this is very likely to fail Change-Id: I21f638bba75846a58b12373a903d414fa89b4bc6 Signed-off-by: Juan Vidal <juan.vidal.allende@ericsson.com>
2017-08-11xci: Fix nfs server ip for glanceFatih Degirmenci1-1/+1
Change-Id: Iebb9945bfd9cad9ebfda127f130542f4a6334aec Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Merge configure-targethosts.yml playbooks into oneFatih Degirmenci4-78/+0
They are exactly same now after moving nfs to compute00. Change-Id: I9d9d2f037aa3684ba130ae0a4f9aef2e733f2d2e Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Change cinder backend and move nfs to compute00Fatih Degirmenci8-50/+62
This change - switches cinder backend from lvm to nfs as examplified in [1]. - moves nfs shares from compute01 to compute00 in order to merge configure-targethosts.yml playbook in upcoming change. - updates openstack_user_config.yml to reflect the switch to nfs and move of nfs server to compute00. - removes exports file due to switching to lineinfile module. [1] https://docs.openstack.org/project-deploy-guide/openstack-ansible/draft/app-config-prod.html Change-Id: I0715c98a89ab124256e3857c1047e374f261e39b Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Force sync timeFatih Degirmenci4-0/+30
For some reason, when the nodes are provisioned, they are out of sync time-wise. This prevents neutron agents on compute nodes from starting and then everything fails. This change restarts chrony after the network configuration is updated and the nodes have internet access and force synchs time on nodes. Change-Id: Ib27b1fa0313223b52fa2e6229d5a179581d62686 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>