summaryrefslogtreecommitdiffstats
path: root/xci
AgeCommit message (Collapse)AuthorFilesLines
2017-08-11prototypes: xci: Add 'ansible-hardening' role to the requirements fileMarkos Chandras1-0/+4
Upstream commit 2798cf403c47d165cd8d34d7888990dce1f827f8 ("Use new ansible-hardening role") added the new 'ansible-hardening' role to the ansible-role-requirements.yml file and as such we also need to take that into consideration. The sha hash does not matter much at this point since it will be fixed the next time we tag all the roles. Change-Id: I5444af948e2b83de49863e7e28f3918fed88c90d Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Define an extra vars path and copy into playbooksYolanda Robla3-0/+13
This will allow to define the XCI_EXTRA_VARS_PATH, that can contain group_vars/all (or any other valid files), and those will be copied inside releng and bifrost playbooks. Change-Id: I95e4b0bfb67f26bfa1eb10c97096784eb7f3a87a Signed-Off-By: Yolanda Robla <yroblamo@redhat.com>
2017-08-11Explain the need of having an ssh keyYolanda Robla1-0/+6
In order for XCI to work, the user needs to have a ssh key. Explain that on the README and show the command to generate it. Change-Id: I517e15de15ba1b09977dbd62881bc2625c8b2833
2017-08-11prototypes: xci: Make CLEAN_DIB_IMAGES variable configurableMarkos Chandras2-1/+2
The CLEAN_DIB_IMAGES variable determines whether the /httpboot or /tftpboot directories will be removed before running the xci-deploy script. This forces XCI to re-create the target OS images. We moved this variable from env-vars to user-vars since this is something that users normally want to tweak in their deployments. Change-Id: Ia48bd63979e3837aab7e47c906c2501e7c7dd0e1 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11Fix ansible installation for non-rootYolanda Robla1-2/+2
Currently ansible was set to be installed with pip, but this fails for non-root user. Instead of that, execute pip with --user flag, so we don't need root permissions and it is only installed for the current user. Change-Id: Ib37a2a3866b4b48aca834b894cdd128ee63b31d6
2017-08-11Remove sudo usage from xci-deployYolanda Robla1-1/+1
The script was refactored to don't need sudo on the latest versions, and the documentation is still referencing it. Change-Id: I7064317ab7b756a78638f7474f2944472d3a4ba2
2017-08-11prototypes: xci: Look for SSH keys in $HOME directoryMarkos Chandras1-1/+1
We no longer run everything as root so we need to look for SSH keys in the appropriate home directory. Change-Id: Iae1f7eb80059e7d369c8e0c8b6c33c6a4f673f94 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Move host keys away from / directoryMarkos Chandras4-2/+11
Putting the host keys in '/' requires root privileges so it's best if we place them in the same directory like the rest of the XCI files. Change-Id: I030ed3d6cbb57bb984a78aeffb4eca2bd5c10bb0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Add ability to use local repositories for testingMarkos Chandras4-0/+88
When developing XCI features it's useful to be able to use the local repositories rather than cloning them from git since that makes it harder to test local modifications against XCI. As such, we add three new variables which can be used to hold local paths to the bifrost, releng and openstack-ansible repositories. We are still cloning the repositories but we then use the 'synchronize' Ansible module to copy modified files from the local repositories. Change-Id: I6d593ea48d8b9c51415d9d0848f77a498ef2f486 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: configure-localhost: Use 'connection: local'Markos Chandras1-1/+1
Since we are operating on localhost, there is no need to do that via ssh so we use 'connection: local' instead. Moreover, we do not need to execute everything as root so we drop the remote user directive. Change-Id: Ib2127edad29e2da1cd1beebf42cb8f0d278ce3ad Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: provision-vm-nodes: Do not provision VMs as root.Markos Chandras1-1/+16
There is no need to provision VM nodes as root. The only thing that we need to run as root is the destroy-env.sh script and for that we move it to its own play so we can use Ansible's become directive. Moreover, since this playbook operates on localhost we can use 'connection: local' so we don't have to execute everything via ssh and possibly speed up the whole process. Change-Id: Ia9efd5f30e95385b5cf193dde352f93551846c0e Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: user-vars: Move root directory away from /optMarkos Chandras1-4/+5
Having everything in the /opt implies root privileges. However, xci should work as normal user and as such we use the /tmp directory as the root directory. Change-Id: I985ded289208e1de53ad0ba878afd5bfd745c3fd Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: xci-deploy: Do not use sudo when connecting to remote hostsMarkos Chandras1-4/+4
In preparation for running xci-deploy as normal user, we do not need to gain elevated privileges in order to connect to remote hosts. SSH keys will be placed in user's ssh directory so we can ssh as normal user. Change-Id: Ieb705515ec191955a646839e8f195d089df184bd Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Do not require root privileges in xci-deployMarkos Chandras2-11/+7
There is no good reason to run the script as root. If elevated privileges are required then we will do it on a per case basis. Change-Id: I7195fb3a0026ea20ce04e0b652dd3c1c56ca5741 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11prototypes: xci: Make sure Ansible dependencies are installedMarkos Chandras2-1/+137
'pip install ansible' is not enough on newly installed hosts which may lack the necessary build tools to install Ansible's dependencies. As such, we add a script similar to the bifrost/scripts/install-deps.sh one to pull in all the necessary distro-specific packages so Ansible and it's dependencies can be installed from scratch. Change-Id: I4b1e74644db9ace451ad763e4c54f1a3a43214fd
2017-08-11xci: Ensure ansible is bootstrapped as last stepFatih Degirmenci1-8/+8
Role requirements file is used during ansible bootstrap on opnfv host so we must ensure the right ansible role requirements is there in advance. Change-Id: I2c5a1edd82a51cbbe1469c31f37b5d638d32ec27 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Always use pinned versions of openstack-ansible-requirementsJuan Vidal1-2/+0
Pinned versions used to be used only if XCI_LOOP == 'daily', but it seems to be required, otherwise this is very likely to fail Change-Id: I21f638bba75846a58b12373a903d414fa89b4bc6 Signed-off-by: Juan Vidal <juan.vidal.allende@ericsson.com>
2017-08-11xci: Fix nfs server ip for glanceFatih Degirmenci1-1/+1
Change-Id: Iebb9945bfd9cad9ebfda127f130542f4a6334aec Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Merge configure-targethosts.yml playbooks into oneFatih Degirmenci4-78/+0
They are exactly same now after moving nfs to compute00. Change-Id: I9d9d2f037aa3684ba130ae0a4f9aef2e733f2d2e Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Change cinder backend and move nfs to compute00Fatih Degirmenci8-50/+62
This change - switches cinder backend from lvm to nfs as examplified in [1]. - moves nfs shares from compute01 to compute00 in order to merge configure-targethosts.yml playbook in upcoming change. - updates openstack_user_config.yml to reflect the switch to nfs and move of nfs server to compute00. - removes exports file due to switching to lineinfile module. [1] https://docs.openstack.org/project-deploy-guide/openstack-ansible/draft/app-config-prod.html Change-Id: I0715c98a89ab124256e3857c1047e374f261e39b Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Force sync timeFatih Degirmenci4-0/+30
For some reason, when the nodes are provisioned, they are out of sync time-wise. This prevents neutron agents on compute nodes from starting and then everything fails. This change restarts chrony after the network configuration is updated and the nodes have internet access and force synchs time on nodes. Change-Id: Ib27b1fa0313223b52fa2e6229d5a179581d62686 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Fix network configurationFatih Degirmenci11-92/+66
This change - sets static IP for controller nodes' vxlan bridges as stated in osa documentation: compute and network hosts must have an IP [1] - configures IPs for controller nodes' vxlan bridges. - removes the additional vlan ip from compute nodes as it seems to be there for all in one. - removes glean.rules as they mess with the network configuration once the network changes are applied. - removes the contents of interfaces.d as osa populates config for lxc bridges and includes this config in interfaces file thus all the files in interfaces.d must be removed in order not to mess with the network configuration. - updates modules appropriately. [1] https://docs.openstack.org/project-deploy-guide/openstack-ansible/draft/app-config-prod.html Change-Id: I84d5f07216c8d0246da06080fb0ed19f917d6637 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Fix images directory permissionsFatih Degirmenci1-1/+1
Change-Id: Ibc1b9662bb9865c0e309128327d620324ddca85c Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Exclude aodh, ceilometer, and gnocchi from deploymentsFatih Degirmenci4-83/+0
Aodh is broken on the master branch due to use of conflicting sqlAlhcemy package, causing issues during osa repo_build. This patch disables aodh and the related services ceilometer and gnocchi since the reason for us to have these 2 is to get aodh. See the failure from https://build.opnfv.org/ci/job/xci-deploy-virtual-xenial-daily-master/26/consoleFull Change-Id: Ic6b74328cf1692b5e26c5f30b3eb3799240c7117 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Log info about provisioned VMs to consoleFatih Degirmenci1-1/+3
Change-Id: If29aaf0be6ddd860450eeb10432fe75e12b1d03b Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: SHA bump as of April 4Fatih Degirmenci3-60/+53
The bifrost, osa, and ansible role requirements sha1s are bumped based on the HEAD of the master of the projects on April 4. designate is also disabled based on the comment the osa commit mentions. https://review.openstack.org/gitweb?p=openstack/openstack-ansible.git;a=commit;h=d9e1330c7ff9d72a604b6b4f3af765f66a01b30e Change-Id: Ib47515490c799d3b1d2b4b319277c8900cd37328 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Introduce XCI_LOOP variable to control what to do properlyFatih Degirmenci3-0/+4
XCI has different jobs/loops to run - patchset verification jobs (currently bifrost and osa in future) - periodic jobs (bifrost and osa) - daily jobs (for OPNFV platform deployment and testing) The same scripts/playbooks used by XCI will also be used by developers. We need to do different things depending on the context the scripts and playbooks are executed. - periodic jobs will use latest of everything to find working versions of the components. (periodic osa will use unpinned role requirements for example) - daily jobs will use pinned versions in order to bring up the platform and run OPNFV testing against it. (daily deployment will use pinned versions and role requirements for example) - developers might choose to use pinned versions or latest Depending on what loop we are running, we need to do things differently in scripts and playbooks. This variable will help us to do this in easy way. We can of course do pattern matching of the job name but it will not work if the scripts are used outside of Jenkins. The default loop for non-Jenkins execution is set to daily as we want developers to use working versions unless they change it to something else intentionally. Change-Id: Iff69c77ae3d9db2c14de1783ce098da9e9f0c83d Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Fix the haproxy service and gnocchi db sync problemFatih Degirmenci4-0/+9
This patch reflects the fix Tianwei found for the keystone and gnocchi issues with HA deployment. https://gerrit.opnfv.org/gerrit/#/c/32367/ The keystone issue should be fixed upstream but until that happens, we can use the patch here to move us forward. Previously removed OpenStack services have also been enabled to try as close deployment as possible to upstream openstack-ansible. Change-Id: Ib7b1cfe0f363381466b43a3249667c638c09d9fd Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Adjust and fix jobs and scriptsFatih Degirmenci2-2/+12
This change - fixes GIT_BASE in jobs - adjusts build blockers - fixes where the flavor vars should be sourced from - sources flavor vars for bifrost periodic jobs - renames the xcimaster node to opnfv to be consistent - removes obsolete xci-provision.sh script - enable additional openstack services including tempest to ensure the deployment is sane Change-Id: Ifdce1da52d68a26c2b87e1bd3b1996ec119d8d90 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Start developer documentationFatih Degirmenci1-0/+31
Change-Id: Ib5200372a44ee6a22b45c2105b3729d362eacb1d Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Configure daily job to use xci scriptsFatih Degirmenci1-0/+1
Daily jobs are now switched to using xci-deploy.sh script together with the environment variables, playbooks, and so on. These jobs do not set any version for any of the component as the point with these jobs is to test the platform itself, not the tools. DIB related stuff is still left as parameters to deploy job for SUSE and CentOS jobs so we can override what is set by xci depending on the distro. Change-Id: I5a5292e6bae8fdd052858108db409e8d495fbf24 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Update main script and readmeFatih Degirmenci2-7/+373
This change - updates the main script to run the playbooks - creates the initial version of the readme file Change-Id: I4110433bce541a5d9645f34457f9e1b1ee93df19 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Create playbook to destroy, create and provision VM nodesFatih Degirmenci5-29/+44
This change moves preparation (cloning repos, combining opnfv/bifrost with openstack/bifrost), destroying and creating VM nodes from the script into a separate playbook. This requires the host to have ansible installed. The version of ansible to install using pip currently matches to what bifrost uses but it is hardcoded and needs to be fixed properly. The reason for having this as a playbook to simplify the script and increase the reuse. This playbook will be used for - developer sandbox - periodic bifrost jobs to run against latest on given branch and promoting bifrost sha1 to pin later on - daily jobs to run using pinned versions of bifrost Change-Id: I033f12290dfea19d4c74be80eea7203211c0369e Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Add ansible files for sandbox ha flavorFatih Degirmenci5-0/+389
This patch adds noha target host configuration playbook, inventory, variable files and so on to use until we have time to fix things more properly and some other functionality becomes available in upstream. This approach will result in duplicates but it is something we can perhaps live with. The playbook to configure target hosts will be same with the one we will have for noha flavor but for different for the flavor mini. The user variables and openstack user configuration will be different between flavors and keeping deployment specific configuration. Change-Id: I4e2b375b9f8f6bd5f8c5da91a522b78d61a58125 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Add ansible files for sandbox mini flavorFatih Degirmenci5-0/+273
This patch adds mini target host configuration playbook, inventory, variable files and so on to use until we have time to fix things more properly and some other functionality becomes available in upstream. This approach will result in duplicates but it is something we can perhaps live with. The user variables and openstack user configuration will be different between flavors and keeping deployment specific configuration. Change-Id: Ie312dfa2047d094b590de970c1436d69f4c69668 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Add ansible files for sandbox noha flavorFatih Degirmenci7-0/+325
This patch adds noha target host configuration playbook, inventory, variable files and so on to use until we have time to fix things more properly and some other functionality becomes available in upstream. This approach will result in duplicates but it is something we can perhaps live with. NFS configuration is converted into a role and the rest is adjusted accordingly. The playbook to configure target hosts will be same with the one we will have for ha flavor but for different for the flavor mini. The user variables and openstack user configuration will be different between flavors and keeping deployment specific configuration. Change-Id: I8faceae846285fc3e4260cbced5b740a9b0c3766 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Move role configure-network to the right placeFatih Degirmenci1-3/+2
Incorrectly placed role configure-network causes ansible to skip the roles at the beginning of the playbook. Change-Id: Id85e9684eb86acad7c11104b434a319c37e16903 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Add ansible files for sandbox aio flavorFatih Degirmenci4-0/+27
This patch adds aio opnfv host configuration playbook, inventory, variable files and so on to use until we have time to fix things more properly and some other functionality becomes available in upstream. This approach will result in duplicates but it is something we can perhaps live with. The playbook to configure opnfv host is different from the playbook used for the rest of the flavors as our target host is the opnfv host itself. This is the reason for not having target host configuration playbook. This installation of aio will be driven by upstream scripts/playbooks entirely and we just execute the scripts we have in openstack-ansible repo. Change-Id: Ica814bdac1d324414f3add382a8241de49c93a2e Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Networking fixes to introduce role and interface varFatih Degirmenci6-57/+65
Network configuration task and accompanied handler put into a role and handler is converted to task. Distro specific var, interface, is introduced to ensure we do not hardcode the interface which might not be available. Update the templates accordingly. Change-Id: I667620fe22c93a9b20a1d8c1b7b0051d7647b591 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Ensure using flavor inventoryFatih Degirmenci1-0/+4
This is needed in order to configure target hosts that are in the flavor inventory file rather than what is there by default, which is aio. Change-Id: Iab27ed71a3d7c03f8fbbec8ceb7af92259320412
2017-08-11xci: Add common and OS family ansible var filesFatih Degirmenci6-0/+52
OS family vars_files are currently empty and put there to show the intend. (kind of TODO) opnfv.yml holds non-distro and non-flavor specific variables. Change-Id: I65aff2650257f2df00fd1f0a0638fd1aff596ac4 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Add main playbooks and filesFatih Degirmenci14-0/+643
This patch adds the main/common playbooks, files, and templates to be used for all flavors. The provisioning and OpenStack installation process will be as below - provision VMs for flavor using bifrost - once the VMs are provisioned, configure-localhost.yml playbook will be run, preparing the localhost in order to ensure the right playbooks (configure-opnfvhost.yml and configure-targethosts.yml), inventory files and var files are in place before we proceed with configuring opnfv host. - after getting the right files for the flavor, opnfv host will be configured using configure-opnfvhost.yml playbook. - finally, the target hosts will be configured. - once the above process is completed, openstack-ansible playbooks will be run, setting up hosts, infrastructure and OpenStack. Change-Id: I6e08b2cfdab9627f765e6fc414917b09f953cab2 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11prototypes: bifrost: Remove /opt/stack from previous deploymentsMarkos Chandras1-1/+1
/opt/stack is being created by bifrost so the destroy-env.sh script should be responsible for cleaning up all the remnants. That way we can use the destroy-env.sh script outside of Jenkins as well. Change-Id: I72d9737cf2446ff3ffc02f195337a6873eceb5c0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-08-11xci: Make OSA playbook to use during installation configurableFatih Degirmenci1-0/+8
Change-Id: Ie6f079bb61905ca0458778b78fef799777b24efa Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Rename the deployment host to opnfvFatih Degirmenci8-13/+13
Reflect the changes to other impacted files as well. Change-Id: I106f4e47fe5c75d288f4878fe6ec9f8ff39c652e Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Add/update variables to tool, user, and flavorsFatih Degirmenci5-10/+18
Updates - RAM allocations for the flavors mini and noha increased to 12GB. - Distro variables introduced to tool as it will be used by the CI and the developers. - LOG_PATH and RUN_TEMPEST are now available in user variables. Change-Id: I4a73d7b79761d81f605cdb3fa9771ca40e9f9f02 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Yet another try to find how best to structure stuffFatih Degirmenci30-10/+1
Summary of changes are - flavors directory has been removed and the flavor config files are moved into config and renamed to <flavor>-vars - common files are put under file - files specific to flavors are put under file/<flavor> directories - templates and var files are stored in template and var directories respectively - 3 playbooks are created Change-Id: I8a93e0947ccb02f93a6c8f00da27e0cc6b4dc21e Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Enable node provisioning and shuffle/rename stuffFatih Degirmenci23-71/+115
This change enables node provisioning using bifrost based on the chosen flavor. Other changes include - move flavor specific stuff (playbooks, inventory, vars) into their own folders so it is easier to copy them over. - rename flavors and get rid of xci from them. - introduce env-vars to keep variables that are not really user variables. Please note that this patch contains empty files and so on due to still trying to find best way to structurei things. Apart from this, there will probably be lots of duplications here and there which will be taken care of once things settle. Change-Id: I04a5d422e5b018439bafea5e68e65255ae38d22b Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-08-11xci: Set up the basics for xci developer sandboxFatih Degirmenci19-0/+195
This change creates - initial version of the script to initiate the VM node creation and provisioning with bifrost and OpenStack installation with openstack-ansible - pinned-versions to hold the "known working" versions. releng will use master until the development is complete. - user-vars to hold user variables - configuration files to keep settings for different flavors - ansible files (playbooks, inventory files, variables) per flavor. These files are empty at the moment in order to save the chosen path makes sense. Once the way is agreed on, these will be similar to the files listed below with different content per flavor. https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=prototypes/openstack-ansible/playbooks/inventory https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=prototypes/openstack-ansible/playbooks/configure-xcimaster.yml https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=prototypes/openstack-ansible/var/ubuntu.yml These new files will be the ones used when the actual deployment is done with openstack-ansible based on the flavor chosen by developer (or CI). Change-Id: Ia2f38416a161abd186cbcca61e105c6a68f78b54 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>