summaryrefslogtreecommitdiffstats
path: root/jjb/xci
AgeCommit message (Collapse)AuthorFilesLines
2017-05-08jjb: xci: Run bifrost provisioning as normal userMarkos Chandras2-4/+4
Ansible can request elevated privileges using the 'become' directive. As such there is no need to run ansible using sudo. Furthermore, OpenStack CI runs all jobs as normal user and we need to do the same in order to compare behavior and results properly. Change-Id: Id198efd1d6fdf4619de931fbedc7caba8d68d2a0
2017-05-03prototypes: bifrost: Add keystone rolesMarkos Chandras1-2/+2
The os_client_config Ansible module used by all the ironic-* roles depends on the keystone roles so we need to make use of them even though they do not affect the end result. This fixes the following OPNFV CI problem due to not having a clouds.yaml file present to be consumed by the os_client_config Ansible module. fatal: [controller00]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} An exception occurred during task execution. The full traceback is: Traceback (most recent call last): File "/tmp/ansible_ApkCUQ/ansible_module_os_client_config.py", line 75, in <module> main() File "/tmp/ansible_ApkCUQ/ansible_module_os_client_config.py", line 63, in main for cloud in config.get_all_clouds(): File "/usr/lib/python2.7/site-packages/os_client_config/config.py", line 798, in get_all_clouds cloud, region_name=region['name'])) File "/usr/lib/python2.7/site-packages/os_client_config/config.py", line 1071, in get_one_cloud auth_plugin = loader.load_from_options(**config['auth']) File "/usr/lib/python2.7/site-packages/keystoneauth1/loading/base.py", line 162, in load_from_options raise exceptions.MissingRequiredOptions(missing_required) keystoneauth1.exceptions.auth_plugins.MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url fatal: [opnfv]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} Moreover, we cleanup the ~/openstack directory which may contain some bifrost artifacts such as a clouds.yaml file. Finally, we use 'sudo -H' because for the keystone roles we need the HOME variable to be set properly. Change-Id: I45b08bd33dd8ea9505fe10eb4b2b10956b3b683c
2017-04-06Merge "xci: Introduce XCI_LOOP variable to control what to do properly"Fatih Degirmenci5-3/+20
2017-04-05jjb: xci: bifrost-cleanup-job: Fix undefined variableMarkos Chandras1-1/+1
Variable 'i' does not exist. It should be 'try_to_rm' instead. Change-Id: Ic2aa5a1115715851379e2c7ba5c11699cb8e7407 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-04-04xci: Introduce XCI_LOOP variable to control what to do properlydanube.1.0Fatih Degirmenci5-3/+20
XCI has different jobs/loops to run - patchset verification jobs (currently bifrost and osa in future) - periodic jobs (bifrost and osa) - daily jobs (for OPNFV platform deployment and testing) The same scripts/playbooks used by XCI will also be used by developers. We need to do different things depending on the context the scripts and playbooks are executed. - periodic jobs will use latest of everything to find working versions of the components. (periodic osa will use unpinned role requirements for example) - daily jobs will use pinned versions in order to bring up the platform and run OPNFV testing against it. (daily deployment will use pinned versions and role requirements for example) - developers might choose to use pinned versions or latest Depending on what loop we are running, we need to do things differently in scripts and playbooks. This variable will help us to do this in easy way. We can of course do pattern matching of the job name but it will not work if the scripts are used outside of Jenkins. The default loop for non-Jenkins execution is set to daily as we want developers to use working versions unless they change it to something else intentionally. Change-Id: Iff69c77ae3d9db2c14de1783ce098da9e9f0c83d Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-04-04Merge "xci: Do not log info to console if running daily deployment"Fatih Degirmenci1-12/+14
2017-04-04xci: Do not log info to console if running daily deploymentFatih Degirmenci1-12/+14
Periodic and daily jobs use same script and we should not log info about osa sha1 to pin into console if it is not run as part of periodic job. Change-Id: I8203a822638603af92fd046d8038a3b4122dbf37 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-04-04xci: Fix build blocker so downstream jobs can be startedFatih Degirmenci1-1/+0
Change-Id: Ie33bcbbe1d4c4fd2166c8931bd9f135cf4200263 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-04-04Merge "xci: Create periodic osa jobs to find working version"Fatih Degirmenci2-0/+188
2017-04-03xci: Adjust and fix jobs and scriptsFatih Degirmenci5-114/+19
This change - fixes GIT_BASE in jobs - adjusts build blockers - fixes where the flavor vars should be sourced from - sources flavor vars for bifrost periodic jobs - renames the xcimaster node to opnfv to be consistent - removes obsolete xci-provision.sh script - enable additional openstack services including tempest to ensure the deployment is sane Change-Id: Ifdce1da52d68a26c2b87e1bd3b1996ec119d8d90 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-04-02xci: Create periodic osa jobs to find working versionFatih Degirmenci2-0/+188
These jobs run periodically against the HEAD of master and stable/ocata branches in order to attempt deployment and help us to find more recent working versions to pin. In perfect world, these jobs should be run post-merge, helping us to move faster but lack of resources prevent us doing that. One thing to note here is that we use the pinned version of bifrost in order to make sure we change one thing at a time to reduce the delta. Change-Id: Ieaff7e80581c7577b253b3646ca386313295a45a Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-04-02Merge "xci: Create periodic bifrost jobs to find working version"Fatih Degirmenci2-0/+253
2017-03-31xci: Create periodic bifrost jobs to find working versionFatih Degirmenci2-0/+253
These jobs run periodically against the HEAD of master and stable/ocata branches in order to attempt provisioning and help us to find more recent working versions to pin. In perfect world, these jobs should be run post-merge, helping us to move faster but lack of resources prevent us doing that. Change-Id: I6f6b2fcc58eda112b1e811a7b11269e928bbceef Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-31xci: Configure daily job to use xci scriptsFatih Degirmenci2-199/+64
Daily jobs are now switched to using xci-deploy.sh script together with the environment variables, playbooks, and so on. These jobs do not set any version for any of the component as the point with these jobs is to test the platform itself, not the tools. DIB related stuff is still left as parameters to deploy job for SUSE and CentOS jobs so we can override what is set by xci depending on the distro. Change-Id: I5a5292e6bae8fdd052858108db409e8d495fbf24 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-29prototypes: bifrost: Remove /opt/stack from previous deploymentsMarkos Chandras2-2/+2
/opt/stack is being created by bifrost so the destroy-env.sh script should be responsible for cleaning up all the remnants. That way we can use the destroy-env.sh script outside of Jenkins as well. Change-Id: I72d9737cf2446ff3ffc02f195337a6873eceb5c0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-03-29Make prescm step workspace permission fix a macro available to allFatih Degirmenci1-12/+1
bifrost-verify jobs have been using this without issues and it is needed for many other jobs in order to avoid unnecessary failures. This change makes workspace permission fix prescm step a macro. Change-Id: Ib5678dbf21c17ce640a600d2a06ebed9674663fa Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-24jjb: xci: bifrost-cleanup: Retry 'gsutil rm' on failuresMarkos Chandras1-3/+10
Sometimes 'gsutil rm' may fail for unknown reasons like the following one: [...] / [7/10 objects] 70% Done / [8/10 objects] 80% Done / [9/10 objects] 90% Done Operation completed over 10 objects. CommandException: 1 files/objects could not be removed. Try a few more times to remove the objects before giving up. Change-Id: I6905a806446dac41ddc43d4195514a5233480ccd Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-03-24Merge "xci: Configure ansible verbosity based on job type"Fatih Degirmenci2-0/+6
2017-03-24xci: Configure ansible verbosity based on job typeFatih Degirmenci2-0/+6
Daily jobs will use pinned versions of bifrost and openstack-ansible in order to make sure we use known good versions for them since focus of the daily jobs is OPNFV platform testing. Ansible log can easily become massive depending on the level it is set and it is not so relevant for daily jobs to have it like this due to less probability of dailies failing to provision/deploy. On the other hand, the jobs that verify/test bifrost and openstack-ansible will have max verbosity enabled to troubleshoot these components themselves. New set of jobs will be created running periodically against the HEAD of master for bifrost and openstack-ansible to catch working versions and help us promote them to use for daily platform jobs. Verbosity of these jobs will be enabled like how we are doing for bifrost-verify jobs. Change-Id: Ic9fd2ff418fdaa3ad7fbfc22d8ec2dc1174df65f Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-24xci: Fix where the vm_defaults are placedFatih Degirmenci1-13/+13
Change-Id: I3d9d9b8b7c3de0c22a62945de027b9e8994d35e8 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-24xci: Merge bifrost provisioning scripts and renameFatih Degirmenci4-12/+61
Jobs are adjusted as well. Change-Id: I0bc5bdda37277053d81d7da394e0156902da817a Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-23xci: Pass parameters to downstream jobsFatih Degirmenci1-6/+18
Change-Id: I02f8c47fb19e6d47027c4655c233bf1952fabedd Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-23jjb: xci: xci-daily-jobs: Fix typoMarkos Chandras1-1/+1
Change-Id: Ib9116927c2169e033911d8b3fc64ba39232da6b0 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-03-22xci: Make it possible to select component versionsFatih Degirmenci3-21/+96
By replacing the branch with version, we now can specify sha1s per project; opnfv/releng, openstack-ansible, and bifrost. If nothing is specified manually (or via promotion), the defaults will be set to branches, master and stable/ocata. Change also includes further logging. Change-Id: Iae0f8cc60391caeb0f2735610050007014374a70 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-22jjb: xci: Fix permissions in workspace before removing itMarkos Chandras2-0/+32
From time to time we end up in the unfortunate situation where the jenkins job breaks for random reasons and the logs are left with 'root' permissions. On the next run, the job fails again but this time it's because Jenkins does not have enough permissions to delete them. We workaround this by fixing up the permissions before removing the workspace. Change-Id: I2fda20ded527bac73605ffa5818a2c44bf6c9026 Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-03-22xci: Fix the script name and remove openstack servicesFatih Degirmenci1-1/+1
Change-Id: I774e12f15561de75920ca50bcb2fbaf917e004a9 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-22xci: Logging, naming, and job updatesFatih Degirmenci1-6/+9
- adjust build blockers in daily jobs. - explicity set SLAVE_LABEL for provision, deploy, and functest jobs so they run on correct nodes even if they are started manually. - rename files and update scripts accordingly. osa_deploy.sh -> osa-deploy.sh targethost_configuration.yml -> configure-targethosts.yml jumphost_configuration.yml -> configure-xcimaster.yml - add/fix console logging in osa-deploy.sh. - add license header to osa-deploy.sh. Change-Id: I57a37dccaa4bcda554ef79969e5d288b49937807 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-22xci: Updates to releng/osa to make it work on JenkinsFatih Degirmenci1-1/+68
- change OSA_BRANCH to OPENSTACK_BRANCH to be consistent everywhere. - create opnfv-setup-openstack.yml playbook, getting rid of some components and copy it to xcimaster to use during the deployment. - switch to opnfv-setup-openstack.yml from setup-openstack.yml. - create Jenkins job wrapper to execute osa deploy script. - disable ansible strict host key checking. - use sudo -E as needed in order to make env vars set by Jenkins job and wrapper available to other scripts and to ansible. - rename jumphost to xcimaster so people do not mix it with actual jumphost located in Pharos PODs. - other minor fixes. Change-Id: I5736fa700c2aa39c598d301bc74450d766b74411 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-22xci: Updates to releng/bifrost to make it work on JenkinsFatih Degirmenci2-5/+5
- get rid of BAREMETAL_DATA_FILE and use BIFROST_INVENTORY_SOURCE so it works both for master and ocata for osa-bifrost. - set BIFROST_INVENTORY_SOURCE file according to branch bifrost is executed from for osa-bifrost. - explicitly set what the SSH public key file is as ansible copies the wrong public key to nodes if bifrost is executed on Jenkins using sudo. - set branches if they are not set so the scripts can be used manually as well. - rename jumphost to xcimaster so people do not mix it with actual jumphost located in Pharos PODs. Change-Id: Iff7631fa99816ad75316b62c5ac20714f67cd86a Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-21jjb: xci: Fix trigger for cleanup jobsAric Gardner1-10/+4
Trigger should only run on change-merged and change-abandonded The documentation could be imporved more explicitly here: https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#trigger-on but the modified lines do not take true or false they stand alone Fixes: ea33eda6954a ("jjb: infra: bifrost-cleanup-job: Do not trigger on new patchsets") Fixes: f6bba721fd13 ("jjb: infra: bifrost-cleanup-job: Do not trigger on patchset creation") Change-Id: Ibc574220ea86c5a039855d6f4fe35eef0aac7e0e Signed-off-by: Aric Gardner <agardner@linuxfoundation.org> Signed-off-by: Markos Chandras <mchandras@suse.de>
2017-03-20xci: Enable VM provisioning using bifrost for daily runsFatih Degirmenci3-43/+88
- create script to provision VMs (log upload is not enabled yet) - remove leftover inventory files (baremetal.csv, baremetal.json) - set BIFROST_INVENTORY_SOURCE as needed by stable/ocata - rename playbook used during bifrost and adjust scripts using it - remove puppet-infracloud stuff Change-Id: I1c4412d0725c00ab9a1b3497e358d33998b4e7e6 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-20xci: Create jobs for stable/ocataFatih Degirmenci1-0/+2
Change-Id: Ie45900e1724d761d6f547b159f5ec276d705326b Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2017-03-20xci: Create skeletons for jobs and rename directoryFatih Degirmenci7-0/+724
This patch - creates job skeletongs based on the structure of other OPNFV jobs upstream job per scenario triggering and controlling the downstream jobs downstream jobs to run provisioning, deployment and functest - renames infra directory to xci Change-Id: I9a7ac3590d660f4270ab387598586e4270787ecb Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>