summaryrefslogtreecommitdiffstats
path: root/config/utils
AgeCommit message (Collapse)AuthorFilesLines
2018-01-28[IDF] Add optional 'idf.installer' arrayAlexandru Avadanii1-15/+7
There is no easy way to determine automatically if a PDF/IDF set is compatible with a specific installer adapter. Most common incompatibilities are caused by: - installer adapters hardcode more network ports than nodes have defined in PDF; - installer adapters hardcode more nodes than PDF defines; Previous attempt at handling this dynamically (cd54edf) fails for recently added PODs, so let's handle this explicitly per-pod via a new optional IDF property, 'idf.installer'. If 'idf.installer' is defined, it will be used by `check-jinja2.sh` as a filter-in list of installer adapters to check against. The default (no 'idf.installer' present) is to check all IAs. Leverage the new property for: - arm-pod5: not enough network ports on computes for Compass, Joid; - arm-pod7: same as arm-pod5; - bii-pod1: no 'storage' network defined in IDF for Daisy; - zte-pod9: 3 cluster nodes are only supported by Daisy for now; SPEC changes: - net_config (previously mandatory for IDF version '0.1') is now mandatory only if 'idf.fuel' or 'idf.daisy' is defined; Fixes: cd54edf Change-Id: I706c05519e5491ad631069d7cc090e9c8bfd3011 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-26generate_config: Check yaml arg is really a fileAlexandru Avadanii1-1/+1
In order to support bash process substition, e.g. $ ./generate_config.py -j /path/to/template.j2 -y <(echo stuff); do not consume the contents of ARGS.yaml unless it is a file. Change-Id: I89053cfaadc158f6a15f85836892b21980081772 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-24[IDF] Add IDF schema validation based on versionAlexandru Avadanii3-5/+20
- add basic structure for IDF schema validation; - fill in Fuel section; - add stub for Daisy section; - add stub for Compass section; - add stub for net_config; - enable IDF schema validation via `check-schema.sh`; - prepare for new IDF versions with different schemas, while allowing subsection schema reuse from IDF v0.1; - extend validate_schema.py for IDF; Change-Id: I59b1c9d857e9dee9e6ff9956e81b2ae6dc27a767 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-22[PDF] [SPEC] Add 'version: 1.0'Alexandru Avadanii2-1/+14
- add 'version: 1.0' to PDF spec; - add 'version: 1.0' to all existing lab PDFs; - extend schema with new property; - add workaround for value-based decision-making in schema version selection via `validate-template.py`; - add support for multiple schema versions; - add versions for all schema blocks defined so far; - fix PDF schema pattern for disk size decimals (e.g. '1.8T'); Change-Id: Ie8f768803ec19f1f9a7982fe5ca59df80764fc4a Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-22Merge "[PDF] check-jinja: Filter-out incompatible PDF/IA"Alexandru Avadanii1-0/+18
2018-01-22Merge "generate_config: Add netmask jinja filter"Alexandru Avadanii1-0/+10
2018-01-22Merge "check-jinja: Add optional filtering of test matrix"Alexandru Avadanii1-2/+11
2018-01-22Merge "[PDF] config: Add/update license headers"Aric Gardner4-4/+4
2018-01-19check-jinja: Add optional filtering of test matrixAlexandru Avadanii1-2/+11
When adding new PDFs, it is sometimes useful to check only the newly added PDF (or all the PDFs for a specific lab) against all or only some installer templates. Change-Id: I1d02943527597df1e1908a3ff4398bc398c887e7 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-18generate_config: Add netmask jinja filterGuillermo Herrero1-0/+10
- Add custom filter to convert integer prefix network mask to IP format network mask string Change-Id: I13d6a2725b93132f858b64d7f93de3a4e0dc5961 Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
2018-01-16[PDF] check-jinja: Filter-out incompatible PDF/IAAlexandru Avadanii1-0/+18
While checking all PDFs against all installers, some mandatory requirements are sometimes not met: - minimum cluster nodes (e.g. Apex requires 5 nodes, 1 PDF has 3); - minimum interface count (e.g. Compass requires min 3 NICs, some PDFs only have two interfaces or less); The added filter is far from ideal: - node requirements are based on the latest (not highest) index used by the installer adapter to accomodate special handling of 3-node cluster in Daisy templates; - interface req is based on the average interfaces/node, with quite some margin - this is a very loose mechanism; With this in, `check-jinja` has a return code of 0 (note that yamllint failures of output files are not affecting the return code). Change-Id: I43ade3567bf3026069ff93eca17abc212fab211c Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-14[PDF] Add schema validationAlexandru Avadanii2-0/+64
- create new YAML schema for PDF validation; - add basic python script for checking a PDF against the schema; - add bash wrapper for checking all PDFs in Pharos, to be leveraged later via a new verify CI job; Change-Id: I47e02642756b7a231138dec3d5258b100b4db72b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-14[PDF] config: Add/update license headersAlexandru Avadanii4-4/+4
This only covers the `config` subdir. Change-Id: I1fb9470e216cc2de9a4ac91de44d99e7044e708e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-14[PDF] check-jinja: cleanup, compact outputAlexandru Avadanii1-16/+16
The Pharos git repo already has a CI verify job for `yamllint`-ing the PDF files (check-jinja input yaml), so drop that check and keep linting only output YAML files. While at it, slightly rework the output log to make it easier to read. Change-Id: I2e47902d71514709eb48432f87d6ce68fb795d73 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-14[PDF] check-jinja: Replace encrypted str w/ dummyAlexandru Avadanii1-1/+3
During PDF validation using `check-jinja.sh`, most if not all encrypted strings will fail to decrypt due to missing keys on the build server. The templater will fallback to using raw plaintext, leading to `line too long` yamllint violations. Since we don't care about the actual value of the unencrypted string, replace the encrypted one with a dummy (shorter) string. Change-Id: I0f96db0e055bf84ad43a1df4a0f2bd86cc50fd22 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-14[PDF] check-jinja: Disable check against templateAlexandru Avadanii1-1/+1
config/pdf/pod1*yaml files are not valid PDFs, but only templates. Trying to expand said files against installer adapters would lead to funky results, since most fields are expressed as '{val1|val2}' in the template files. Change-Id: Ieec4d596d0b5246aa506239183a31463951403e7 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-14[PDF] generate_config: eyaml only encrypted PDFsAlexandru Avadanii2-5/+9
Silence misleading warnings about failed decription for PDFs that do not use this feature. While at it, print yamllint version used by check-jinja.sh. Change-Id: Ica1ff90abaee8c9bb20996899c8f0a7527133618 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-20validate-pdf-jinja2-templates-master failingagardner2-4/+4
If keys do not exist, the skipping decryption python stuff doesn't work, see: https://build.opnfv.org/ci/job/validate-pdf-jinja2-templates-master/16/consoleFull Still does not fallback to plain in the case of a 'bad decryption', but the current code does not either, bad decryption is logged in any case. Change-Id: I8da2c7cf8568a6e77ca24cc9ce665410d93b7ba9 Signed-off-by: agardner <agardner@linuxfoundation.org>
2017-12-15Update the example config filename in eyaml READMEAlex Yang1-1/+1
The example configuration file in the directory pharos/config/utils/ is config.example.yaml. Change-Id: I493274e4ddb689eb729ed642b5f495b5be0b75b0 Signed-off-by: Alex Yang <yangyang1@zte.com.cn>
2017-12-13Update eyaml READMETrevor Bramwell1-1/+2
On CentOS7 systems the gem command is contained in the 'rubygems' package, not 'ruby-gems'. After the keys have been moved, there is not reason to keep around the 'keys' directory. Change-Id: I5cb9f474276de75b650ddf323a279e139fb62d37 Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
2017-12-09generate_config.py: Fix running without eyamlAlexandru Avadanii1-3/+7
If eyaml is not installed or misconfigured, we should output raw encrypted data and throw a warn/error accordingly, instead of bailing out completely. Needed for Fuel deploys on PODs that do not use PDF encryption, e.g. lf-pod2. Change-Id: I2905df11a3cf2f2eb1ab9bfd8bd88619af2cab04 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-24Jenkins does not have /usr/local/bin/ in pathagardner1-0/+1
Stupid jenkins Change-Id: Ibfd83e7dadd674cfe7d7a9c2a712ef069fb8d5de Signed-off-by: agardner <agardner@linuxfoundation.org>
2017-11-23Moving check-jinja2 to pharos.agardner1-0/+66
Securedlab will be going away now that we can encrypt strings (eyaml) we need to check jinja2 PDF templates that will be moving into pharos This adds one pod yaml files and fixes paths in check-jinja2.sh After this is merged I can create a jenkins job for pharos. Then I can start moving pdf files into the pharos repo Change-Id: I2f2f2925275de49035d7d0160249d75b25ca0d20 Signed-off-by: agardner <agardner@linuxfoundation.org>
2017-11-17generate_config: Use eyaml to decrypt secret valuesAlexandru Avadanii3-4/+101
Note: IDF data encryption is not supported. Supporting that is trivial, but it leads to slightly more complicated code, plus it breaks support for multiline scalar encrypted data in the PDF ('>'), forcing us to define each encrypted value as inline string. While at it, fix silly limitation of jinja2 path residing in a subdir of CWD. Change-Id: I441ec754d8b6e4aad2ed73aba0b9b18ed65f05f4 Signed-off-by: agardner <agardner@linuxfoundation.org> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-26PDF: generate_config: Add IDF parsing supportAlexandru Avadanii1-0/+8
For a given 'podN.yaml' file, check whether a file named 'idf-podN.yaml' exists in the same directory, and if it does, pass its contents under the 'idf' dict key to the j2 template. This assumes the contents of IDF yaml have a root 'idf' key. Change-Id: I6c6f1b9f28c38989f8a6ed4a389c9a1da423d76d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-18config/utils: Add generate_config.pyAlexandru Avadanii1-0/+53
Previously, generate_config.py resided in securedlab git repo, but since we want to be able to use it in both securedlab (for validation of new PDF files during verify jobs) and installer projects (for actually parsing the PDF file into usable installer inputs), we decided to move it to a common location, also available for regular users. This change merely replicates the file from securedlab git repo. Change-Id: I9ff7889e408338d3911853fe01b752b013de1db7 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>