summaryrefslogtreecommitdiffstats
path: root/config/utils/generate_config.py
AgeCommit message (Collapse)AuthorFilesLines
2018-04-30[PDF] Preparations for vPDF supportAlexandru Avadanii1-2/+2
- extend check-{jinja2,schema}.sh to scan vPDFs and construct proper IDF path for vPDFs; - add support for '/' in PDF fields, by default filtered by j2 template rendering; SPEC changes: - extend remote_management 'type' enum with 'libvirt'; SCHEMA changes: - add new enum value 'libvirt' according to SPEC update; - add separate handling of 'remote_management' for baremetal/virtual; - remote_management 'versions' and 'mac_address' are mandatory for all but 'libvirt' (used to be mandatory for all); Change-Id: Ib3f711cae807c3ab27920391402ba74844f155c3 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-05generate_config: minor code formatting updatesAlexandru Avadanii1-2/+4
Change-Id: If0ee12287f4257f2de315bd6ad82edb709e24492 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-02generate_config: New `-b` arg for batch processingAlexandru Avadanii1-14/+32
Sometimes, parsing multiple j2 templates based on the same PDF+IDF is needed, in which case we'd have to call generate_config once for each template. Add a new argument, `-b`, which allows batch processing of multiple templates in one go: - files ending in '.j2' (e.g. 'template.yml.j2') will be expanded to a file without the '.j2' suffix in the same directory as the source template (e.g. 'template.yml'); - templates not ending in '.j2' are skipped in batch mode; - in order to pass multiple templates, `-j` argument can now be passed multiple times, similar to `-i`; - although relative filepaths still work, it is highly recommended to use the full path (abspath) for j2 templates; - to avoid filename collisions, j2 parser will open '/' (as well as any directories specified via `-i`) and lookup each j2 template by its full path; Add a new argument, `-v` for verbose logging. While at it, import only needed functions from 'os.path' to make our code more compact and easier to read. Change-Id: I2291935d42e4196813cd75154d5b674bcc330183 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-02Merge "config/utils: YAML: Use C bindings if available"Alexandru Avadanii1-4/+5
2018-03-01generate_config: New `-i` arg for j2 includes dirAlexandru Avadanii1-1/+2
jinja2 does not support imports from parent directories, so in order for j2 to be able to use '{% from <some>.j2 import ... %}' constructs with included j2 residing on a separate dirpath than current j2, add support for explicitly adding directories to the templater search path. This also allows calling generate_config with only file basename as '-j' arg value, as long as the dirname is passed via '-i' (can be extended later to parsing multiple j2 files at once). Change-Id: I943fda9262a6f7de5ee6fcc3e7bc9828c339651a Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-01generate_config: Enable 'do' j2 extensionAlexandru Avadanii1-1/+4
'do' is useful to avoid hacks like {% set _ = x.update(...) %}. Instead: {% do x.update(...) %}. [1] http://jinja.pocoo.org/docs/2.10/extensions/ Change-Id: I886beb97e0383ba11b81aa21cf12b11ebc4b05b2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-01config/utils: YAML: Use C bindings if availableAlexandru Avadanii1-4/+5
Time diff before/after this change for the same template via generate_config expansion: -user 0m0.144s +user 0m0.096s Change-Id: Id574afcd8a74a530d791ebed8b72ccae53703611 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-20generate_config: Move filters to external libraryGuillermo Herrero1-43/+7
Clean and isolate main script logic from custom filters by moving all them to a external library file. Library has a function to load all the existing filters at once. New enhanced IP handling custom filters: - ipnet_hostaddr - ipnet_hostmin - ipnet_hostmax - ipnet_broadcast - ipnet_netmask - ipnet_contains_ip - ipnet_contains_iprange - ipnet_range_size ipnet filters work with proper IP network with prefixlen mask, ensuring consistent IP calculations and error handling. Previous IP handling filters should be deprecated. Change-Id: I83c41d7ad3c6bd1d9df1deca6cc5b9d2481ecf52 Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
2018-01-26generate_config: Check yaml arg is really a fileAlexandru Avadanii1-1/+1
In order to support bash process substition, e.g. $ ./generate_config.py -j /path/to/template.j2 -y <(echo stuff); do not consume the contents of ARGS.yaml unless it is a file. Change-Id: I89053cfaadc158f6a15f85836892b21980081772 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-22Merge "generate_config: Add netmask jinja filter"Alexandru Avadanii1-0/+10
2018-01-18generate_config: Add netmask jinja filterGuillermo Herrero1-0/+10
- Add custom filter to convert integer prefix network mask to IP format network mask string Change-Id: I13d6a2725b93132f858b64d7f93de3a4e0dc5961 Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
2018-01-14[PDF] config: Add/update license headersAlexandru Avadanii1-1/+1
This only covers the `config` subdir. Change-Id: I1fb9470e216cc2de9a4ac91de44d99e7044e708e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-14[PDF] generate_config: eyaml only encrypted PDFsAlexandru Avadanii1-5/+6
Silence misleading warnings about failed decription for PDFs that do not use this feature. While at it, print yamllint version used by check-jinja.sh. Change-Id: Ica1ff90abaee8c9bb20996899c8f0a7527133618 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-20validate-pdf-jinja2-templates-master failingagardner1-3/+3
If keys do not exist, the skipping decryption python stuff doesn't work, see: https://build.opnfv.org/ci/job/validate-pdf-jinja2-templates-master/16/consoleFull Still does not fallback to plain in the case of a 'bad decryption', but the current code does not either, bad decryption is logged in any case. Change-Id: I8da2c7cf8568a6e77ca24cc9ce665410d93b7ba9 Signed-off-by: agardner <agardner@linuxfoundation.org>
2017-12-09generate_config.py: Fix running without eyamlAlexandru Avadanii1-3/+7
If eyaml is not installed or misconfigured, we should output raw encrypted data and throw a warn/error accordingly, instead of bailing out completely. Needed for Fuel deploys on PODs that do not use PDF encryption, e.g. lf-pod2. Change-Id: I2905df11a3cf2f2eb1ab9bfd8bd88619af2cab04 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-17generate_config: Use eyaml to decrypt secret valuesAlexandru Avadanii1-4/+23
Note: IDF data encryption is not supported. Supporting that is trivial, but it leads to slightly more complicated code, plus it breaks support for multiline scalar encrypted data in the PDF ('>'), forcing us to define each encrypted value as inline string. While at it, fix silly limitation of jinja2 path residing in a subdir of CWD. Change-Id: I441ec754d8b6e4aad2ed73aba0b9b18ed65f05f4 Signed-off-by: agardner <agardner@linuxfoundation.org> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-26PDF: generate_config: Add IDF parsing supportAlexandru Avadanii1-0/+8
For a given 'podN.yaml' file, check whether a file named 'idf-podN.yaml' exists in the same directory, and if it does, pass its contents under the 'idf' dict key to the j2 template. This assumes the contents of IDF yaml have a root 'idf' key. Change-Id: I6c6f1b9f28c38989f8a6ed4a389c9a1da423d76d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-18config/utils: Add generate_config.pyAlexandru Avadanii1-0/+53
Previously, generate_config.py resided in securedlab git repo, but since we want to be able to use it in both securedlab (for validation of new PDF files during verify jobs) and installer projects (for actually parsing the PDF file into usable installer inputs), we decided to move it to a common location, also available for regular users. This change merely replicates the file from securedlab git repo. Change-Id: I9ff7889e408338d3911853fe01b752b013de1db7 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>