Feature: Templates in configuration

@TeresiaOlsson

Description, motivation and use case
I thought harder after the community meeting to try and find a way to simplify configuration. Typically when writing the configuration files, they include devices with highly repeatable config. A "template" could be used to drastically reduce the number of lines in the configuration files.

@TeresiaOlsson @simoneliuzzo @gubaidulinvadim @GamelinAl @gupichon @JeanLucPons
I would very much like your opinion.

Proposed solution
A template in the configuration file can be used to repeat devices.

Example
The idea would be to have something like this in the configuration file:

- class: pyaml.configuration.template.Template
  template: |
    class: pyaml.magnet.hcorrector.HCorrector
    name: %template_parameter%
    model:
      class: pyaml.magnet.linear_model.LinearMagnetModel
      unit: rad
      hardware_unit: str
      calibration_factor: 1.0
      powerconverter: MAGNET/%template_parameter%/B1L
  string_to_replace: "%template_parameter%"
  parameter_list:
    - PKDK_WL_108
    - PKDK_WL_93
    - PKDK_WL_79
    - PKDK_WL_64
- class: pyaml.magnet.hcorrector.HCorrector
  name: ANOTHER
  model:
    class: pyaml.magnet.linear_model.LinearMagnetModel
    unit: rad
    hardware_unit: str
    calibration_factor: 1.0
    powerconverter: MAGNET/ANOTHER/B1L

then this could be expanded when building the storage ring.

Additional context
I have opened a draft PR #442 where I tried to implement something like this. It is definitely not perfect and there are a lot of edge cases that may cause problems.

Checklist

I've assigned this issue to a project
I've @-mentioned relevant people

I think it's a nice idea. I'm wondering if it might be useful to combine with jinja? Or maybe that's already part of the idea?

I think it's a nice idea. I'm wondering if it might be useful to combine with jinja? Or maybe that's already part of the idea?

I have not considered jinja. I do not know anything about it actually.

I think it's a nice idea. I'm wondering if it might be useful to combine with jinja? Or maybe that's already part of the idea?

I have not considered jinja. I do not know anything about it actually.

I think it's quite nice. I only know of it because catap uses it for it's configuration. And then I discovered that the copier template also use it when I started setting that up.

I'm wondering now if it might be possible to generate the templates automatically from the JSON Schema? The JSON Schema is kind of a template. Just a very non user-friendly one... But to extract the JSON Schema is very easy and it might be possible to write a second step that from it produces user-friendly templates?

I agree that it is a good idea. I'm rather against Jinja which already causes me strong headache.

You should find a way to define your template and instanciate it. Not necessary link to a class.
We could imagine to have a template placeholder.
i.e.:

template:
   - name: SHI
     parameters:
       - sh_name
     code: 
      type: pyaml.magnet.cfm_magnet
      name: ${sh_name}
      mapping:
        - [B0, ${sh_name}-H]
        - [A0, ${sh_name}-V]
        - [A1, ${sh_name}-SQ]
        - [B2, ${sh_name}-S]
      model:
        type: pyaml.magnet.linear_cfm_model
        multipoles: [B0,A0,A1,B2]
        pseudo_factors: [1.0,-1.0,-1.0,-1.0]
        units: [rad,rad,m-1,m-2]
        hardware_units: ["A","A","A","A","A"]
        curves:
          - type: pyaml.magnet.csvcurve
            file: sr/magnet_models/SHI_h_strength.csv
          - type: pyaml.magnet.csvcurve
            file: sr/magnet_models/SHI_v_strength.csv
          - type: pyaml.magnet.csvcurve
            file: sr/magnet_models/SHI_sq_strength.csv
          - type: pyaml.magnet.csvcurve
            file: sr/magnet_models/SHI_sext_strength.csv
        matrix:
          type: pyaml.magnet.csvmatrix
          file: sr/magnet_models/SHI_matrix.csv
        powerconverters:
          - srmag/ps-corr-${sh_name[:4]}/${sh_name[:3]}-${sh_name[4]}-ch1/current
          - srmag/ps-corr-${sh_name[:4]}/${sh_name[:3]}-${sh_name[4]}-ch2/current
          - srmag/ps-corr-${sh_name[:4]}/${sh_name[:3]}-${sh_name[4]}-ch3/current
          - srmag/ps-corr-${sh_name[:4]}/${sh_name[:3]}-${sh_name[4]}-ch5/current
          - srmag/ps-corr-${sh_name[:4]}/${sh_name[:3]}-${sh_name[4]}-ch6/current

devices:
- SHI("SH1A-C04")

The syntax could be discussed. Here the ${} i propose will conflict with macros, but the idea is there.
Inside the ${ } a python code could be executed rather than a Jinja code. (Jinja code is not convenient at all)
It is rather easy to do directly in python.

I like the template idea!

Maybe jinja is not the best option then but I think it is anyway worth to investigate if there is some other template standard we can use. Inventing our own syntax for something that already exists feels like it can just mean more maintenance work.

A quick search gave me these options that might be easier but I haven't tried them.

Here the syntax is just python expect for the delimiter, compatible with yaml and our architecture.
IMHO, we don't need something very complex and adding new dependencies is more maintenance as you have to follow them and expect that you will have good support.
For instance what I really dislike in Jinja is that you need to produces a file which is no longer a yaml file neither a Jinja file and you lose code highlighting, code completion, etc, (may be it exits some plugin for vscode to handle this) and you also have to write a model for it.

Mm. I did try the jinja extension in VS code since it's needed for the docs. It didn't work well for me...

@JeanLucPons

I have opened a second pull request to implement a template that is more-or-less what @JeanLucPons proposed here.

variation 1, PR #442 :

class: pyaml.accelerator.Accelerator
machine: sr
facility: PETRAIII
energy: 6e9
controls:
 - class: tango.pyaml.controlsystem.TangoControlSystem
   tango_host: ebs-simu-3:10000
   name: live
   catalog:
     class: tango.pyaml.tango_catalog.TangoCatalog
     disconnected: True
devices:
- class: pyaml.configuration.template.Template
  name: my_template
  template: |
    class: pyaml.magnet.hcorrector.HCorrector
    name: {sh_name}
    model:
      class: pyaml.magnet.linear_model.LinearMagnetModel
      unit: rad
      hardware_unit: str
      calibration_factor: 1.0
      powerconverter: tango/MAGNET/{sh_name}/B1L
  string_to_replace: "{sh_name}"
  parameter_list:
    - SHI-01
    - SHI-02
    - SHI-03

variation 2, PR #456 :

class: pyaml.accelerator.Accelerator
machine: sr
facility: PETRAIII
energy: 6e9
controls:
 - class: tango.pyaml.controlsystem.TangoControlSystem
   tango_host: ebs-simu-3:10000
   name: live
   catalog:
     class: tango.pyaml.tango_catalog.TangoCatalog
     disconnected: True
devices:
- ${template:SHI,SHI,01}
- ${template:SHI,SHI,02}
- ${template:SHI,SHI,03}
templates:
  - name: SHI
    parameters:
      - sh_name
      - ii
    config:
      class: pyaml.magnet.hcorrector.HCorrector
      name: "{sh_name}-{ii}"
      model:
        class: pyaml.magnet.linear_model.LinearMagnetModel
        unit: rad
        hardware_unit: str
        calibration_factor: 1.0
        powerconverter: tango/MAGNET/{sh_name}-{ii}/B1L

To me, Variation 2 looks much better from a user's point of view. A con of Variation 1 is that it will trigger an error if the user forgets to put the "|" character next to "template: ".

On the other hand, I find Variation 1's implementation cleaner and less invasive. Both variations though needed to put a special case in either factory.py (Var1) or fileloader.py (Var2).

I personally favor Variaton 2.

The pull requests are not ready to merge but they are in state where the example runs and the implementations can be reviewed before continuing with tests, docs, e.t.c.

Personally, i prefer the version 2 which looks more standard.

Navigation Menu

Repository navigation

Feature: Templates in configuration #443

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Feature: Templates in configuration #443

Description

Activity

TeresiaOlsson commented on Sep 18, 2026

kparasch commented on Sep 18, 2026

TeresiaOlsson commented on Sep 18, 2026

JeanLucPons commented on Sep 18, 2026

JeanLucPons commented on Sep 18, 2026

simoneliuzzo commented on Sep 18, 2026

TeresiaOlsson commented on Sep 18, 2026

JeanLucPons commented on Sep 18, 2026

TeresiaOlsson commented on Sep 18, 2026

kparasch commented on Sep 28, 2026

JeanLucPons commented on Sep 29, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions