[jo-mueller]

So...since updating version keys across the project already bit me a few times, this would be a solution to starting having this key replaced by a placeholder in the various places where it occurs.

I have added a new function in the _prebuild.py script, that goes through all json in the examples dir, searches for "{{ VERSION }}" and replaces it with whatever is written in _version.py, so that would be the only place in the future where the version would need to be updated. For this to work, all the json examples need to sit in a examples/_static directory and the updated json files are then kept at examples/_build.

To be considered: It would be nice to have this ´_version.py´ under the control of a versioning tool like setuptools_scm, but these fundamentally conflict with versions that have .dev in them. So if we wanted to do this in the future, maybe we should think about a different tag. But maybe just updating the version in a single file wouldn't be too much overhead 🤷

This is to be done with

• json files under examples/
• index.md (The actual spec)
• json file in tests

This PR will conflict big time with #140 , so I would opt for that to go in first and then I'll continue here or I'll rebase right away.

cc @jni

[jni]

Oh amazing, I would have loved to have this for #190 LOL.

[jo-mueller]

Oh amazing, I would have loved to have this for #190 LOL.

Right? It's very annoying to replace all the versions everywhere and a perfect source for mistakes.

[clbarnes]

Points I made in sync:

1. Yes please, not having to do a ton of version replacement and adding that git churn would be great
2. I'm wary about mixing template files and non-template files without clearly marking which are which. I think if we're doing templating, we should do templating. I would suggest minijinja, which is a simplified dialect of the well-known jinja2 system with broad language support (albeit some of it vibe-coded, although by a very competent developer) and high-performance implementations. There is a sphinx plugin (confusingly called jinja2, despite not using jinja2) which has direct support for MyST markdown. Slight annoyance is that adding .template (or .jinja) everywhere may break some IDE auto-highlighting and previewing.

[jni]

@clbarnes I expect we will want to merge some version of this PR before 0.9.dev2? I like your point about marking template files.

[jo-mueller]

If I understood correctly, then the entirety of the zarr-attributes examples in the tests directory of the repo will be moved out to oztest, right? In that case, this PR just needs to be rebased/conflict-resolved and should be the bulk of the work on making this happen?

[clbarnes]

@jni I'm not opposed to this change generally but I'm also not convinced that this needs to go into 0.9.dev2 (or before). It involves some significant subjective DX changes; mainly that practically no files (schemas, test cases, or markdown) will be usable straight from the repo without pre-building, and that the pre-built files then go in a subdirectory (which I prefer to the current system of scattering them throughout the repo) which is what sphinx then works on. Not adding a file extension for the templates would be confusing, but adding the file extension means we lose out-of-the-box language-specific tools like syntax highlighting.

Once oztest is ready for use and conformance tests are moved out of this repo the version string replacing gets significantly less onerous.

I don't think any of these are blockers but I don't think they're slam-dunk enough to be worth holding up dev2 and making it even less reviewable. I'd be more inclined towards a halfway house like a python script which trawls through the specs, examples, and test cases to update version strings in a context-aware way rather than relying on a context-unaware sed s/0.9.dev1/0.9.dev2/g.

[jo-mueller]

fwiw, I would contain the kind of templating as it is done here in the PR to only these files. @clbarnes is right though that the usage of a template tool (i.e. jinjia2) will be the appropriate way to handle filling the placeholders. The tricky part will be to do it in such a way that we don't need to do a lot of specialcasing when it comes to embedding all the documents over at the ngff page 🙈

[clbarnes]

I wrote a version-bumping script here https://github.com/clbarnes/ngff-spec/pull/4/changes

It does everything except index.md at the moment. By default it prints all the diffs; there's an --execute flag to write the changes. It would be easy enough to add an interactive mode where you verify each diff one at a time, but that would still be tedious.

It doesn't change the git churn but helps with false positives in find-and-replace.

[jni]

It doesn't change the git churn but helps with false positives in find-and-replace.

That is still very very useful 😅

[clbarnes]

Looks like there is a myst plugin which handles this https://github.com/myst-contrib/myst-substitutions (found at jupyter-book/mystmd#852 (comment) ). If that could cover the myst files and a script can do the structured replacing, that might solve it without having to hand-roll or bring in a whole other templating engine ourselves.

[clbarnes]

citing.md is already using some sort of template syntax - how is that handled?

[jo-mueller]

I think I stumbled over myst-substitutions as well, but figured that it didn't look mature enough to be used here :/

[clbarnes]

Ah, it doesn't work in code blocks. Fair, I guess. Well, my branch is up for updating the structured files, and I was thinking about an interactive picker for updating index.md, but I don't want it to hold up dev2.

[jo-mueller]

Will close this. @clbarnes introduced a version bump script on the 0.9dev branch which does this more elegantly.