astropy/asdf-astropy#326 reminds me that it would be really good to cross-check more of libasdf's outputs against the Python reference implementation (ensure they can at least be read).
Here we are less concerned about whether the files are treated exactly equivalently (there's no reason they shouldn't be), but to at least ensure that files output by libasdf can be parsed and validated by Python ASDF. This can reveal bugs both in the Python code if libasdf's tests are generating cases that weren't tested in Python as in the aforementioned PR. It can also of course reveal bugs in libasdf, especially w.r.t. to the schemas, since libasdf does not have any schema validation yet.
Would be good to run these checks against files in tests/fixtures/ that were generated from libasdf, as well as at least any temp files written by the test suite. This would be an optional test that can be run if Python is available and a virtualenv can be made available with the necessary dependencies. Would not want such a test enabled by default since it would slow down the test runner, but good to have at least as a special case that is run once (e.g. in the CI).
astropy/asdf-astropy#326 reminds me that it would be really good to cross-check more of libasdf's outputs against the Python reference implementation (ensure they can at least be read).
Here we are less concerned about whether the files are treated exactly equivalently (there's no reason they shouldn't be), but to at least ensure that files output by libasdf can be parsed and validated by Python ASDF. This can reveal bugs both in the Python code if libasdf's tests are generating cases that weren't tested in Python as in the aforementioned PR. It can also of course reveal bugs in libasdf, especially w.r.t. to the schemas, since libasdf does not have any schema validation yet.
Would be good to run these checks against files in
tests/fixtures/that were generated from libasdf, as well as at least any temp files written by the test suite. This would be an optional test that can be run if Python is available and a virtualenv can be made available with the necessary dependencies. Would not want such a test enabled by default since it would slow down the test runner, but good to have at least as a special case that is run once (e.g. in the CI).