Skip to content

generators.md

Latest commit

 

History

History
63 lines (46 loc) · 5.15 KB

generators.md

File metadata and controls

63 lines (46 loc) · 5.15 KB
Raw

Generators

Generators are provided in the generators/ directory and may be used to generate test cases. If it is present, the file generators/generators.yaml specifies which test cases should be generated and which commands should be run to generate them. See generators.yaml for a full example configuration with comments explaining all the valid keys.

When generators/generators.yaml is present, all test cases in data/{sample,secret,...} must be generated by it. It is not allowed to generate some test cases and some not.

See ./generators.yaml for an exhaustive example file with explanations.

A Json schema can be found here, which can be used together with most YAML LSP's to provide autocompletion and validation when editing generator.yaml files.

A CUE specification can be found here. A generators.yaml file can be validated against this spec using the cue command line tool.

Specification

The two main object types are directory and generator. The root of generators.yaml is a directory which corresponds to the data/ directory.

Directory objects take the following keys:

  • test_group.yaml: Optional yaml configuration that will be copied to test_group.yaml in this directory.
  • solution: Optional invocation of a solution to be used to generate .ans files. Set to empty to disable generating .ans. (Useful for e.g. the data/samples/ directory.) This must be an absolute path relative to the problem root.
  • random_salt: Optional string that will be prepended to each command before computing its {seed}. May be used to regenerate all random cases and to prevent predictable seeds.
  • retries: Optional int that specifies the maximum number of invocation that will be tried if the generator fails. Each invocation will use a different value for {seed}.
  • data: The test cases / test groups contained in this directory. This may take two forms:
    • A dictionary, each key is the name of a test case/test group, and each value must be a directory or generator object.
    • A list of dictionaries as above. In this case, test cases will be prefixed with zero padded 1-based integers in the order of the list. Items in the same dictionary will get the same number.
  • include: Optional list of Directory object names (as strings) e.g. - "sample". All test cases from those directories are linked for this directory.

Generator objects have the following forms:

  • generate: <generator_name> <arguments>. <generator_name> must either be a program (file/directory) in generators/ or else a name in the top level generators dictionary (see below). Arguments may contain {name} to refer to the name of the test case and {seed} or {seed:(0-9)+} to add a random seed. Arguments are separated by white space (space, tab, newline). Quoting white space is not supported.
  • copy: <base_path>: For all known extensions <ext> the corresponding file <base_path>.<ext> will be copied to the specified location should it be present. <base_path> must be relative to generators/.
  • <ext>: <content>: A file with extension ext and the content will be generated. <ext> must be a known file extension.
  • count: <int> | [<int>] | <int>..=<int>. To generate multiple Generator objects. If generate is used and {seed} or {seed:(0-9)+} is present all Generator objects will use a different seed. The arguments of generate may contain {count} to refer to the index of this generator invocation. By default, the indices generated by count: 3 are [1,2,3]. Specify other indices as a list count: [1,2,3] or as a (Rust-like, endpoint-inclusive) range expression count: 1..=3.
  • match: Optional str or list of str. Each entry should be a regex pattern. If the generated test case does not match a pattern a warning will be shown.
  • solution, random_salt, and retries: see Directory objects.

The follwoing things should hold:

  • A .in file must be specified/generated by this
  • If a .ans file is not specified/generated, a solution must be provided that will be used to generate the .ans. For interactive or multi-pass problems, an empty .ans will be generated.

Root object The root of the generators.yaml is a directory object with one optional additional key:

  • generators: a dictionary mapping generator names to a list of dependencies. This must be used when using non-directory generators that depend on other files in the generators/ directory. Each key of the dictionary is the name of a generator, and values must be lists of file paths relative to generators/.

    When this dictionary contains a name that's also a file in generators/, the version specified in the generators dictionary will have priority.

    Generators specified in the generators dictionary are built by coping the list of files into a new directory, and then building the resulting program as usual. The first dependency listed will be used to determine the entry point.

    Other generators are built as (file or directory) programs.