csv-spectrum

A fixed set of CSV files that stress-test CSV parsers, each paired with the JSON its parse result should equal. Use it as an acid test for your parser.

The crate hands you the raw bytes of each CSV and its expected JSON. You run your parser and assert the result matches.

What the corpus covers

• Commas inside quoted fields
• Escaped (doubled) quotes
• Embedded LF and CRLF inside quoted fields
• Empty quoted fields as empty strings
• Multibyte UTF-8 in a field
• UTF-8 replacement bytes passed through unchanged
• A JSON document stored in a single cell
• LF and CRLF record separators

Installation

[dependencies]
csv-spectrum = "0.1"

Usage

let samples = csv_spectrum::spectrum();
assert_eq!(samples.len(), 12);

for sample in samples {
    // sample.csv  -> raw CSV bytes
    // sample.json -> raw expected JSON bytes
    // sample.name -> e.g. "comma_in_quotes"

    // location_coordinates masks its phone field and its JSON is a lone
    // object, so a correct parse cannot equal it. Check that one separately.
    if sample.name == "location_coordinates" {
        continue;
    }
    let parsed = my_parser(sample.csv);
    let expected = serde_json::from_slice(sample.json).unwrap();
    assert_eq!(parsed, expected, "parser failed on {}", sample.name);
}

SAMPLES exposes the same data as a const array for use in const contexts.

Raw bytes

csv and json are &[u8]. The bytes come back exactly as stored. CRLF stays CRLF. Trailing newlines are kept or omitted per file. Treat the corpus as opaque bytes.

The samples

name	exercises
comma_in_quotes	comma inside a quoted field, leading-zero string 08123
empty	empty quoted fields become empty strings
empty_crlf	same as empty, CRLF line endings
escaped_quotes	doubled quotes collapse to one
json	a JSON document held in one cell
location_coordinates	UTF-8 replacement bytes, expected JSON is a lone object
newlines	LF inside a quoted field
newlines_crlf	CRLF inside a quoted field, kept as \r\n
quotes_and_newlines	embedded newlines and escaped quotes together
simple	header plus one unquoted row
simple_crlf	baseline with CRLF
utf8	multibyte UTF-8, final cell is ʤ

Two quirks live in the corpus on purpose. location_coordinates.json is a single object while every other expected JSON is an array. Its phone value is masked and differs from the CSV cell. Both are kept as is.

License

Licensed under the MIT license.