run tests according to purl-test.schema-0.2.json - #106
petergardfjall wants to merge 1 commit into
Conversation
| git submodule update --init | ||
|
|
||
| # Bring in the latest version of the upstream testspec. | ||
| testsuite-update: testdata/purl-spec/tests |
There was a problem hiding this comment.
Let's make this an explicit step, instead of pulling in changes every time we run make test (which makes testing rely on timing).
There was a problem hiding this comment.
Yeah, that makes sense.
| ExpectedOutput ComponentsOrPurl `json:"expected_output"` | ||
| ExpectedFailureReason *string `json:"expected_failure_reason"` | ||
| // TestGroup indicates a conformance group for a [PurlTest]. | ||
| type TestGroup string |
There was a problem hiding this comment.
I've tried to update types and comments to mirror https://github.com/package-url/purl-spec/blob/main/schemas/purl-test.schema-0.2.json (also described in https://github.com/package-url/purl-spec/blob/main/docs/tests/test-suite.md)
| if err != nil { | ||
| t.Logf("%s failed: %s", tc.Description, err) | ||
| t.Fail() | ||
| return |
There was a problem hiding this comment.
Avoids nil pointer panic. Since we've already failed we can just abort.
| // TestCoreSpec runs purl-spec tests that are for the core specification and not for a specific PURL | ||
| // type. | ||
| func TestCoreSpec(t *testing.T) { | ||
| testFiles, err := readJSONFilesFromDir("testdata/purl-spec/tests/spec/") |
There was a problem hiding this comment.
I think we did not run through this test file before, only the files for purl types.
b96ff7c to
dffbbbb
Compare
keshav-space
left a comment
keshav-space
left a comment
There was a problem hiding this comment.
Thanks @petergardfjall, LGTM! 🙇
Also, we should enter an issue for the failing tests.
2 participants
Updates the test execution and test case parsing for the upstream purl-spec tests.
These tests have been updated (from version
0.1to0.2) to follow a new schema: https://github.com/package-url/purl-spec/blob/main/docs/tests/test-schema-changes.mdThis PR intends to fix the testsuite execution.
It should be noted that not all tests pass but I think that should be fixed in a separate PR, since it might require more substantial changes to the library (the separation of parsing and validation). The
pkg:gittestcases are still up for debate also: