feat(sbom): report the source language of cataloged packages - #328
Conversation
Verification
Review focus
|
a223005 to
af1e841
Compare
CI status (run 35580475428, 8 attempts)Every failure class is environmental; none is attributable to this diff:
Also: |
698e369 to
6e1d92d
Compare
af1e841 to
46af676
Compare
There was a problem hiding this comment.
Static review (no build or tests). No major issues found; below are minors worth closing before the draft is lifted.
What was checked: the 23-file diff against origin/main (merge-base c4efe7aa), the ConvergeWithMerge → FilterBOMBySourcePaths → MergeBOMs/os-pm → Upsert flow, the sbom merge → NewImageSBOM → ContainerAssembler/OSSAssembler flow, purl-based dedup in DedupBOM, presence of the e2e fixtures inject/pip_simple, inject/npm_simple and the FindComponent/buildTrustedBuilderBase helpers.
Remarks not tied to a line:
- Language vocabulary: pm vs werf. Directives write
Go/Python/…, while os-pm takessrcLanguagesfrom the pm catalogue as is.lo.Uniqis case-sensitive, sogofrom the catalogue andGofrom a directive would yieldGo, goon the product component. As long as builder images carry nosrcLanguages(marked UNVERIFIED in the description) there is nothing to test against; either pin the vocabulary on the pm side or normalize case innormalizeSourceLangs. - First-wins dedup.
mergeOrderputs the target last anddedupComponentsByPURLkeeps the first occurrence. If the same purl is already present in the SBOM of a base image built by an older werf (an external builder whose artifact the format-version bump does not regenerate), the copy withoutGOST:source_langssurvives even though the current scan stamped it. The dedup behavior is not new, but the description's "property appears on rebuild" holds only for images whose base SBOMs are regenerated as well. pkg/sbom/ispras/gost_test.go:13,27— per AGENTS.md thecomponentWithLangs/imageSBOMhelpers belong inhelpers_test.go.- The merge level (product/container) is covered by unit tests only; there is no e2e on
sbom mergewithGOST:source_langs. The PR description promises the product-level union — worth covering with at least one e2e case.
Not checked: build/lint/unit tests (static review by design), e2e (needs Linux+kind), the actual shape of srcLanguages in pm, and how the ISPRAS exporters split the value (see the separator comment).
| func aggregateSourceLangs(images []*ImageSBOM) []string { | ||
| var langs []string | ||
| for _, img := range images { | ||
| langs = append(langs, img.GOST.SourceLangs...) |
There was a problem hiding this comment.
The product union takes only img.GOST.SourceLangs, i.e. the languages collected from components. Languages already present at the image BOM level (img.BOM.Properties → they become the properties of the container component and are unioned in setMissingGOSTOnComponent, which the test "unions the image languages with the ones already set on the container component" pins down as a supported scenario) never get here.
Consequences: in the container format the product line is not the union of the container lines (in the test the container gets Rust, Go, the product only Go); in the oss format, where there are no container components, Rust is lost entirely. This contradicts the description's claim that "the product component carries the sorted, deduplicated union of the languages of all images".
Either account for gost.GetComponentSourceLangs of the image's BOM-level properties here (and cover it with a test for both formats), or drop the scenario from the test/description if it is intentionally unsupported.
There was a problem hiding this comment.
Fixed. aggregateSourceLangs now unions via gost.CollectBOMSourceLangs, which reads the image BOM's document properties, the root metadata component and the component tree, so the product line equals the union of the container lines in both formats and the oss format no longer loses BOM-level languages. Covered by the "includes the BOM-level languages of an image in the product union" table for both assemblers.
|
|
||
| for _, cat := range c.Catalogers { | ||
| args = append(args, "cataloger", cat.Name) | ||
| args = append(args, "cataloger", cat.Name, cat.SourceLang) |
There was a problem hiding this comment.
SourceLang is a static function of cat.Name (the mapping in config.ecosystems), and Name is already in the checksum. Per the calculateStableChecksum docstring, generator logic changes are covered by sbomArtifactFormatVersion (which is bumped here anyway), so this addition is redundant and slightly blurs the documented split between "what the checksum covers / what the format version covers". Not blocking, but either drop it or extend the docstring.
There was a problem hiding this comment.
Removed — SourceLang is out of the checksum; the regeneration is carried by the sbomArtifactFormatVersion bump alone, as the docstring describes.
|
|
||
| const PropertySourceLangs = "GOST:source_langs" | ||
|
|
||
| const sourceLangsSeparator = ", " |
There was a problem hiding this comment.
The description says the exporters read the first property and "split its value on commas". With the ", " separator, the second and subsequent tokens come out with a leading space (" Python") unless the exporter trims. If this has not been verified against the real exporter, "," is the safer choice; GetComponentSourceLangs already trims and will read both forms identically.
There was a problem hiding this comment.
Changed the separator to ",": the exporters' trimming behavior is unverified, and GetComponentSourceLangs reads both forms identically.
| } | ||
|
|
||
| filtered := lo.Filter(*bom.Components, func(comp cdx.Component, _ int) bool { | ||
| matches := func(comp *cdx.Component, f catalogerFilter) bool { |
There was a problem hiding this comment.
nit: matches takes a *cdx.Component but dereferences it in componentFoundByCataloger(*comp, …) and once more in one of the branches — two copies of a large struct per component×filter pair, whereas before the refactoring lo.Filter copied once. Simpler to pass cdx.Component by value into matches and take the pointer only for SetComponentSourceLangs.
There was a problem hiding this comment.
Gone with the rework: after #307 merged, the stamping moved back to the per-directive scan (scanCatalogerDir), and FilterBOMBySourcePaths with the matches closure no longer exists on this branch.
b285f10 to
c599e70
Compare
reyreavman
left a comment
reyreavman
left a comment
There was a problem hiding this comment.
Reviewed statically against main at head c599e70 (build/lint/tests not run at reviewer's request).
Blocking (in unchanged code, so noted here — pkg/sbom/cyclonedxutil/canonicalize.go:665 dedupProperties): GOST:source_langs can end up as a repeated property on a component after werf sbom merge, which contradicts the PR's "never repeated properties, the exporter reads only the first one" guarantee.
dedupProperties only collapses attack_surface/security_function; every other property is deduped by exact name+value. When two components with the same purl are merged (mergeComponentInto just appends the duplicate's properties) but carry different language sets — e.g. curl cataloged as ["C"] in one image and ["C","Assembly"] in another, or a user-imported BOM that already has its own GOST:source_langs — the survivor ends up with two GOST:source_langs properties, and the ISPRAS exporter reads only the first → the languages column is silently truncated.
Fix belongs in dedupProperties: add a branch for gost.PropertySourceLangs that unions the comma-separated values into a single property (same shape as the gost.Max handling for the other two GOST props). Please add a canonicalize_test.go case with two same-purl components carrying different languages, and an ispras/gost_test.go case where an image has the property both on its root component and in BOM.Properties (the existing "unions the image languages..." test only sets it on the document, so it doesn't catch the duplicate).
Everything else is minor: misleading setMissingGOSTOnComponent name, redundant tree walk in aggregateGOST, container-vs-product ordering inconsistency, hardcoded , separator, an undocumented struct field, and thin e2e coverage (pip/npm only). Please also close out the srcLanguages UNVERIFIED note against the pm source before undrafting.
| } | ||
|
|
||
| newAccessor(comp).setRawProperty(PropertySourceLangs, strings.Join(normalized, sourceLangsSeparator)) | ||
| } |
There was a problem hiding this comment.
The single-property guarantee holds only for direct writes through this function. It does not survive werf sbom merge: cyclonedxutil.canonicalize.go:665 dedupProperties doesn't know about GOST:source_langs, so when two same-purl components with different language sets are folded together (or when an image carries the property on both its root component and its document), the surviving component ends up with two GOST:source_langs properties. Since the exporter reads only the first, the union is lost. Please teach dedupProperties to union this property into one entry, and add a same-purl/different-languages test.
There was a problem hiding this comment.
Fixed. dedupProperties now has a GOST:source_langs branch that unions repeated values into the first entry via gost.MergeSourceLangsValues, alongside the gost.Max branch for the other two. Tests: canonicalize_test.go "unions GOST:source_langs of same-purl components into a single property" (C + Assembly,C → one property Assembly,C), and ispras/gost_test.go "keeps a single GOST:source_langs property when the image carries it on both the root component and the document" (root Lua + document Rust + component Go → exactly one property Go,Lua,Rust on the container). Both fail with the branch removed — checked by mutation; without it the container carried Go,Lua and Rust as two properties, exactly the case you described.
The fix lives in the aggregation commit (feat(sbom): collect package source languages on image and product level) rather than a separate fix: the bug never existed on main, so a fix subject would have put a phantom line into the changelog, and the test that pins it would have been red for three commits otherwise.
Independently of canonicalization, SetComponentSourceLangs now folds any repeated GOST:source_langs the component already carries into the one it writes, so applyGOSTToContainer leaves a single property even on a path that does not canonicalize; GetComponentSourceLangs unions repeats when reading a non-canonical (imported) component.
|
|
||
| // CollectSourceLangs unions the source languages of the given components and their | ||
| // nested ones, in order of first appearance. | ||
| func CollectSourceLangs(components []cdx.Component) []string { |
There was a problem hiding this comment.
Ordering is inconsistent with the product level: here languages keep first-appearance order, but the product component is sorted (ispras/gost.go:37 sort.Strings). In the listing one image would show Go,Python and another Python,Go for the same set. Cheapest fix is to sort in one place that both paths go through — either here in CollectSourceLangs or in SetComponentSourceLangs.
There was a problem hiding this comment.
Sorted in one place: normalizeSourceLangs now dedups and sorts, and every writer (SetComponentSourceLangs, CollectSourceLangs, CollectBOMSourceLangs, MergeSourceLangsValues, the new UnionSourceLangs) goes through it, so container and product lines spell a given set identically. The sort.Strings in aggregateSourceLangs is gone as redundant.
| } | ||
|
|
||
| return normalizeSourceLangs(strings.Split(raw, ",")) | ||
| } |
There was a problem hiding this comment.
strings.Split(raw, ",") hardcodes the separator while sourceLangsSeparator is declared and used only in the Join. Use the constant here (and at line 57) so the split and join can't drift apart.
There was a problem hiding this comment.
Done — joinSourceLangs/splitSourceLangs wrap the constant; no strings.Split(…, ",") on language values remains.
| result.AttackSurface = gost.Max(result.AttackSurface, nested.AttackSurface) | ||
| result.SecurityFunction = gost.Max(result.SecurityFunction, nested.SecurityFunction) | ||
| } | ||
| result.SourceLangs = gost.CollectSourceLangs(components) |
There was a problem hiding this comment.
aggregateGOST is recursive and calls CollectSourceLangs (itself recursive) at every level, while the nested nested.SourceLangs is discarded. The component subtree is re-walked once per depth. Either compute langs per-node inside the loop (normalize(append(GetComponentSourceLangs(&components[i]), nested.SourceLangs...))), or hoist a single CollectSourceLangs call to the caller in container.go:59.
There was a problem hiding this comment.
Done — aggregateGOST now appends GetComponentSourceLangs of the node plus nested.SourceLangs per level and normalizes once at the end; the separate CollectSourceLangs walk is gone.
| SecurityFunction: security, | ||
| }) | ||
|
|
||
| gost.SetComponentSourceLangs(comp, append(gost.GetComponentSourceLangs(comp), values.SourceLangs...)) |
There was a problem hiding this comment.
For source langs this performs a union, not a fill-if-absent, so the name setMissingGOSTOnComponent misdescribes half the behavior. Consider renaming (e.g. applyGOSTToContainer) or moving this line out to the caller next to aggregateGOST. Note also this only updates the first GOST:source_langs occurrence: if container.go:49 already concatenated the property from both the root component and the image document, the second copy survives Canonicalize — fixing dedupProperties closes that.
There was a problem hiding this comment.
Renamed to applyGOSTToContainer with a doc stating the fill-if-absent vs union split. The second-copy case is closed by the dedupProperties fix above (covered by the root+document test).
| Name string | ||
| SourcePaths []string | ||
| OptionalSourcePaths []string | ||
| SourceLang string |
There was a problem hiding this comment.
The struct doc comment above documents every field except the new SourceLang. Please add a sentence so the doc stays complete (what it is, and that it's empty for os-pm/arbitrary-language ecosystems).
There was a problem hiding this comment.
Added to the struct doc: what it is, that it is stamped as GOST:source_langs, and that it is empty for os-pm, which takes its languages from the pm catalog.
| License string `json:"license"` | ||
| OriginalRepo string `json:"originalRepo"` | ||
| Repo string `json:"repo"` | ||
| SrcLanguages []string `json:"srcLanguages"` |
There was a problem hiding this comment.
The PR marks the srcLanguages shape as UNVERIFIED against a real pm release. Because json.Unmarshal silently ignores an unknown or renamed key, a mismatch would drop the language from every os-pm row with no error and no log. Before this leaves draft, please confirm the field name and type against the pm source (string array vs string), or run the ospm_basic e2e on a rebuilt builder image with a language assertion.
There was a problem hiding this comment.
Verified against the pm source: pkg/packet/packet.go declares SrcLanguages []string \json:"srcLanguages,omitempty"`, and pm info --installed-jsonencodes the packet index as is — werf'sPmPackageInfo.SrcLanguages []string `json:"srcLanguages"`matches in name and type. The PR description now says exactly that; what remains unverified is only the *content* (no builder image ships a catalogue with populatedsrcLanguages` yet), so the os-pm e2e cannot assert a language until they are rebuilt.
|
|
||
| bom := sbomtest.MustParseSBOMOutput(sbomOut) | ||
| sbomtest.AssertSourceLangsOnComponent(bom, componentName, componentVersion, []string{expectedLang}) | ||
| }, |
There was a problem hiding this comment.
e2e covers only pip and npm, though fixtures already exist for gomod_*, cargo_simple, lua_simple, and ospm_basic. Please add at least go-mod here — it's the primary product case; os-pm can follow once the builder images are rebuilt with the pm release that writes srcLanguages.
There was a problem hiding this comment.
Added go-mod (inject/gomod_license, github.com/pkg/errors v0.9.1), rust-cargo (inject/cargo_simple, anyhow 1.0.86) and lua-rock (inject/lua_simple, werf-sbom-lua-app 0.1-1) to the table — the same fixtures and component/version pairs the ecosystem specs already assert. os-pm stays out until the builder images carry a catalogue with srcLanguages.
c99e24c to
8026e27
Compare
Every component cataloged through a packages directive now carries the GOST:source_langs property required by the FSTEC component listing: a single property with the languages of the ecosystem that installed the package, which is the form the ISPRAS tooling reads (it takes the first property with that name and splits its value on commas). The language comes from the packages directive itself, registered per ecosystem and carried down to the per-directive scan, so it does not depend on scanner metadata. Packages installed by os-pm are prebuilt binaries of an arbitrary language and stay without the property. The SBOM artifact format version is bumped so that cached SBOMs are regenerated. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
A merged SBOM now carries GOST:source_langs on the product component and, in the container format, on the container component of every image, holding the sorted union of the languages of the components below it. Without it the image and product rows of the tabular component listing, which is generated from these properties, stayed empty while the package rows were filled in. Languages already present on a component or at the image BOM level (e.g. from a user-imported BOM) are unioned with the aggregated ones instead of suppressing them. Canonicalization upholds the single-property shape across the merge: when same-purl components with different language sets are folded together, or an image carries the property on both its root component and its document, the repeated GOST:source_langs entries collapse into one union — the ISPRAS exporters read only the first property of a name, so a second one would silently truncate the listing. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Assert GOST:source_langs on a component cataloged by each ecosystem with a fixture — go-mod, python-pip, rust-cargo, javascript-npm and lua-rock — so the language registry stays wired from the packages directive down to the generated SBOM. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Describe how GOST:source_langs is derived from the packages directive, why os-pm packages carry no language, and how the languages are collected on merge. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
An os-pm component carries GOST:source_langs read from the srcLanguages the pm catalogue declares for the package, which pm writes into /var/lib/pm/index.json (Packet.SrcLanguages, a string array; pm releases from 2026-09-12 on). The languages come from the package's own description, nothing is guessed: packages without the declaration stay without the property, and the image and product level aggregation picks the languages up with no further changes. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Pass context through the new source-language APIs and require an allocated BOM, following the public API conventions. Preserve the existing contextless conversion and canonicalization APIs at their pure-data boundaries, and correct the merge-order comment. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Use disjoint language sets in both component orders so last-wins behavior cannot satisfy the union test. Assert the raw container property for a language present only in a descendant; the product assertion alone did not exercise that traversal. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
86da328 to
45b4f46
Compare
reyreavman
left a comment
reyreavman
left a comment
There was a problem hiding this comment.
Round 3, reviewed statically against main @ a0b171e at head 45b4f46 (build/lint/tests not run, as before).
All 8 findings from the previous rounds are resolved and survived the rebase: dedupProperties now unions GOST:source_langs (both component orders tested), repeats on root+document are folded, applyGOSTToContainer is honestly named, aggregateGOST walks the tree once, ordering is deterministic through the single normalizeSourceLangs writer, the separator constant is used everywhere, SourceLang is documented, and e2e covers go-mod/cargo/lua in addition to pip/npm. The 45b4f46 test commit does the mutation pass I asked for (disjoint sets in both orders so last-wins can't pass; raw container-property assertion for a descendant-only language). PR description is consistent with the code again (6 → 7, srcLanguages verified against pm/pkg/packet/packet.go).
No blockers. Two non-blocking notes inline: context.Background() is now used in library code (three occurrences, the only ones in non-test pkg/sbom), and dedupProperties keeps an empty-valued GOST:source_langs where SetComponentSourceLangs would write nothing. Both are at the author's discretion before undrafting.
Residual, not code: end-to-end propagation of srcLanguages from a real pm catalogue is still UNVERIFIED (now lower risk since the field name/type are confirmed) — an ospm_basic e2e assertion once the builder images are rebuilt would close it.
| continue | ||
| case gost.PropertySourceLangs: | ||
| if pos, exists := gostPos[prop.Name]; exists { | ||
| result[pos].Value = gost.MergeSourceLangsValues(context.Background(), result[pos].Value, prop.Value) |
There was a problem hiding this comment.
Non-blocking: context.Background() here and at line 713 (plus os_pm.go:84) are the only three occurrences in non-test pkg/sbom — this PR introduces the pattern. The ctx was threaded through the gost helpers to satisfy CODESTYLE ("public functions take ctx first"), but every helper ignores it (_ context.Context), and where there is no caller ctx to pass we now manufacture one. That's ceremony without propagation.
MergeSourceLangsValues and NormalizeSourceLangsValue are pure string functions with no I/O; inside the private dedupProperties (whose public entry Canonicalize has no ctx, pre-existing) there is nothing to propagate. Either accept that these pure helpers don't take ctx, or thread ctx into Canonicalize for real — but context.Background() in library code is the worst of both. Worth deciding before undrafting so the pattern doesn't spread.
There was a problem hiding this comment.
Addressed in 1952e50: Canonicalize, CanonicalizeDocument and MergeBOMs now receive the caller context, propagated through the component/service/vulnerability paths down to property normalization. Build and ISPRAS callers pass their existing contexts; tests use SpecContext. No Background context is manufactured in these library paths. The pure helpers still do not claim cancellation support.
| continue | ||
| } | ||
| gostPos[prop.Name] = len(result) | ||
| result = append(result, cdx.Property{Name: prop.Name, Value: gost.NormalizeSourceLangsValue(context.Background(), prop.Value)}) |
There was a problem hiding this comment.
Nit: for an imported BOM carrying GOST:source_langs: " , ", NormalizeSourceLangsValue returns "" and the component keeps a property with an empty value (source_langs_test.go:138 pins this). SetComponentSourceLangs in the same situation writes nothing — so the two writers disagree on the empty case. Harmless for the exporter (empty column = no column), but a continue here when the normalized value is empty would make them consistent.
There was a problem hiding this comment.
Addressed in f30ac15: normalize each source-language value and skip empty results before recording gostPos. Regression tests cover blank-only values, repeated blanks, empty/nonempty values in either order, adjacent properties, removal of a sole blank property, and idempotence. Four assertions failed before the fix and pass afterward. The setter remains a no-op for an empty input; canonicalization removes existing empty properties.
|
|
||
| comp.Hashes = digestToHashes(pkg.Digest) | ||
| comp.Properties = packageProperties(pkg, containerFactoryVersion) | ||
| gost.SetComponentSourceLangs(context.Background(), &comp, pkg.SrcLanguages) |
There was a problem hiding this comment.
Non-blocking, same point as in canonicalize.go: ConvertToCycloneDX is public and per CODESTYLE should take ctx first anyway (pre-existing gap); its only caller collect.go:57 already has a ctx. Threading it through would remove this context.Background() for real instead of papering over it.
There was a problem hiding this comment.
Addressed in 1952e50: ConvertToCycloneDX receives ctx from CollectBOM and passes it to the language setter. Also closed the real-catalogue verification gap in 3e36a03: a new digest-pinned fixture installs curl and jq into scratch using pm 0.1.8 and the actual v3.1.5 catalogue. The e2e independently checks the installed pm index with networking disabled, then the raw SBOM properties C,Perl and C,YAML. It passed on Linux; suppressing language propagation fails the SBOM assertion, and the restored 13-spec GOST/merge suite passes. No synthetic index is used.
Pass the existing build and assembly context through merging, canonicalization, and pm conversion instead of constructing background contexts at legacy API boundaries. Keep pure transformations unchanged and use spec contexts in their tests. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Drop blank imported language values before recording their deduplication position. Preserve nonempty unions and neighboring properties in either order, and cover empty-only input and repeated canonicalization. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Install curl and jq into scratch with a digest-pinned pm carrier and published catalogue. Check the installed index independently with networking disabled, then require one exact language property per SBOM component; suppressing pm language propagation must fail the test. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
reyreavman
left a comment
reyreavman
left a comment
There was a problem hiding this comment.
Round 4, reviewed statically at head 3e36a03 against main @ a0b171e (build/lint/tests not run, as before).
All round-3 notes are closed properly rather than cosmetically:
1952e50threads the callerctxthroughCanonicalize/CanonicalizeDocument/MergeBOMs/ConvertToCycloneDXand the privatecanonicalize*/merge*Into/dedupPropertieschain.context.Background()in non-testpkg/sbomandpkg/build: zero occurrences. Callers of the four changed public signatures withoutctxacrosscmd/,pkg/,test/pkg/: zero.f30ac15makesdedupPropertiesnormalize first and skip an emptyGOST:source_langs, so it now agrees withSetComponentSourceLangson the empty case. The 6-entry table (empty before/after/between languages, repeated empties) plus the sole-empty →Properties == nilcase and the double-Canonicalizeidempotency check cover it.3e36a03closes the remaining UNVERIFIED item with a real e2e: builds on a builder shipping pm ≥ 0.1.7, readspm info --installed --jsonfrom the very image viadocker run --network=none, and asserts both the pm output and the SBOM againstcurl: C,Perl/jq: C,YAMLwith a raw single-property check. Helper signatures (report.NewProjectWithReport().BuildWithReport,utils.RunCommandWithSeparateStreams,SuiteData.GetBuildReportPath) exist and match the pattern already used infinal_repo_test.go/vex_coexistence_test.go.
PR description is updated to match (merge-time collapse, empty-value omission, pm ≥ v0.1.7 requirement, 6 → 7).
No blockers. One minor inline (docs don't mention the pm ≥ v0.1.7 threshold the description now states) and two nits on the e2e fixture/assert design. Ready to undraft from my side once the docs sentence lands.
Not verified: compilation of the signature sweep in 1952e50 (3 packages + 4 test files) — only the grep invariant above; the exact pm info --installed --json CLI form and the hardcoded language sets for PACKAGES_VERSION: ebf1512… were not checked against pm itself.
|
|
||
| The `GOST:source_langs` property is filled in automatically and needs no configuration: every component cataloged through a `packages` directive gets the source language of that directive's ecosystem (`go-mod` — `Go`, `python-pip`/`python-poetry`/`python-uv` — `Python`, `rust-cargo` — `Rust`, `javascript-npm`/`javascript-yarn`/`javascript-pnpm` — `JavaScript`, `lua-rock` — `Lua`). | ||
|
|
||
| Packages installed by `os-pm` are prebuilt binaries, so their languages cannot be derived from the directive: they carry the languages declared for the package in the pm catalogue (the `srcLanguages` field), and packages without that declaration carry no property. When SBOMs are merged with `werf sbom merge`, the languages of all images are collected on the product component, and in the `container` format the languages of an image's components are additionally collected on that image's container component. |
There was a problem hiding this comment.
Minor: the PR description now states that srcLanguages is only present in indexes written by pm v0.1.7+, and that older installed indexes yield os-pm components without the property. The docs say "packages without that declaration carry no property" but don't name the version threshold — a user on an older container-factory base will see an empty languages column and have no way to learn why from here. One sentence here (and the mirror in pages_ru) would close that: e.g. "The field is written by pm v0.1.7 and newer; images built on an older pm carry no GOST:source_langs for os-pm packages."
There was a problem hiding this comment.
Addressed in 6157a88, mirrored in Russian: source-language persistence requires pm v0.1.7+ and a catalogue declaring srcLanguages. The documentation also explains that replacing the pm binary alone does not populate existing installed-index entries; packages must be freshly installed with the compatible pm/catalogue when rebuilding the image.
|
|
||
| Свойство `GOST:source_langs` заполняется автоматически и не требует настройки: каждый компонент, найденный по директиве `packages`, получает язык исходного кода экосистемы этой директивы (`go-mod` — `Go`, `python-pip`/`python-poetry`/`python-uv` — `Python`, `rust-cargo` — `Rust`, `javascript-npm`/`javascript-yarn`/`javascript-pnpm` — `JavaScript`, `lua-rock` — `Lua`). | ||
|
|
||
| Пакеты, устанавливаемые через `os-pm`, представляют собой собранные бинарные файлы, поэтому их язык не выводится из директивы: они получают языки, заявленные для пакета в каталоге pm (поле `srcLanguages`), а пакеты без такого описания остаются без свойства. При объединении SBOM командой `werf sbom merge` языки всех образов собираются на компоненте продукта, а в формате `container` языки компонентов образа дополнительно собираются на container-компоненте этого образа. |
There was a problem hiding this comment.
Minor: same as the pages_en comment — please add the pm ≥ v0.1.7 threshold here too, so both language versions stay in sync.
There was a problem hiding this comment.
Addressed in 6157a88 with the same version, catalogue, and existing-index guidance as the English page.
| bom := sbomtest.MustParseSBOMOutput(werfProject.SbomGet(ctx, &werf.SbomGetOptions{ | ||
| CommonOptions: werf.CommonOptions{ExtraArgs: []string{"app"}}, | ||
| })) | ||
| for _, expected := range []struct { |
There was a problem hiding this comment.
Nit, not blocking: the expected languages are hardcoded and cross-checked against pm info. That is deliberate and good for diagnosis (a failure on the pm assertion means the catalogue moved, on the SBOM assertion means werf broke), but every PACKAGES_VERSION bump now also requires editing this list. An alternative is to assert the SBOM carries exactly sort(pkg.SrcLanguages) from the pm output and keep the hardcoded set only as a non-empty sanity check. Your call.
There was a problem hiding this comment.
Keeping the independent exact expectations deliberately. PACKAGES_VERSION is an immutable catalogue digest, not a moving release tag, and the carrier is digest-pinned too. Updating this fixture is a reviewed change that should recheck its declared language sets. Both assertions use known catalogue expectations rather than deriving the SBOM oracle from the pm output; the Linux mutation run demonstrated that lost language propagation fails the SBOM assertion after the installed-index assertion succeeds.
| standard: cyclonedx@1.6 | ||
| --- | ||
| image: carrier | ||
| from: registry.deckhouse.io/container-factory@sha256:002c52a107a3583cd9d7745018a6ba66df2c89baccaa46fd8e3feaec2ee743a2 |
There was a problem hiding this comment.
Nit: this fixture pins container-factory@sha256:002c52a… while ospm_basic pins 7aac8d9…. Intentional (the older builder predates srcLanguages), but it is fixture drift: two os-pm e2e suites now exercise two different builders. Once ospm_basic moves to a pm ≥ 0.1.7 builder it is worth collapsing both to one digest (and ideally one PACKAGES_VERSION).
There was a problem hiding this comment.
Keeping the two fixtures separate in this PR. ospm_basic intentionally continues exercising its existing catalogue and metadata expectations (package hashes and dependency graph included); switching it to v3.1.5 would be a broader fixture migration. The new language fixture imports only pm/certificates into scratch and pins the real v3.1.5 catalogue independently. Consolidation can be considered when ospm_basic and its dependent assertions are migrated together.
Document the pm v0.1.7 minimum and the need for a catalogue that declares source languages in both translations. Explain why updating the binary alone cannot populate an existing installed-package index. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
🤖 I have created a release *beep* *boop* --- ## [3.6.1-dk.2](v3.6.1-dk.1...v3.6.1-dk.2) (2026-10-01) ### Features * **sbom:** add --warnings-non-fatal flag to sbom validate ([#331](#331)) ([91e1a30](91e1a30)) * **sbom:** carry STREEBOG source-distribution digests through the CycloneDX 1.6 SBOM ([#383](#383)) ([1065a95](1065a95)) * **sbom:** generate file-based package SBOMs without docker.sock ([#307](#307)) ([6681160](6681160)) * **sbom:** mark only entry-point packages as directly attackable ([#332](#332)) ([a0b171e](a0b171e)) * **sbom:** report the source language of cataloged packages ([#328](#328)) ([df4ba44](df4ba44)) ### Bug Fixes * **ci:** honor cancellation of build and test workflows ([#381](#381)) ([63f7581](63f7581)) * **sbom, vex, build:** keep artifacts with the image in every repository ([#282](#282)) ([5544c87](5544c87)) * **sbom:** restore the build after the file-based SBOM scan merge ([#380](#380)) ([4d67f69](4d67f69)) * **sbom:** stop ispras validation failing on containers without a description ([#382](#382)) ([c3c9abd](c3c9abd)) * **sbom:** stop printing every validation finding twice in sbom validate ([#350](#350)) ([6f0af68](6f0af68)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
1 participant
Summary
SBOMs of stapel images now carry the
GOST:source_langsproperty the FSTEC tabular component listing is generated from: every component cataloged through apackagesdirective is stamped with the source language of that directive's ecosystem, os-pm components get the languages declared in the pm package catalogue, andwerf sbom mergerolls the languages up to the image and product level. Nothing to configure; the property appears on rebuild.What
Per-package property
packagesdirective carriesGOST:source_langs:go-mod→Go,python-pip/python-poetry/python-uv→Python,rust-cargo→Rust,javascript-npm/javascript-yarn/javascript-pnpm→JavaScript,lua-rock→Lua.","(no space, in case the exporters do not trim the tokens they split) — never repeated properties, because the ISPRAS exporters read only the first property with a given name and split its value on commas. Canonicalization upholds this acrosswerf sbom mergetoo: when two same-purl components with different language sets are folded together, or an imported BOM carries the property on both its root component and its document, the repeatedGOST:source_langsentries collapse into one union (the same wayattack_surface/security_functioncollapse to their strongest value).os-pmcomponent carries the languages declared for the package in the pm catalogue (thesrcLanguagesfield of/var/lib/pm/index.json); a package without the declaration carries no property — the language of a prebuilt binary is never guessed.srcLanguages: pm v0.1.7 and newer preserve this field from the catalogue. Existing installed indexes without it still produce components without the language property; no languages are inferred for them.Merge-level aggregation
werf sbom merge, the product component carries the sorted, deduplicated union of the languages of all images.containeroutput format, each image's container component additionally carries the sorted union of the languages of the components below it (the same order as the product line, so the listing shows one spelling for one set); theossformat has no container components and gets only the product-level property.GOST:source_langsvalues containing only whitespace or comma separators; empty duplicates do not suppress nonempty languages.Cache
Why
The "programming language(s)" column of the tabular component listing (ППК) is produced by the ISPRAS exporters from
GOST:source_langs; without the property the column stays empty for every package, image and product row. The language comes from thewerf.yamlpackagesdirective, not from syft package metadata: stapel images are scanned one directive at a time, so the attribution is exact and does not depend on what syft happens to emit. Deriving the language from purl types (the efs-sbom approach) was rejected as the primary source — it is a heuristic, while the directive type and the pm catalogue are ground truth someone declared.