Skip to content

Add tests for the SSMS installer's VSIX signature check - #624

Merged
erikdarlingdata merged 2 commits into
devfrom
test/618-vsix-check-tests
Sep 29, 2026
Merged

erikdarlingdata merged 2 commits into
devfrom
test/618-vsix-check-tests

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 29, 2026 •

Copy link
Copy Markdown
Owner

Closes #618

What this adds

A signed SSMS installer accepts a VSIX only when three things hold. The VSIX has one valid signature. The signature was made with the installer's own certificate. The signature covers every entry in the ZIP file.

Until now the only end-to-end check was the --verify-only step in release.yml. That step sees one real signature that passes. This PR adds tests that build VSIX files on purpose and check the reason CheckVsix gives for each one.

No installer logic changed. The only edit to the installer project is an InternalsVisibleTo item, so the tests can call the internal method CheckVsix. FindUncovered is private, so the tests reach it through CheckVsix.

What each test covers

There are 77 tests in tests/PlanViewer.Ssms.Installer.Tests. Each one asserts the exact reason string, or null when the file is accepted. Many of them also show that the signature still verifies, so the entry rule is what refuses the file. Several show that the same file passes before the change that breaks it.

  • Control (VsixSignatureTests): a VSIX signed with the installer's certificate that covers every part is accepted. The certificate can sit in the signature part or in its own part. The test first checks that every part really is signed.
  • Unsigned, other certificate, two signatures (VsixSignatureTests): the reasons are "not signed", "signed with a different certificate" and "more than one signature". Two signatures are refused for two different certificates, and for the installer's certificate twice. A signature with no certificate gives "not valid (CertificateRequired)". A certificate with the right thumbprint and other bytes is refused, so the whole certificate is compared. A file signed by another certificate and also damaged gives "different certificate". That shows the compare runs before the verification.
  • Part changed after signing (VsixSignatureTests): a changed part, a removed part, and a changed content type in [Content_Types].xml. Also a changed signature value, and a certificate part swapped for another certificate. Also a file that is not a ZIP, a missing file, and a ZIP that is not a package.
  • Extra entries (VsixEntryTests): an extra part with a content type. An extra entry that OPC does not list as a part. A part left out of the signature. A signature that signs only one part, where the reason names three entries and counts the rest. A directory entry. A name with .., a backslash, a control character or a very long name.
  • Names that differ only in case (VsixCaseTests): two non-part entries in different case, and a second [Content_Types].xml in another case. A signed part or [Content_Types].xml renamed to another case. The same rename for relationship parts, the origin part, the signature part and the certificate part. OPC refuses two files before the coverage rule runs. Each has a second entry with the name of a signed part, in the same case or another. They give "failed to read the file".
  • Relationship parts (VsixRelationshipTests): an unsigned relationship part is refused. A relationship part signed whole is accepted. So is a relationship picked by an Id or Type selector. A relationship added after signing is refused when selectors cover the rest. It breaks the signature when the part was signed whole or selected by type. External relationships, certificate-type relationships on ordinary parts, and a relationship part for a part that does not exist are refused.
  • Origin relationship exception (VsixRelationshipTests): the origin relationship of the package is accepted with no signature over it. The test shows that the package relationship part really is unsigned. The exception ends where the rules end. It refuses a second origin relationship that points elsewhere, and an origin relationship on an ordinary part. It also refuses another relationship of a signature type, and any other package relationship that nothing covers. A second origin relationship to the origin part is refused by OPC.
  • Origin and certificate parts (VsixOriginAndCertificateTests): an empty unsigned origin part is accepted. So is a signed origin part with content. An unsigned one with content is refused. Relationships of the origin part and the signature part are accepted when they point to a signed part. They are refused when they point to an unsigned entry, a missing entry or outside the package. A certificate part with exactly the installer's certificate is accepted. Another certificate, one byte added, no link from the signature part, and the wrong content type are refused.

How the fixtures and certificates are made

Every VSIX is built at test time with System.IO.Packaging. The parts look like a real one: extension.vsixmanifest, catalog.json, an assembly stand-in, LICENSE.txt, and a part with mixed case and an escape in its name. The signing uses PackageDigitalSignatureManager, the signing API of the same library. Where a test needs something OPC will not write, it edits the ZIP entries directly. Each test uses its own temp folder and deletes it.

The two certificates come from CertificateRequest.CreateSelfSigned and live in memory only. No certificate, key or binary file is committed. Nothing touches a Windows certificate store. The PFX round trip in the brief was not needed. These certificates sign as they are on .NET Framework 4.8, on this machine and on the runtime of windows-latest.

The tests hand CheckVsix the installer certificate as the installer holds it: a plain X509Certificate with no private key.

The workflow

.github/workflows/ssms-installer-tests.yml runs on windows-latest. It runs on pull requests to dev and main that touch src/PlanViewer.Ssms.Installer/**, tests/PlanViewer.Ssms.Installer.Tests/**, Directory.Packages.props or the workflow file. It also has workflow_dispatch. It has contents: read and the same pinned actions as release.yml.

It builds and runs only the new project. It uses the same dotnet test and hang dump options as ci.yml. It is not a required check. The project is not in PlanViewer.sln, because ci.yml builds that on ubuntu-latest and cannot run net472 tests.

The project uses xunit.v3 on Microsoft Testing Platform, with the same 15 minute session timeout as PlanViewer.Core.Tests. Every version comes from Directory.Packages.props, so I added no package. Test parallelism is off. The tests are quick and share the two certificates.

Local results

  • New tests: 77 of 77 pass, run with the exact command in the workflow.
  • dotnet build src/PlanViewer.Ssms.Installer/PlanViewer.Ssms.Installer.csproj -c Release: 0 warnings, 0 errors.
  • dotnet build PlanViewer.sln -c Release: 0 warnings, 0 errors.
  • To check that the tests can fail, I made 17 temporary changes to the check logic, one at a time, and reran the suite. Examples: comparing names without case, dropping the origin exception, comparing thumbprints only, and never reporting uncovered entries. Each change failed at least one test. I restored Program.cs afterwards, and none of it is in this PR.

Defects

I found no defect, so no test is skipped. Every case gave the reason that the rules in #612 describe.

Two things I noticed and did not pin with a test:

  • An unsigned relationship part with no relationships in it is accepted, because it has nothing to cover. Its bytes are not inspected, like [Content_Types].xml. Check the VSIX signature before a signed SSMS installer installs it #612 already lists that limit.
  • Some reasons end with a message from .NET, such as "Format error in package". Windows shows that message in its own language, so the tests match only the start of those reasons.

Dependabot

.github/dependabot.yml now lists /tests/PlanViewer.Ssms.Installer.Tests. The file names every project directory, because a scan of PlanViewer.sln skips the projects that are not in it. .github/workflows/release.yml is not touched.

Generated with Claude Code

https://claude.ai/code/session_019tS6P95Dtzs4aeMpb1xqzX

erikdarlingdata and others added 2 commits September 29, 2026 15:39
CheckVsix decides whether a signed installer accepts a VSIX, and the only
end-to-end check so far is the --verify-only run in release.yml, which sees
one real signature that passes. This adds a net472 test project that builds
each VSIX at test time with System.IO.Packaging, signs it with certificates
made in memory, and asserts the reason CheckVsix returns.

- tests/PlanViewer.Ssms.Installer.Tests: 77 tests in five classes (signature,
  entries, case-only names, relationship parts, origin and certificate parts).
- InternalsVisibleTo on the installer project, so the tests can call CheckVsix.
- .github/workflows/ssms-installer-tests.yml runs the project on windows-latest.
  The project is not in PlanViewer.sln: ci.yml builds that on ubuntu-latest.

No installer logic changed.

Closes #618

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tS6P95Dtzs4aeMpb1xqzX
dependabot.yml names every project directory, because PlanViewer.sln
leaves some out and a solution-level scan would skip them. The new
net472 test project is not in the solution either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tS6P95Dtzs4aeMpb1xqzX
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 29, 2026 19:46
@claude

claude Bot commented Sep 29, 2026

Copy link
Copy Markdown

Reviewed the diff. This PR only adds a net472 test project for the installer's signature check, a Windows-only workflow, an InternalsVisibleTo entry and a dependabot entry. It changes no production logic.

  • Package versions: xunit.v3, Microsoft.NET.Test.Sdk and HangDump are all pinned in Directory.Packages.props.
  • Certificates: the test certificates are generated in memory, so no keys are committed.
  • Workflow: actions are pinned by SHA and permissions are contents: read.
  • Version bump / T-SQL / Web csproj: none of the version-bump, T-SQL or Web-csproj conventions apply here.

Two things I didn't check:

  • I didn't build or run the tests, and this checkout has no Windows runner. The new workflow is the only place they run.
  • tests/PlanViewer.Ssms.Installer.Tests sits outside PlanViewer.sln, so ci.yml never builds it. A break in the installer that affects the tests will only show in the new path-filtered workflow. That is a reasonable trade-off, since the workflow is not a required check.

No blocking findings.

@erikdarlingdata
erikdarlingdata merged commit 41b0cd3 into dev Sep 29, 2026
7 checks passed
@erikdarlingdata
erikdarlingdata deleted the test/618-vsix-check-tests branch September 29, 2026 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant