Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 78 additions & 7 deletions README-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,38 @@ SiriusData/
└── MasterMemory/
```

## 歌曲浏览器 JSON 契约

`src/Sirius.Protocol/Models/InternalApi/PlatformSongExplorerApiContracts.cs` 定义平台
歌曲浏览器的查询、列表、谱面详情、分布、错误和本地历史恢复结果 DTO。它们使用 Web JSON
的 camelCase 命名策略,不实现 `IDataObject`,不改变 MessagePack Union 编号或 MasterMemory 布局。

`PlatformSongExplorerRebuildResult` 与现有受控 `rebuild-local` 返回一致,字段为
`dryRun`、`scanned`、`eligible`、`ineligible`、`skipped`、`nextAfterResultId`、`hasMore`
和 `warnings`。`skipped` 是原因 key 到整数计数的对象,不是总计数或来源枚举。
`nextAfterResultId` 是十进制字符串(尚无结果时可以为 `"0"`),保留 64 位游标 ID 精度。
消费方应使用稳定 key,保留 nullable 与真实零值,并在目录、schema 和成绩版本之外,
保留列表/详情的 `statisticsVersion`、`resourceVersion`、`counterRulesVersion`。

`PlatformSongSummary.CoverUrl` 是 CDN 主封面地址。可选的 `CoverFallbackUrl`
(Web JSON 字段名 `coverFallbackUrl`)提供固定到同一资源版本的 GitHub 地址,供主封面
加载失败时回退使用。省略或传入 null 均有效;旧响应反序列化后该字段为 null,原有位置
构造函数签名保持不变。

运行 `dotnet test tests/Sirius.Protocol.Tests -c Release` 验证 JSON 边界,包括新增回退字段
之前的旧响应兼容性。

2026-10-04 的歌曲浏览器验证中,源码项目与本地 NuGet 包的 11 项契约测试均已通过。
Release MasterMemory 工具对两份真实发布数据均返回 `BYTE-EXACT ROUNDTRIP: PASS`:
两份均为 229 表,分别含 84,822 与 84,877 行。输入/输出的哈希与长度一致,原始输入
保持不变。这验证了解码与无编辑重打包,不代表已验证编辑表后的重建。复跑时将
`SIRIUS_TEST_MASTER_DB` 设置为获准使用的样本路径,在仓库根目录执行:

```powershell
dotnet run --project ./src/Sirius.Protocol/tools/Wds.MasterMemory.Tool -c Release -- `
roundtrip "$env:SIRIUS_TEST_MASTER_DB" ./artifacts/mastermemory-roundtrip
```

## 构建

### 环境要求
Expand Down Expand Up @@ -362,6 +394,28 @@ Sirius.MasterData
- 开发机优先引用同级最新源码;
- CI / 独立构建环境使用预编译 Package。

### 歌曲浏览器本地联调包

Dashboard 和 Server 当前通过 NuGet 使用滚动包。本地验证尚未发布的歌曲浏览器契约时,
保持这一包引用边界,在本仓库为两个库指定相同且唯一的预发布版本:

```powershell
$packageVersion = '1.0.3-song-explorer.local.1' # 示例;内容变化时换用新版本。
$packageDirectory = Join-Path (Get-Location).Path 'artifacts/packages'
dotnet pack ./src/Sirius.Protocol/Sirius.Protocol.csproj -c Release `
"-p:PackageVersion=$packageVersion" -o $packageDirectory
dotnet pack ./src/Sirius.MasterData/Sirius.MasterData.csproj -c Release `
"-p:PackageVersion=$packageVersion" -o $packageDirectory
```

消费方 restore 时设置 `UseSiriusDataRollingPackages=false`,将计算得到的
`$packageDirectory` 加入 `RestoreAdditionalProjectSources`,并将现有
`PackageReference` 固定到 `$packageVersion`。Dashboard 可使用 MSBuild 属性
`SiriusProtocolPackageVersion`;如果消费项目直接写了 `*-*`,还需显式覆盖该引用项的版本。
联调使用隔离的 `RestorePackagesPath`,不要改回兄弟源码 `ProjectReference`。
以上命令只生成本地产物,不发布包或 release。契约再次变化时应换用新的预发布版本,
避免 NuGet 缓存复用旧包。

## MasterMemory 使用

`mastermemory.db` 是 MasterMemory 数据库,不能把整个文件当成普通 MessagePack 根对象直接反序列化。
Expand Down Expand Up @@ -494,13 +548,30 @@ dotnet run --project .\src\Sirius.Protocol\tools\Wds.MasterMemory.Tool -- `
## CI

GitHub Actions 会在 push 到 `main`、针对 `main` 的 Pull Request 以及 `v*`
标签上运行。流程使用 .NET 10 SDK 完成 Restore、Release Build 和测试,然后
为 `net10.0` 打包 `Sirius.Protocol`。

每次运行都会发布可下载的 artifact,其中包含带版本号的 NuGet 包(`.nupkg`
和 `.snupkg`)以及 Release 二进制文件。普通 CI 使用
`1.0.0-ci.<运行编号>` 版本;例如 `v1.2.3` 标签会生成 `1.2.3`。artifact
由 GitHub Actions 保留 30 天。
标签上运行。流程使用 .NET 10 SDK 完成 Restore、Release Build 并调用 solution 级测试,
然后为 `net10.0` 打包 `Sirius.Protocol` 与 `Sirius.MasterData`。

`tests/Sirius.Protocol.Tests` 未加入 `SiriusData.sln`,因此当前 CI 的 solution 级测试
步骤不会执行这些契约测试。合并契约变更前应显式运行
`dotnet test tests/Sirius.Protocol.Tests -c Release`。

每次运行都会上传可下载的 artifact,包含两个 NuGet 包(`.nupkg` 和 `.snupkg`)、
Release 二进制 ZIP 和 `SHA256SUMS.txt`。普通 CI 包版本为 `<Version>-ci.<运行编号>`,
其中 `<Version>` 读取自 `Sirius.Protocol.csproj`;例如 `v1.2.3` 标签会生成 `1.2.3`。
artifact 中的包文件使用固定名称,由 GitHub Actions 保留 30 天。

### 歌曲浏览器发布顺序

1. 审核并将 Data 契约 PR 合并到 `main`。PR 构建会提供验证产物,但不会更新滚动发布。
2. 等待该次 `main` push 的 `build` 与 `rolling-release` 任务成功。workflow 会更新
`latest` release 中的 `Sirius.Protocol.nupkg` 与 `Sirius.MasterData.nupkg`。
确认 release 记录的提交对应已审核合并,且两个包使用相同的生成版本。
3. Dashboard 与 Server 保持现有滚动 NuGet 接入,重新执行 restore/build 检查后再
完成各自 PR。其 `Directory.Build.targets` 从 `releases/download/latest/` 下载两个包
并校验版本一致,无须添加兄弟源码引用或修改消费方包源。本地预发布包仅用于验证,
不在审核前发布。

仅 `v*` 标签构建会推送到 GitHub Packages;滚动消费流程使用 GitHub Release 附件。

其他仓库也可以直接复用这个 workflow:

Expand Down
102 changes: 95 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,44 @@ SiriusData/
└── MasterMemory/
```

## Song Explorer JSON contracts

`src/Sirius.Protocol/Models/InternalApi/PlatformSongExplorerApiContracts.cs` contains
the platform Song Explorer query, page, chart detail, distribution, error, and local
history rebuild result DTOs. These use the web JSON camelCase naming policy and do
not implement `IDataObject` or change MessagePack union IDs or MasterMemory layouts.

`PlatformSongExplorerRebuildResult` matches the existing controlled `rebuild-local`
response: `dryRun`, `scanned`, `eligible`, `ineligible`, `skipped`,
`nextAfterResultId`, `hasMore`, and `warnings`. `skipped` is an object mapping reason
keys to integer counts, not a total count or a source enum. `nextAfterResultId` is a
decimal string (including `"0"` before any result), preserving 64-bit cursor IDs.
Consumers should use stable keys, preserve nullable values and genuine zeroes, and
retain page/detail `statisticsVersion`, `resourceVersion`, and `counterRulesVersion`
alongside the catalog, schema, and score versions.

`PlatformSongSummary.CoverUrl` is the primary CDN image URL. The optional
`CoverFallbackUrl` (`coverFallbackUrl` in web JSON) supplies a GitHub URL pinned to
the same resource revision for use if the primary image fails. An omitted or null
fallback remains valid; older payloads deserialize with a null fallback, and the
existing positional constructor is unchanged.

Run `dotnet test tests/Sirius.Protocol.Tests -c Release` to verify the JSON boundary,
including responses from before the fallback field was added.

The 2026-10-04 Song Explorer validation passed all 11 contract tests, both against
the source project and against the local NuGet packages. The Release MasterMemory
tool also returned `BYTE-EXACT ROUNDTRIP: PASS` for two real publications: 229
tables with 84,822 and 84,877 rows respectively. Input/output hashes and lengths
matched, and both inputs remained unchanged. This verifies decoding and no-edit
repacking; it does not test edited-table rebuilds. To repeat with an authorized
sample, set `SIRIUS_TEST_MASTER_DB` to its path and run from the repository root:

```powershell
dotnet run --project ./src/Sirius.Protocol/tools/Wds.MasterMemory.Tool -c Release -- `
roundtrip "$env:SIRIUS_TEST_MASTER_DB" ./artifacts/mastermemory-roundtrip
```

## Build

### Requirements
Expand Down Expand Up @@ -345,6 +383,31 @@ that source and reference the matching version with `PackageReference`.

Consumer repositories may use a conditional `ProjectReference` / `PackageReference` strategy so developer environments use live source while isolated build environments use the packed artifact.

### Local Song Explorer integration package

Dashboard and Server's current rolling-package integration uses NuGet. Keep that
boundary when testing unreleased Song Explorer contracts locally. From this
repository, pack both libraries with the same unique prerelease version:

```powershell
$packageVersion = '1.0.3-song-explorer.local.1' # Example; change for new contents.
$packageDirectory = Join-Path (Get-Location).Path 'artifacts/packages'
dotnet pack ./src/Sirius.Protocol/Sirius.Protocol.csproj -c Release `
"-p:PackageVersion=$packageVersion" -o $packageDirectory
dotnet pack ./src/Sirius.MasterData/Sirius.MasterData.csproj -c Release `
"-p:PackageVersion=$packageVersion" -o $packageDirectory
```

For consumer restores, set `UseSiriusDataRollingPackages=false`, add the computed
`$packageDirectory` to `RestoreAdditionalProjectSources`, and pin the existing
`PackageReference` to `$packageVersion`. Dashboard supports the
`SiriusProtocolPackageVersion` MSBuild property; a consumer with a literal `*-*`
reference needs an explicit version override for that item as well. Use an isolated
`RestorePackagesPath` for these test packages. Do not replace consumer references
with sibling source `ProjectReference`s. These commands create local artifacts;
they do not publish a package or release. Use a new prerelease version after any
further contract changes to avoid stale NuGet caches.

## MasterMemory Usage

`mastermemory.db` is a MasterMemory database and should not be treated as a plain MessagePack root object.
Expand Down Expand Up @@ -473,13 +536,38 @@ A successful byte-exact no-op round trip is a strong signal that the model still
## CI

GitHub Actions runs on pushes and pull requests targeting `main`, and on `v*`
tags. It restores dependencies, builds and tests the solution with the .NET 10
SDK, then packs `Sirius.Protocol` for `net10.0`.

Each run publishes a downloadable artifact containing the versioned NuGet
package (`.nupkg` and `.snupkg`) and the Release binaries. CI builds use a
`1.0.0-ci.<run number>` version; a tag such as `v1.2.3` produces version
`1.2.3`. Artifacts are retained by GitHub Actions for 30 days.
tags. It restores dependencies, builds the solution and invokes solution-level
tests with the .NET 10 SDK, then packs `Sirius.Protocol` and `Sirius.MasterData`
for `net10.0`.

`tests/Sirius.Protocol.Tests` is not included in `SiriusData.sln`, so the current
solution-level CI test step does not run these contract tests. Run the explicit
`dotnet test tests/Sirius.Protocol.Tests -c Release` command before merging
contract changes.

Each run uploads a downloadable artifact containing both NuGet packages (`.nupkg`
and `.snupkg`), Release binary ZIPs, and `SHA256SUMS.txt`. CI package versions use
`<Version>-ci.<run number>`, with `<Version>` read from `Sirius.Protocol.csproj`;
a tag such as `v1.2.3` produces version `1.2.3`. Artifact files have stable package
names and are retained by GitHub Actions for 30 days.

### Song Explorer rollout order

1. Review and merge the Data contract PR into `main`. PR builds provide validation
artifacts but do not update the rolling release.
2. Wait for that `main` push's `build` and `rolling-release` jobs to succeed. The
workflow updates the `latest` release with `Sirius.Protocol.nupkg` and
`Sirius.MasterData.nupkg`. Check that its recorded commit is the reviewed merge
and that both packages have the same generated version.
3. Rerun Dashboard and Server restore/build checks using their existing rolling
NuGet integration, then complete their PRs. Their `Directory.Build.targets`
downloads both packages from `releases/download/latest/` and checks matching
versions. No sibling source reference or consumer package-source change is
required. The local prerelease package is for validation only and is not
published ahead of review.

Only `v*` tag builds push packages to GitHub Packages; the rolling consumer flow
uses the GitHub Release assets instead.

The workflow is also reusable from another repository:

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
using System.ComponentModel.DataAnnotations;

namespace Sirius.Protocol.InternalApi;

// JSON-only platform contracts. These types deliberately do not implement IDataObject.
public sealed record PlatformSongExplorerQuery
{
// Empty search is valid; MVC must not turn it into null before validation.
[DisplayFormat(ConvertEmptyStringToNull = false)]
public string Search { get; init; } = "";
public int Difficulty { get; init; } = 1;
public string ChartKind { get; init; } = "all";
public string CoverType { get; init; } = "all";
public string Length { get; init; } = "all";
public string Sort { get; init; } = "title-asc";
public string? Cursor { get; init; }
public int Limit { get; init; } = 18;
}

public sealed record PlatformSongSourceCoverage(
string Local, string Official, string Counts, string[] Warnings);
public sealed record PlatformSongChartPersonal(
string RecordStatus, double? Achievement, double? MetricValue,
long? TimesCompleted, int? HighestClearLamp, PlatformSongSourceCoverage SourceCoverage);
public sealed record PlatformSongNoteCount(string Kind, string Label, long Count);
public sealed record PlatformSongNoteStatistics(long ComboCount, PlatformSongNoteCount[] NoteCounts);
public sealed record PlatformSongChartSummary(
string Key, string Kind, string Id, string MusicId, int Difficulty, string Label,
string Level, string? SpecialType, string MetricKind,
PlatformSongNoteStatistics? NoteStatistics, PlatformSongChartPersonal Personal);
public sealed record PlatformSongSummary(
string Id, string Title, string Pronunciation, string Composer, string Lyricist,
string Arranger, string Singer, string? CoverUrl, string CoverType, bool IsLongVersion,
int DurationSeconds, string? ReleasedAt, long KnownTimesCompletedSum, string CountCoverage,
PlatformSongChartSummary[] Charts, string[] MatchedChartKeys)
{
// Optional pinned GitHub fallback for the primary CDN CoverUrl.
public string? CoverFallbackUrl { get; init; }
}
public sealed record PlatformSongExplorerPage(
PlatformSongSummary[] Items, string? NextCursor, int TotalCount,
string SchemaVersion, string CatalogVersion, string ScoreVersion,
PlatformSongSourceCoverage SourceCoverage)
{
public string StatisticsVersion { get; init; } = "0";
public string ResourceVersion { get; init; } = "";
public string CounterRulesVersion { get; init; } = "";
}
public sealed record PlatformSongChartDetail(
PlatformSongSummary Song, PlatformSongChartSummary Chart,
string StatisticsStatus, string? StatisticsSourceUrl, string? StatisticsContentHash,
string? StatisticsErrorCode, string SchemaVersion, string CatalogVersion, string ScoreVersion)
{
public string StatisticsVersion { get; init; } = "0";
public string ResourceVersion { get; init; } = "";
public string CounterRulesVersion { get; init; } = "";
}
public sealed record PlatformSongDistributionRange(double Min, double Max);
public sealed record PlatformSongDistributionBin(double Lower, double Upper, long Count);
public sealed record PlatformSongDistributionQuery
{
public string Metric { get; init; } = "achievement";
public double? Min { get; init; }
public double? Max { get; init; }
public int Bins { get; init; } = 24;
public string? SnapshotId { get; init; }
}
public sealed record PlatformSongDistribution(
string ChartKey, string Metric, string Unit, bool Applicable, string SnapshotId,
string AsOf, string ExpiresAt, PlatformSongDistributionRange Domain,
PlatformSongDistributionRange Range, PlatformSongDistributionBin[] Bins,
long SampleCount, long VisibleSampleCount, long NoRecordCount, long NoEligibleRecordCount,
long UnavailableCount, long UnderflowCount, long OverflowCount, double? Mean, double? SelfValue,
string SchemaVersion, string CatalogVersion, string ScoreVersion,
PlatformSongSourceCoverage SourceCoverage);
public sealed record PlatformSongExplorerError(string Error, string? Detail = null);

public sealed record PlatformSongExplorerRebuildResult(
bool DryRun, int Scanned, int Eligible, int Ineligible,
IReadOnlyDictionary<string, int> Skipped, string NextAfterResultId, bool HasMore, string[] Warnings);
57 changes: 57 additions & 0 deletions tests/Sirius.Protocol.Tests/SongExplorerContractTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
using System.Text.Json;
using Sirius.Protocol.InternalApi;
using Xunit;

namespace Sirius.Protocol.Tests;

public sealed class SongExplorerContractTests
{
private static readonly JsonSerializerOptions Json = new(JsonSerializerDefaults.Web);

private const string LegacySong = """
{"id":"1","title":"Song","pronunciation":"song","composer":"Composer",
"lyricist":"","arranger":"","singer":"Singer","coverUrl":"https://cdn.example/cover.webp",
"coverType":"original","isLongVersion":false,"durationSeconds":120,"releasedAt":null,
"knownTimesCompletedSum":0,"countCoverage":"local","charts":[],"matchedChartKeys":[]}
""";

[Theory]
[InlineData("")]
[InlineData(",\"coverFallbackUrl\":null")]
public void Legacy_or_null_fallback_keeps_the_primary_cover(string additionalField)
{
var payload = LegacySong.TrimEnd().TrimEnd('}') + additionalField + "}";
var song = JsonSerializer.Deserialize<PlatformSongSummary>(payload, Json)!;

Assert.Equal("https://cdn.example/cover.webp", song.CoverUrl);
Assert.Null(song.CoverFallbackUrl);
Assert.Null(song.ReleasedAt);
Assert.Equal(0, song.KnownTimesCompletedSum);
}

[Fact]
public void Pinned_fallback_round_trips_in_page_and_detail_web_json()
{
const string fallback = "https://raw.githubusercontent.com/example/covers/0123456789abcdef0123456789abcdef01234567/cover.webp";
var coverage = new PlatformSongSourceCoverage("available", "unavailable", "local", []);
var chart = new PlatformSongChartSummary("normal:1", "normal", "1", "1", 1, "Normal",
"1", null, "achievement", null, new("none", null, null, 0, null, coverage));
var song = new PlatformSongSummary("1", "Song", "song", "Composer", "", "", "Singer",
"https://cdn.example/cover.webp", "original", false, 120, null, 0, "local", [chart], [])
{
CoverFallbackUrl = fallback
};
var page = new PlatformSongExplorerPage([song], null, 1, "1", "catalog-1", "score-1", coverage);
var detail = new PlatformSongChartDetail(song, chart, "unavailable", null, null, null,
"1", "catalog-1", "score-1");
var pageJson = JsonSerializer.Serialize(page, Json);
var detailJson = JsonSerializer.Serialize(detail, Json);

using var document = JsonDocument.Parse(pageJson);
var encodedSong = document.RootElement.GetProperty("items")[0];
Assert.Equal(fallback, encodedSong.GetProperty("coverFallbackUrl").GetString());
Assert.False(encodedSong.TryGetProperty("CoverFallbackUrl", out _));
Assert.Equal(fallback, JsonSerializer.Deserialize<PlatformSongExplorerPage>(pageJson, Json)!.Items[0].CoverFallbackUrl);
Assert.Equal(fallback, JsonSerializer.Deserialize<PlatformSongChartDetail>(detailJson, Json)!.Song.CoverFallbackUrl);
}
}
Loading